feat: add ue_scene websocket integration

Reviewed-on: #8

.claude/commands/cleanup.md

@@ -12,19 +12,6 @@ allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
 
 若 `$ARGUMENTS` 非空，则只检查指定文件/目录；否则检查所有未提交修改（`git diff HEAD`）。
 
-## 节省上下文规则
-
-优先用确定性的 CLI 检查缩小范围，不要一上来把完整文件或大 diff 读入上下文：
-
-```bash
-git diff --name-only HEAD
-git diff --unified=0 HEAD -- <path>
-git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
-```
-
-只有 focused diff 不足以安全判断或修改时，才读取完整文件。
-
 ## 审查清单
 
 按优先级检查以下问题（只报告在本次 diff 中**新增或修改**的代码里存在的问题）：
@@ -72,13 +59,8 @@ git diff HEAD --name-only
 
 ### Step 2 — 逐文件阅读并分析
 
-先从 focused diff 开始：
-
-```bash
-git diff --unified=0 HEAD -- <file>
-```
-
-用 `rg`、`git diff --check`、编译器或 linter 输出确认确定性问题。只有需要上下文时才用 Read 读取完整文件。对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式。
+- 用 Read 工具读取完整文件（不只读 diff）
+- 对照审查清单，记录每个问题：文件名、行号、问题类型、建议修复方式
 
 ### Step 3 — 报告问题清单
 
@@ -113,7 +95,6 @@ git diff --unified=0 HEAD -- <file>
 - 只改在审查清单中发现的问题，不做额外优化
 - 每次 Edit 只修改确实有问题的行，保持 diff 最小
 - 改完后用 `grep` 验证旧的坏代码已消失
-- 优先做精确补丁；只有仓库已有对应格式化流程时，才运行格式化工具
 
 ### Step 5 — 输出总结
 

.claude/commands/docs.md

@@ -1,93 +0,0 @@
----
-description: Create or update repository documentation from current code changes
-argument-hint: Optional: topic to document, or leave empty to infer from git diff
-allowed-tools: ["Read", "Edit", "Write", "Bash", "Glob", "Grep"]
----
-
-# /docs — Documentation Workflow
-
-## Goal
-
-Create or update documentation that explains why a change exists, how it behaves, and what maintainers need to know. Keep this command generic. Repository-specific coverage rules live in the repository and must be loaded separately.
-
-## Repository Rules
-
-Before deciding scope, check whether the repository has a documentation rules file:
-
-```bash
-test -f docs/documentation-coverage-rules.md && sed -n '1,240p' docs/documentation-coverage-rules.md
-```
-
-If it exists, apply it as the project-specific coverage checklist. If it does not exist, continue with the generic workflow below.
-
-## Workflow
-
-### Step 1 — Understand The Change
-
-```bash
-git diff HEAD --stat
-git diff HEAD --name-only
-git log --oneline -10
-rg --files docs
-```
-
-If `$ARGUMENTS` specifies a topic, focus on that topic. Otherwise infer the documentation topic from the changed files. Do not read the full repository diff by default; inspect focused files only:
-
-```bash
-git diff HEAD -- <path>
-rg -n "class |def |function |export |router|@router|interface |type " <path>
-```
-
-### Step 2 — Decide Scope
-
-- Prefer updating an existing relevant document over creating a duplicate.
-- Use one document for one coherent topic.
-- Split documents only when the change crosses meaningful domains.
-- Keep filenames lowercase and hyphenated.
-- Apply the repository-specific rules file before writing.
-
-For ambiguous or large documentation changes, briefly state the intended doc plan before editing. For clear small changes, proceed directly.
-
-### Step 3 — Write
-
-Explain:
-
-- Background/problem: what was wrong or missing before.
-- Core design decisions and rationale.
-- Operational or user-facing impact.
-- Relevant code paths, only when useful for future maintainers.
-
-Style:
-
-- Follow the repository’s existing language and heading conventions.
-- Use fenced code blocks with language tags.
-- Prefer tables for comparisons or parameter lists.
-- Keep snippets concise and relevant.
-
-### Step 4 — Verify
-
-- Read the completed docs once for clarity and stale statements.
-- Verify referenced paths exist with `test -e` or `rg --files`.
-- Run applicable checks from `docs/documentation-coverage-rules.md`.
-- Check Markdown links use readable user-facing titles unless repository rules allow otherwise.
-
-### Step 5 — Report
-
-Summarize changed docs and verification:
-
-```md
-Updated:
-- path/to/doc.md — what changed
-
-Verified:
-- checks that passed
-- checks that could not be run, if any
-```
-
-## Hard Constraints
-
-- Do not leave placeholder docs.
-- Do not duplicate bilingual files byte-for-byte.
-- Do not reference PR numbers, issue numbers, or the current conversation unless explicitly requested.
-- Do not write changelog-style lists without the reasoning and tradeoffs behind the change.
-- Keep docs maintainable and concise.

.claude/commands/goal-driven.md

@@ -1,93 +0,0 @@
----
-description: 用 goal-driven 方法推动一个复杂任务持续执行，直到明确成功标准被满足
-argument-hint: 建议填写任务目标；若同时给出成功标准更好
-allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
----
-
-# /goal-driven — 目标驱动执行模式
-
-使用 `lidangzzz/goal-driven` 的核心思想来推进复杂任务：先固定目标与成功标准，再持续执行和反复验收，直到标准真正满足。
-
-适用场景：
-
-- 长周期实现任务
-- 高复杂度工程任务
-- 可被明确验收的研究、实现、迁移、验证类工作
-
-不适用场景：
-
-- 纯脑暴
-- 无法定义成功标准的模糊任务
-- 很小的一次性修改
-
-## 输入要求
-
-若 `$ARGUMENTS` 只包含目标，没有成功标准，先补全一版可执行的成功标准再开始。
-
-启动时先输出：
-
-```md
-Goal
-- ...
-
-Criteria for success
-- ...
-
-Plan
-1. ...
-2. ...
-3. ...
-
-Verification
-- ...
-```
-
-## 执行规则
-
-1. 先把任务固化为两个核心块：
-   - `Goal`
-   - `Criteria for success`
-
-2. 成功标准必须尽量客观，可验证，可落地。
-   优先写成：
-   - 需要交付什么
-   - 需要通过哪些测试或验证
-   - 如何判断结果真的完成
-
-3. 进入持续执行循环：
-   - 完成一个阶段
-   - 检查当前结果是否满足成功标准
-   - 若未满足，明确剩余差距并继续推进
-
-4. 任何“完成了”“差不多了”“已实现”之类的结论，都必须经过验证，不能直接接受。
-
-5. 如果验证失败：
-   - 明确指出哪条成功标准没满足
-   - 继续工作，不要把阶段性进展误判为完成
-
-6. 只有在以下情况之一才能停止：
-   - 成功标准已满足
-   - 用户明确要求停止
-
-## 执行风格
-
-- 重证据，轻口头判断
-- 优先使用确定性工具证据：`rg`、`git diff --stat`、`git diff -- <path>`、测试、构建、lint、`curl`、数据库查询等能直接证明成功标准的方式
-- 不把大段命令输出粘进回复；保留在工具调用里，回复只总结关键证据
-- 重验收，轻自我感觉
-- 优先用测试、日志、产物、对比结果来证明完成
-- 对长期任务保持“未达标就继续”的节奏
-
-## 简版模板
-
-```md
-Goal: [[[[[在此填写最终目标]]]]]
-
-Criteria for success: [[[[[在此填写成功标准]]]]]
-
-循环执行：
-1. 推进任务
-2. 检查是否满足成功标准
-3. 若未满足，继续工作
-4. 直到满足标准或用户明确停止
-```

.claude/commands/release.md

@@ -28,19 +28,6 @@ allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
 - `docs/CHANGELOG.md`
 - `docs/version-history.md`
 
-## 节省上下文规则
-
-发版判断应以确定性 CLI 证据为主，优先使用紧凑命令和定点读取：
-
-```bash
-git status --short
-git diff --stat HEAD
-git diff --name-only HEAD
-rg -n "version|^## |^Released:|当前开发版本|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
-```
-
-除非需要判断某个代码变更是否属于本次发版，否则不要读取完整 diff。
-
 ## 执行步骤
 
 ### Step 1 — 环境检查
@@ -58,7 +45,7 @@ cat VERSION                        # 读取当前版本
 ### Step 2 — 确定发版类型与新版本号
 
 - 若 `$ARGUMENTS` 提供了明确类型（`feature` / `bugfix`），直接使用
-- 否则根据 `git diff --stat HEAD`、`git diff --name-only HEAD`、必要的 focused diff 和 `git log` 推断
+- 否则根据当前 `git diff HEAD` 和 `git log` 推断
 - 计算新版本号（例：`0.26.2` → bugfix → `0.26.3`）
 - **先输出发版计划供用户确认**：
 
@@ -104,13 +91,12 @@ cat VERSION                        # 读取当前版本
 
 针对本次变更范围做最小验证：
 
-- Python 文件有修改：先用 `git diff --name-only HEAD -- '*.py'` 列出，再运行 `python3 -m py_compile <changed_files>`
-- Frontend 文件有修改：先用 `git diff --name-only HEAD -- frontend` 判断范围，再运行项目标准检查（若无则跳过并说明）
+- Python 文件有修改：`python3 -m py_compile <changed_files>`
+- Frontend 文件有修改：运行项目标准检查（若无则跳过并说明）
 - 版本号一致性检查：用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
 
 ```bash
-cat VERSION
-rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
+grep -h "version" VERSION frontend/package.json pyproject.toml
 ```
 
 ### Step 7 — 提交前预览

.codex/config.toml

@@ -1,3 +0,0 @@
-approval_policy = "never"
-
-sandbox_mode = "danger-full-access"

.codex/skills/cleanup/SKILL.md

@@ -21,19 +21,6 @@ If the user specifies a file or directory, check only that. Otherwise check all
 
 Only report issues present in **newly added or modified** lines of this diff — do not audit unchanged code.
 
-## Token-Saving Rule
-
-Prefer deterministic CLI checks before reading files into model context:
-
-```bash
-git diff --name-only HEAD
-git diff --unified=0 HEAD -- <path>
-git diff --check
-rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
-```
-
-Read full files only when the focused diff does not provide enough surrounding context to make a safe edit.
-
 ## Checklist
 
 ### 1. Duplicate Logic
@@ -77,13 +64,7 @@ Filter to the user-specified path if one was provided.
 
 ### Step 2 — Read and analyze each file
 
-Start with focused diffs:
-
-```bash
-git diff --unified=0 HEAD -- <file>
-```
-
-Use `rg`, `git diff --check`, and compiler/linter output for deterministic findings. Read the full file only for files that need surrounding context. For each issue found, record filename, line number, category, and suggested fix.
+Read the full file (not just the diff) with the Read tool. For each file, record every issue found: filename, line number, category, and suggested fix.
 
 ### Step 3 — Report findings before touching anything
 
@@ -118,7 +99,6 @@ Principles:
 - Only fix issues identified in the checklist — no extra improvements
 - Keep each Edit as small as possible
 - After fixing, verify the old bad pattern is gone with grep
-- Prefer `apply_patch` for targeted edits; use formatters only when the repository already uses them for the touched file type
 
 ### Step 5 — Summary
 

.codex/skills/docs/SKILL.md

@@ -1,82 +0,0 @@
----
-name: docs
-description: Create or update repository documentation from current code changes. Use when the user asks to write docs, update docs, summarize implementation changes into docs, or check documentation coverage. Load repository-specific coverage rules from docs/documentation-coverage-rules.md when present.
----
-
-# Docs
-
-Use this skill when the task is documentation work: creating, updating, checking, or summarizing docs for code or behavior changes.
-
-## Goal
-
-Write documentation that explains why a change exists, how it behaves, and what maintainers need to know. Keep the skill generic; repository-specific rules belong in the repository, not in this skill.
-
-## Repository Rules
-
-Before deciding scope, check whether the repository has a documentation rules file:
-
-```bash
-test -f docs/documentation-coverage-rules.md && sed -n '1,240p' docs/documentation-coverage-rules.md
-```
-
-If it exists, apply it as the project-specific coverage checklist. If it does not exist, continue with the generic workflow below.
-
-## Workflow
-
-1. Gather focused context:
-
-```bash
-git diff HEAD --stat
-git diff HEAD --name-only
-git log --oneline -10
-rg --files docs
-```
-
-If the user gives a topic, focus on that topic. Otherwise infer the doc topic from changed files. Avoid reading large full diffs by default; inspect focused files and symbols:
-
-```bash
-git diff HEAD -- <path>
-rg -n "class |def |function |export |router|@router|interface |type " <path>
-```
-
-2. Decide scope:
-
-- Prefer updating an existing relevant doc over creating a duplicate.
-- Use one document for one coherent topic.
-- Split documents only when changes cross meaningful domains.
-- Keep filenames lowercase and hyphenated.
-
-3. Write the doc:
-
-- Explain background/problem, design decisions, constraints, and operational impact.
-- Keep code snippets short and directly relevant.
-- List related files only when they help future maintainers navigate.
-- Use the repository’s existing language, heading style, and naming conventions.
-
-4. Verify:
-
-- Read the completed doc once for clarity and stale statements.
-- Verify important referenced paths exist with `test -e` or `rg --files`.
-- Run repository-specific doc checks from `docs/documentation-coverage-rules.md` when present.
-- For Markdown links, check that user-facing titles are readable and not raw filenames unless the repository rules allow it.
-
-## Hard Constraints
-
-- Do not leave placeholder docs or copied source text pretending to be documentation.
-- Do not duplicate bilingual files byte-for-byte.
-- Do not reference PR numbers, issue numbers, or the current conversation unless explicitly requested.
-- Do not write changelog-style lists without the reasoning, constraints, and tradeoffs behind the change.
-- Keep docs concise enough to maintain.
-
-## Recommended Output
-
-After editing, summarize:
-
-```md
-Updated:
-- path/to/doc.md — what changed
-
-Verified:
-- checks that passed
-- checks that could not be run, if any
-```

.codex/skills/goal-driven/SKILL.md

@@ -1,103 +0,0 @@
----
-name: goal-driven
-description: Run a goal-driven execution loop for very large, long-horizon, rigorously verifiable tasks. Use when the user explicitly wants the lidangzzz/goal-driven method, a master-agent plus worker-agent style workflow, or a persistent loop that keeps working until concrete success criteria are satisfied.
----
-
-# Goal-Driven
-
-Use this skill when the user wants a strict goal-driven workflow for a hard task with:
-
-- one clear end goal
-- explicit success criteria
-- repeated verification against those criteria
-- continued execution until the criteria are actually met
-
-This skill is adapted from `lidangzzz/goal-driven`, but trimmed for local skill use to avoid bloating context.
-
-## When To Use
-
-Use it for tasks like:
-
-- compilers, interpreters, theorem-like proof work, deep refactors
-- long-running system design or implementation work
-- problems that are expensive and complex, but still objectively testable
-
-Do not use it for:
-
-- vague brainstorming without a success condition
-- short one-shot edits
-- tasks where "done" cannot be evaluated in a meaningful way
-
-## Core Model
-
-The workflow has two roles:
-
-1. Master role
-   Defines the goal, defines the success criteria, audits progress, and decides whether the work is actually complete.
-
-2. Worker role
-   Keeps advancing the task toward the goal. If a result is partial, stalled, or unverifiable, the worker continues.
-
-In Codex, only use actual subagents when the user explicitly asks for delegation or subagent work and the platform supports it. Otherwise emulate the same loop locally: keep working, checkpointing, and re-verifying until the criteria are satisfied.
-
-## Workflow
-
-1. Normalize the task into two blocks:
-   - `Goal`
-   - `Criteria for success`
-
-2. Make the criteria concrete and testable.
-   Good criteria usually include:
-   - required outputs
-   - required validations or tests
-   - edge cases or coverage thresholds
-   - what evidence proves completion
-
-3. Break the work into milestones that can each produce evidence.
-
-4. Execute the next milestone.
-   If subagents are explicitly allowed, the master may delegate bounded worker tasks.
-   If not, do the work locally but keep the master/worker mindset.
-
-5. Whenever work pauses, stalls, or appears complete, audit against the criteria directly.
-   Check artifacts, tests, logs, diffs, metrics, or other real evidence.
-
-6. If the criteria are not met, continue with a specific delta:
-   - what is still missing
-   - what evidence failed
-   - what the next worker pass must improve
-
-7. Stop only when the criteria are met, or when the user explicitly stops the process.
-
-## Operating Rules
-
-- Prefer objective checks over self-reported completion.
-- Prefer deterministic tool evidence over long model summaries: use `rg`, `git diff --stat`, targeted `git diff -- <path>`, tests, builds, linters, `curl`, or database queries when they can prove a criterion.
-- Do not paste large command output into the conversation; summarize the evidence and keep raw output in tool calls.
-- Do not confuse progress with completion.
-- If the worker says "done", verify it.
-- If verification fails, continue from the gap instead of restarting blindly.
-- Keep the goal stable unless the user changes it.
-- Tighten fuzzy criteria before sinking large amounts of effort.
-
-## Recommended Response Shape
-
-When starting a goal-driven task, structure the kickoff like this:
-
-```md
-Goal
-- ...
-
-Criteria for success
-- ...
-
-Current plan
-1. ...
-2. ...
-3. ...
-
-Verification
-- What evidence will prove completion
-```
-
-For a reusable prompt template, read [references/prompt-template.md](references/prompt-template.md).

.codex/skills/goal-driven/agents/openai.yaml

@@ -1,7 +0,0 @@
-interface:
-  display_name: "Goal-Driven"
-  short_description: "Drive complex work until explicit success criteria are met."
-  default_prompt: "Use $goal-driven to turn this task into a concrete goal, explicit success criteria, and a verification-driven execution loop."
-
-policy:
-  allow_implicit_invocation: true

.codex/skills/goal-driven/references/prompt-template.md

@@ -1,38 +0,0 @@
-# Goal-Driven Prompt Template
-
-Use this when you want a reusable kickoff prompt for a master/worker execution loop.
-
-```md
-# Goal-Driven System
-
-Goal: [[[[[DEFINE THE FINAL GOAL HERE]]]]]
-
-Criteria for success: [[[[[DEFINE THE SUCCESS CRITERIA HERE]]]]]
-
-You are the master agent.
-
-Your job is to:
-1. Keep the goal and criteria fixed.
-2. Start worker execution toward the goal.
-3. Audit any claimed progress against the criteria.
-4. If the criteria are not met, continue the work with a precise next delta.
-5. Stop only when the criteria are satisfied or the user explicitly stops the process.
-
-Worker requirements:
-1. Break the task into subproblems.
-2. Keep producing concrete progress toward the goal.
-3. Report evidence, not just claims.
-4. Continue until the criteria are satisfied.
-
-Master audit loop:
-1. Check whether the worker is still making progress.
-2. If the worker stalls or claims completion, verify against the criteria.
-3. If verification fails, resume work from the remaining gap.
-4. Repeat until the criteria are met.
-```
-
-## Notes
-
-- Stronger criteria produce better results than stronger rhetoric.
-- Prefer measurable checks such as tests, parity checks, generated artifacts, benchmarks, or reviewable outputs.
-- If the environment does not support subagents, emulate the same loop locally.

.codex/skills/release/SKILL.md

@@ -18,7 +18,7 @@ Do not use this skill for ordinary commits that are not being released.
 
 ## Versioning Rules
 
-- `feature` -> bump minor and reset patch to `0` (`x.y.z` → `x.(y+1).0`; for example `0.41.2` → `0.42.0`)
+- `feature` -> bump `+0.1.0`
 - `bugfix` -> bump `+0.0.1`
 - `docs`, `maintenance`, and `refactor` do not bump by default unless the user explicitly wants a release
 
@@ -35,19 +35,6 @@ Use `git rev-parse --show-toplevel` to get the repo root. All paths are relative
 - `docs/CHANGELOG.md`
 - `docs/version-history.md`
 
-## Token-Saving Rule
-
-Release work should be driven by deterministic CLI evidence. Prefer compact commands and targeted file reads:
-
-```bash
-git status --short
-git diff --stat HEAD
-git diff --name-only HEAD
-rg -n "version|^## |^Released:|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
-```
-
-Do not inspect full diffs unless deciding whether changed code belongs in the release.
-
 ## Workflow
 
 ### Step 1 — Environment check
@@ -65,10 +52,8 @@ If unrelated uncommitted changes exist, list them and ask the user whether to in
 ### Step 2 — Determine release type and next version
 
 - If the user provided an explicit type (`feature` / `bugfix`), use it
-- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`
-- Compute the next version:
-  - `feature`: increment minor and reset patch to `0` (e.g. `0.41.2` → `0.42.0`)
-  - `bugfix`: increment patch only (e.g. `0.26.2` → `0.26.3`)
+- Otherwise infer from `git diff HEAD` and recent `git log`
+- Compute the next version (e.g. `0.26.2` → bugfix → `0.26.3`)
 - **Show the release plan before making any changes:**
 
 ```
@@ -119,13 +104,12 @@ Get today's date with `date +%Y-%m-%d`.
 
 Run the smallest relevant validation for the changes in scope:
 
-- Python files changed: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `python3 -m py_compile <changed_files>`
-- Frontend files changed: list changed frontend files with `git diff --name-only HEAD -- frontend`, then run the project-standard check if available; otherwise skip and say so
+- Python files changed: `python3 -m py_compile <changed_files>`
+- Frontend files changed: run the project-standard check if available; otherwise skip and say so
 - Version consistency: confirm VERSION, package.json, pyproject.toml, and uv.lock all show the same version
 
 ```bash
-cat VERSION
-rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
+grep -h "version" VERSION frontend/package.json pyproject.toml
 ```
 
 ### Step 7 — Pre-commit preview

.dockerignore

@@ -1,13 +0,0 @@
-**
-
-!pyproject.toml
-!uv.lock
-!aiprovider/
-!aiprovider/**
-
-aiprovider/.env
-aiprovider/.env.*
-!aiprovider/.env.example
-**/__pycache__/
-**/*.pyc
-**/*.pyo

README.md

@@ -227,120 +227,6 @@ bun run build
 
 启动服务后访问: `http://localhost:8000/docs`
 
-## WSL / Windows 局域网访问
-
-如果服务运行在 WSL 中，而你希望：
-
-- Windows 本机浏览器访问开发服务
-- 同一局域网内的手机或其他电脑访问开发服务
-
-推荐按下面顺序排查和配置。
-
-### 1. 在 WSL 中启动服务
-
-```bash
-./planet.sh start --allow-lan
-```
-
-这会让前端监听 `0.0.0.0:3000`，后端监听 `0.0.0.0:8000`。
-
-### 2. 先确认 WSL 内部服务正常
-
-在 WSL 中执行：
-
-```bash
-curl http://localhost:3000
-curl http://localhost:8000/health
-ss -ltnp | grep -E ':3000|:8000'
-```
-
-预期：
-
-- `3000` 返回前端 HTML
-- `8000/health` 返回健康检查 JSON
-- `ss` 中能看到 `0.0.0.0:3000` 和 `0.0.0.0:8000`
-
-如果这一步不通，先不要继续做 Windows 转发。
-
-### 3. 在 Windows 本机验证 localhost 直通
-
-在 Windows PowerShell 中执行：
-
-```powershell
-curl http://localhost:3000
-curl http://localhost:8000/health
-```
-
-在常见的 WSL2 开发环境下，Windows 通常可以直接通过 `localhost` 访问 WSL 中的服务。
-
-### 4. 如果需要让局域网设备访问，再做 Windows 端口转发
-
-注意：下面的命令必须在“以管理员身份运行”的 PowerShell 中执行。
-
-先把 Windows 对外网卡上的 `3000` / `8000` 转发到 Windows 本机 `127.0.0.1`：
-
-```powershell
-netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
-netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=8000
-
-netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 listenport=3000 connectaddress=127.0.0.1 connectport=3000
-netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 listenport=8000 connectaddress=127.0.0.1 connectport=8000
-```
-
-再放行 Windows 防火墙：
-
-```powershell
-New-NetFirewallRule -DisplayName "WSL Planet 3000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 3000
-New-NetFirewallRule -DisplayName "WSL Planet 8000" -Direction Inbound -Action Allow -Protocol TCP -LocalPort 8000
-```
-
-检查转发规则是否生效：
-
-```powershell
-netsh interface portproxy show all
-```
-
-预期能看到：
-
-- `0.0.0.0:3000 -> 127.0.0.1:3000`
-- `0.0.0.0:8000 -> 127.0.0.1:8000`
-
-### 5. 查 Windows 局域网 IP，并让其他设备访问
-
-在 Windows PowerShell 中执行：
-
-```powershell
-ipconfig
-```
-
-找到当前联网网卡的 IPv4 地址，例如 `192.168.8.228`。
-
-局域网其他设备可访问：
-
-- `http://<Windows局域网IP>:3000/earth`
-- `http://<Windows局域网IP>:3000/admin`
-
-例如：
-
-- `http://192.168.8.228:3000/earth`
-
-### 6. 常见现象与判断
-
-- WSL 中 `curl localhost:3000` 能通，但 Windows 访问 `WSL 的局域网 IP:3000` 不通：这是正常现象之一，优先验证 Windows 的 `localhost:3000`
-- Windows `localhost:3000` 能通，但局域网设备访问 `Windows 局域网 IP:3000` 不通：通常缺少 `portproxy` 或防火墙放行
-- `whoami /groups` 中 `S-1-5-32-544` 显示 `deny only`：说明当前 PowerShell 不是提权管理员窗口
-
-### 7. 本项目一次性验证顺序
-
-建议固定按这个顺序验证：
-
-1. WSL 中执行 `curl http://localhost:3000`
-2. WSL 中执行 `curl http://localhost:8000/health`
-3. Windows 中执行 `curl http://localhost:3000`
-4. Windows 中执行 `curl http://localhost:8000/health`
-5. 管理员 PowerShell 配置 `portproxy` 和防火墙
-6. 用手机或其他电脑访问 `http://<Windows局域网IP>:3000/earth`
-
 ## 启动容错参数
 
 `planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试，并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。

TODO.md

@@ -22,19 +22,5 @@
 - [ ] 可选优化（非必做）：将 BGP incident/collector 标点改为 HTML marker（参考 worldmonitor 的 `htmlElementsData` 思路），实现近乎固定屏幕尺寸与更高密度可点击性
 - [ ] 保持 Earth 当前这批纯个人偏好设置继续走本地持久化：`旋转模式`、HUD 面板显示/隐藏、`地形透明度` 暂不升级到后端系统设置，避免把设备级偏好过早做成全局配置
 - [ ] 如果后续明确需要“账号级同步 Earth 偏好”，再单独设计 `Earth user preferences`：优先按用户维度而不是全局系统设置保存，并规划 `localStorage -> backend` 的平滑迁移策略
-- [ ] 为 Planet / Earth 补一个可用的日志查看系统：先明确前后端/AI Provider/采集任务的日志入口、最近日志聚合、筛选与 tail 能力，再决定是先做脚本级统一入口还是控制台内置日志面板
-- [ ] 重写控制台 UI，逐步抛弃 Ant Design，建立自有组件体系，并统一采用 `tabler.io` / Tabler Icons 作为控制台主图标库
 - [ ] 把 Earth 态势新闻源从 [earth_news.py](/home/ray/dev/linkong/planet/backend/app/services/earth_news.py) 的硬编码列表抽成可配置目录，优先保持当前“实时聚合”链路不变，只先解决新闻源不可配置的问题
 - [ ] 为 Earth 态势新闻设计后续采集器化方案：明确新闻数据模型、去重策略、区域映射、过期清理和 Earth/AI 复用方式，再决定何时把新闻从实时抓取升级成正式 collector
-- [ ] AIS v4：开放船只多源聚合策略配置，支持 source priority、字段级规则、freshness 窗口和高级保护开关；保存时校验未知字段、非法模式和危险动态字段锁定，并在聚合接口返回命中的配置版本
-- [ ] AIS v5：实现船舶资料 enrichment 与冲突治理，按 `mmsi + imo + name + callsign` 异步补充船型细分、AIS 大类、旗国、尺寸、建造年份、运营方和图片缓存；详情面板展示缓存资料和字段来源，不在实时 AIS 请求链路现场抓第三方页面
-- [ ] 为 Earth 地球表面增加一层与基础纹理对齐的材质/纹理 overlay，并在同层叠加国界轮廓参考线；要求国界线与底图稳定对齐，且 hover 到国家轮廓时能高亮当前国家，便于校准地表和增强交互
-- [ ] 把 Earth 新闻接入通用巡航队列：按新闻发生地和时间排序生成巡航目标，巡航聚焦到新闻事件时显示对应新闻卡片，并保持实现边界为“通用巡航层 + 新闻业务适配层”，不要再把新闻逻辑直接耦合回 `main.js` 状态机
-- [ ] 为未知位置的算力中心建立分层坐标补全链路：优先 `精确坐标 > 站点/园区命中 > 城市 > 州/省 > 国家内主要算力城市 > 国家质心`，并把每次回退的 `confidence / reason / precision` 明确写进统一 GeoJSON
-- [ ] 为算力中心补一份可维护的本地位置注册表，例如 `canonical_name / aliases / operator / country / region / city / lat / lon / confidence / source_note`，避免把地点知识长期硬编码在 `visualization.py`
-- [ ] 增强 `epoch_ai_gpu` 和相关算力采集器的源页面解析：即使公开 API 不给坐标，也继续尝试从详情页、HTML、内嵌 JSON、schema.org、OpenGraph、脚本变量和 PDF/新闻稿链接里抽地点线索
-- [ ] 为未知位置算力中心增加外部富化策略评估：可选接入公开知识源或搜索兜底，只抓“站点名/园区名/城市名”级别线索，不直接抓经纬度结论，并把结果作为候选证据而不是真值
-- [ ] 为算力中心建立 `operator / cluster name / facility alias` 归一化层，先解决 `xAI / Colossus / Memphis`、`OpenAI / Stargate`、`CoreWeave`、`Lambda`、`Crusoe` 这类同一对象多种写法导致的地点匹配失败
-- [ ] 为估算位置增加更细的视觉和产品表达：除了问号角标，还要支持 tooltip/详情中的“估算依据”“精度级别”“最后核验时间”，并允许在设置中单独开关“仅看精确位置”
-- [ ] 为国家级估算点设计更合理的落点策略：优先落在“该国主要算力/数据中心城市候选集”而不是几何质心，必要时同国多节点做稳定散列分配，避免大量节点堆在荒漠或海上
-- [ ] 为未知位置算力中心建立人工校验工作流：支持导出待核验清单、记录人工确认结果，并把人工确认反哺到位置注册表，逐步减少问号点比例

VERSION

@@ -1 +1 @@
-0.47.0
+0.33.0

aiprovider/Dockerfile

@@ -1,12 +1,6 @@
-# syntax=docker/dockerfile:1.7
+FROM python:3.14-slim
 
-ARG PYTHON_IMAGE=python:3.14-slim
-ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
-
-FROM ${UV_IMAGE} AS uv
-FROM ${PYTHON_IMAGE}
-
-COPY --from=uv /uv /uvx /bin/
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
 
 WORKDIR /app
 
@@ -20,10 +14,9 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
     && rm -rf /var/lib/apt/lists/*
 
 COPY pyproject.toml uv.lock /app/
-RUN --mount=type=cache,target=/root/.cache/uv \
-    uv sync --frozen --no-dev
+RUN uv sync --frozen --no-dev
 
-COPY aiprovider /app/aiprovider
+COPY . /app
 
 EXPOSE 8010
 

aiprovider/main.py

@@ -37,26 +37,8 @@ def verify_service_token(x_provider_token: str | None = Header(default=None)) ->
         )
 
 
-def get_provider_service(
-    x_ai_provider: str | None = Header(default=None),
-    x_ai_provider_api: str | None = Header(default=None),
-    x_ai_base_url: str | None = Header(default=None),
-    x_ai_api_key: str | None = Header(default=None),
-    x_ai_model: str | None = Header(default=None),
-    x_ai_max_tokens: str | None = Header(default=None),
-    x_ai_anthropic_version: str | None = Header(default=None),
-) -> ProviderService:
-    overrides = {
-        "provider": x_ai_provider,
-        "provider_api": x_ai_provider_api,
-        "base_url": x_ai_base_url,
-        "api_key": x_ai_api_key,
-        "model": x_ai_model,
-        "anthropic_version": x_ai_anthropic_version,
-    }
-    if x_ai_max_tokens:
-        overrides["max_tokens"] = x_ai_max_tokens
-    return ProviderService({key: value for key, value in overrides.items() if value not in (None, "")})
+def get_provider_service() -> ProviderService:
+    return ProviderService()
 
 
 @app.get("/health")

aiprovider/provider_service.py

@@ -46,22 +46,19 @@ def _resolve_provider_api(provider: str, configured_api: str) -> str:
 
 
 class ProviderService:
-    def __init__(self, overrides: dict[str, Any] | None = None) -> None:
-        overrides = overrides or {}
-        self.provider = _normalize_provider(overrides.get("provider") or settings.AI_PROVIDER)
+    def __init__(self) -> None:
+        self.provider = _normalize_provider(settings.AI_PROVIDER)
         self.provider_api = _resolve_provider_api(
             self.provider,
-            _normalize_provider_api(overrides.get("provider_api") or settings.AI_PROVIDER_API),
+            _normalize_provider_api(settings.AI_PROVIDER_API),
         )
-        self.base_url = str(overrides.get("base_url") or settings.AI_BASE_URL).rstrip("/")
-        self.api_key = str(overrides.get("api_key") or settings.AI_API_KEY)
-        self.default_model = str(overrides.get("model") or settings.AI_MODEL)
+        self.base_url = settings.AI_BASE_URL.rstrip("/")
+        self.api_key = settings.AI_API_KEY
+        self.default_model = settings.AI_MODEL
         self.timeout = settings.AI_TIMEOUT_SECONDS
         self.http_retry_attempts = max(settings.AI_HTTP_RETRY_ATTEMPTS, 1)
-        self.max_tokens = int(overrides.get("max_tokens") or settings.AI_MAX_TOKENS)
-        self.anthropic_version = str(
-            overrides.get("anthropic_version") or settings.AI_ANTHROPIC_VERSION
-        )
+        self.max_tokens = settings.AI_MAX_TOKENS
+        self.anthropic_version = settings.AI_ANTHROPIC_VERSION
         self.system_prompt = settings.AI_ANALYSIS_SYSTEM_PROMPT
 
     def get_status(self) -> AIProviderStatusResponse:

backend/Dockerfile

@@ -1,10 +1,6 @@
-ARG PYTHON_IMAGE=python:3.14-slim
-ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
+FROM python:3.14-slim
 
-FROM ${UV_IMAGE} AS uv
-FROM ${PYTHON_IMAGE}
-
-COPY --from=uv /uv /uvx /bin/
+COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
 
 WORKDIR /app
 

backend/app/api/v1/datasource_config.py

@@ -1,41 +1,20 @@
 """DataSourceConfig API for user-defined data sources"""
 
-from typing import Any, Optional
+from typing import Optional
 from datetime import datetime
 import base64
-import json
-import re
-from fastapi import APIRouter, Depends, HTTPException
+from fastapi import APIRouter, Depends, HTTPException, status
 from sqlalchemy import select, func
 from sqlalchemy.ext.asyncio import AsyncSession
 from pydantic import BaseModel, Field
 import httpx
 
-from app.core.target_schema_registry import get_target_schema, list_target_schemas
-from app.core.datasource_defaults import DEFAULT_DATASOURCES
 from app.db.session import get_db
 from app.models.user import User
 from app.models.datasource_config import DataSourceConfig
-from app.models.datasource_mapping import DataSourceMappingTemplate
 from app.core.security import get_current_user
 from app.core.cache import cache
 from app.core.time import to_iso8601_utc
-from app.schemas.ai import SituationalAnalysisRequest
-from app.services.ai_client import AIProviderClient, get_ai_provider_client
-from app.services.datasource_mapping import (
-    MappingError,
-    build_heuristic_mapping,
-    execute_mapping,
-    persist_mapped_records,
-    redact_for_llm,
-    stable_payload_hash,
-)
-from app.services.datasource_connectivity import (
-    get_builtin_connection_status,
-    save_connectivity_success,
-    strip_connectivity_validation,
-    test_builtin_connectivity,
-)
 
 router = APIRouter()
 
@@ -80,70 +59,6 @@ class DataSourceConfigResponse(BaseModel):
         from_attributes = True
 
 
-def _is_builtin_config_name(name: str | None) -> bool:
-    return bool(name and name in DEFAULT_DATASOURCES)
-
-
-async def _ensure_builtin_connection_verified(
-    db: AsyncSession,
-    config_data: DataSourceConfigCreate,
-) -> None:
-    if not _is_builtin_config_name(config_data.name):
-        return
-
-    status_result = await get_builtin_connection_status(
-        db,
-        config_data.name,
-        config_data.endpoint,
-        config_data.auth_type,
-        config_data.headers,
-        config_data.config,
-    )
-    if not status_result.get("connected"):
-        raise HTTPException(
-            status_code=400,
-            detail=status_result.get("message") or "请先完成连接验证，再保存内置采集器配置。",
-        )
-
-
-class CustomSampleRequest(BaseModel):
-    datasource_config_id: Optional[int] = None
-    config: Optional[DataSourceConfigCreate] = None
-    limit_bytes: int = Field(default=200000, ge=1000, le=1000000)
-
-
-class MappingProposeRequest(BaseModel):
-    sample_payload: Any
-    target_schema: str
-    use_ai: bool = True
-
-
-class MappingPreviewRequest(BaseModel):
-    sample_payload: Any
-    target_schema: str
-    mapping_json: dict
-    limit: int = Field(default=20, ge=1, le=100)
-
-
-class MappingTemplateCreate(BaseModel):
-    datasource_config_id: int
-    target_schema: str
-    mapping_json: dict
-    sample_payload: Any | None = None
-    sample_payload_hash: Optional[str] = None
-    validation_status: str = Field(default="draft", pattern="^(draft|valid|invalid)$")
-    is_active: bool = False
-
-
-class MappingTemplateUpdate(BaseModel):
-    target_schema: Optional[str] = None
-    mapping_json: Optional[dict] = None
-    sample_payload: Any | None = None
-    sample_payload_hash: Optional[str] = None
-    validation_status: Optional[str] = Field(default=None, pattern="^(draft|valid|invalid)$")
-    is_active: Optional[bool] = None
-
-
 async def test_endpoint(
     endpoint: str,
     auth_type: str,
@@ -181,134 +96,6 @@ async def test_endpoint(
         }
 
 
-def _build_request_headers(auth_type: str, auth_config: dict, headers: dict) -> dict[str, str]:
-    request_headers = {str(key): str(value) for key, value in (headers or {}).items()}
-    auth_type = str(auth_type or "none").lower()
-    auth_config = auth_config or {}
-
-    if auth_type == "bearer" and auth_config.get("token"):
-        request_headers["Authorization"] = f"Bearer {auth_config['token']}"
-    elif auth_type == "api_key" and auth_config.get("api_key"):
-        location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
-        if location != "query":
-            key_name = auth_config.get("key_name", "X-API-Key")
-            request_headers[str(key_name)] = str(auth_config["api_key"])
-    elif auth_type == "basic":
-        username = auth_config.get("username", "")
-        password = auth_config.get("password", "")
-        credentials = f"{username}:{password}"
-        encoded = base64.b64encode(credentials.encode()).decode()
-        request_headers["Authorization"] = f"Basic {encoded}"
-    return request_headers
-
-
-def _build_query_params(auth_type: str, auth_config: dict, config: dict) -> dict[str, Any]:
-    params = {}
-    candidate = (config or {}).get("params") or (config or {}).get("query_params")
-    if isinstance(candidate, dict):
-        params.update(candidate)
-
-    auth_type = str(auth_type or "none").lower()
-    auth_config = auth_config or {}
-    if auth_type == "api_key" and auth_config.get("api_key"):
-        location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
-        if location == "query":
-            key_name = auth_config.get("key_name") or auth_config.get("param_name") or "api_key"
-            params[str(key_name)] = auth_config["api_key"]
-    return params
-
-
-async def fetch_custom_sample_from_config(config: DataSourceConfig, limit_bytes: int) -> Any:
-    request_config = config.config or {}
-    method = str(request_config.get("method") or request_config.get("request_method") or "GET").upper()
-    if method not in {"GET", "POST"}:
-        raise HTTPException(status_code=400, detail="Only GET and POST sample requests are supported.")
-
-    headers = _build_request_headers(config.auth_type, config.auth_config or {}, config.headers or {})
-    params = _build_query_params(config.auth_type, config.auth_config or {}, request_config)
-    timeout = float(request_config.get("timeout", 30))
-    json_body = request_config.get("json_body")
-    if json_body is None and str(request_config.get("body_type") or "").lower() in {"json", ""}:
-        candidate = request_config.get("body")
-        if isinstance(candidate, (dict, list)):
-            json_body = candidate
-
-    async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
-        response = await client.request(
-            method,
-            config.endpoint,
-            headers=headers,
-            params=params or None,
-            json=json_body,
-        )
-        response.raise_for_status()
-        content = response.content[:limit_bytes]
-        if "application/json" in response.headers.get("content-type", ""):
-            return json.loads(content.decode(response.encoding or "utf-8"))
-        return {"text": content.decode(response.encoding or "utf-8", errors="replace")}
-
-
-def _parse_mapping_from_ai_text(content: str) -> dict[str, Any] | None:
-    if not content:
-        return None
-
-    candidates = [content]
-    fenced = re.findall(r"```(?:json)?\s*(\{.*?\})\s*```", content, flags=re.DOTALL)
-    candidates = fenced + candidates
-    for candidate in candidates:
-        try:
-            parsed = json.loads(candidate)
-        except json.JSONDecodeError:
-            continue
-        if isinstance(parsed, dict) and isinstance(parsed.get("fields"), dict):
-            return parsed
-    return None
-
-
-async def _get_config_for_sample(
-    payload: CustomSampleRequest,
-    db: AsyncSession,
-) -> DataSourceConfig:
-    if payload.datasource_config_id is not None:
-        result = await db.execute(
-            select(DataSourceConfig).where(DataSourceConfig.id == payload.datasource_config_id)
-        )
-        config = result.scalar_one_or_none()
-        if not config:
-            raise HTTPException(status_code=404, detail="Configuration not found")
-        return config
-
-    if payload.config is None:
-        raise HTTPException(status_code=400, detail="datasource_config_id or config is required")
-
-    config_data = payload.config
-    return DataSourceConfig(
-        name=config_data.name,
-        description=config_data.description,
-        source_type=config_data.source_type,
-        endpoint=config_data.endpoint,
-        auth_type=config_data.auth_type,
-        auth_config=config_data.auth_config,
-        headers=config_data.headers,
-        config=config_data.config,
-    )
-
-
-def serialize_mapping_template(template: DataSourceMappingTemplate) -> dict[str, Any]:
-    return {
-        "id": template.id,
-        "datasource_config_id": template.datasource_config_id,
-        "target_schema": template.target_schema,
-        "mapping_json": template.mapping_json,
-        "sample_payload_hash": template.sample_payload_hash,
-        "validation_status": template.validation_status,
-        "version": template.version,
-        "is_active": template.is_active,
-        "created_at": to_iso8601_utc(template.created_at),
-        "updated_at": to_iso8601_utc(template.updated_at),
-    }
-
-
 @router.get("/configs")
 async def list_configs(
     active_only: bool = False,
@@ -318,7 +105,7 @@ async def list_configs(
     """List all user-defined data source configurations"""
     query = select(DataSourceConfig)
     if active_only:
-        query = query.where(DataSourceConfig.is_active)
+        query = query.where(DataSourceConfig.is_active == True)
     query = query.order_by(DataSourceConfig.created_at.desc())
 
     result = await db.execute(query)
@@ -345,52 +132,6 @@ async def list_configs(
     }
 
 
-@router.get("/configs/all")
-async def list_all_datasources(
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    """List all data sources: YAML defaults + DB overrides"""
-    from app.core.data_sources import COLLECTOR_URL_KEYS, get_data_sources_config
-
-    config = get_data_sources_config()
-
-    db_query = await db.execute(select(DataSourceConfig))
-    db_configs = {c.name: c for c in db_query.scalars().all()}
-
-    result = []
-    for name, yaml_key in COLLECTOR_URL_KEYS.items():
-        yaml_url = config.get_yaml_url(name)
-        db_config = db_configs.get(name)
-
-        result.append(
-            {
-                "name": name,
-                "default_url": yaml_url,
-                "endpoint": db_config.endpoint if db_config else yaml_url,
-                "is_overridden": db_config is not None and db_config.endpoint != yaml_url
-                if yaml_url
-                else db_config is not None,
-                "is_active": db_config.is_active if db_config else True,
-                "source_type": db_config.source_type if db_config else "http",
-                "auth_type": db_config.auth_type if db_config else "none",
-                "auth_configured": {
-                    "api_key": bool((db_config.auth_config or {}).get("api_key"))
-                    if db_config
-                    else False,
-                },
-                "headers": db_config.headers if db_config else {},
-                "config": strip_connectivity_validation(db_config.config if db_config else {}),
-                "config_id": db_config.id if db_config else None,
-                "description": db_config.description
-                if db_config
-                else f"Data source from YAML: {yaml_key}",
-            }
-        )
-
-    return {"total": len(result), "data": result}
-
-
 @router.get("/configs/{config_id}")
 async def get_config(
     config_id: int,
@@ -435,7 +176,7 @@ async def create_config(
         auth_type=config_data.auth_type,
         auth_config=config_data.auth_config,
         headers=config_data.headers,
-        config=strip_connectivity_validation(config_data.config),
+        config=config_data.config,
     )
 
     db.add(config)
@@ -467,10 +208,6 @@ async def update_config(
 
     update_data = config_data.model_dump(exclude_unset=True)
     for field, value in update_data.items():
-        if field == "config":
-            value = strip_connectivity_validation(value)
-        if field == "auth_config" and value == {} and (config.auth_config or {}):
-            continue
         setattr(config, field, value)
 
     await db.commit()
@@ -573,367 +310,38 @@ async def test_new_config(
         }
 
 
-@router.post("/configs/builtin/connection-status")
-async def get_builtin_config_connection_status(
-    config_data: DataSourceConfigCreate,
+@router.get("/configs/all")
+async def list_all_datasources(
     current_user: User = Depends(get_current_user),
     db: AsyncSession = Depends(get_db),
 ):
-    if not _is_builtin_config_name(config_data.name):
-        raise HTTPException(status_code=400, detail="Only built-in datasource configs are supported.")
+    """List all data sources: YAML defaults + DB overrides"""
+    from app.core.data_sources import COLLECTOR_URL_KEYS, get_data_sources_config
 
-    return await get_builtin_connection_status(
-        db,
-        config_data.name,
-        config_data.endpoint,
-        config_data.auth_type,
-        config_data.headers,
-        config_data.config,
-    )
+    config = get_data_sources_config()
 
+    db_query = await db.execute(select(DataSourceConfig))
+    db_configs = {c.name: c for c in db_query.scalars().all()}
 
-@router.post("/configs/builtin/connect")
-async def connect_builtin_config(
-    config_data: DataSourceConfigCreate,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    if not _is_builtin_config_name(config_data.name):
-        raise HTTPException(status_code=400, detail="Only built-in datasource configs are supported.")
+    result = []
+    for name, yaml_key in COLLECTOR_URL_KEYS.items():
+        yaml_url = config.get_yaml_url(name)
+        db_config = db_configs.get(name)
 
-    result = await test_builtin_connectivity(
-        config_data.name,
-        config_data.endpoint,
-        config_data.auth_type,
-        config_data.headers,
-        config_data.config,
-        db,
-        config_data.auth_config,
-    )
-    if result.get("success") and result.get("checksum"):
-        validation = await save_connectivity_success(
-            db,
-            config_data.name,
-            result["checksum"],
-            result,
-            connected_by="connection_button",
-        )
-        await db.commit()
-        return {
-            **result,
-            "connected": True,
-            "validation": validation,
-        }
-
-    return {
-        **result,
-        "connected": False,
-    }
-
-
-@router.post("/custom/sample")
-async def fetch_custom_sample(
-    payload: CustomSampleRequest,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    """Fetch a sample payload for a saved or draft custom data source."""
-    config = await _get_config_for_sample(payload, db)
-    try:
-        sample = await fetch_custom_sample_from_config(config, payload.limit_bytes)
-    except httpx.HTTPStatusError as exc:
-        raise HTTPException(
-            status_code=exc.response.status_code,
-            detail=f"Sample request failed: HTTP {exc.response.status_code}",
-        ) from exc
-    except httpx.HTTPError as exc:
-        raise HTTPException(status_code=502, detail=f"Sample request failed: {exc}") from exc
-
-    return {
-        "success": True,
-        "sample_payload": sample,
-        "sample_payload_hash": stable_payload_hash(sample),
-        "redacted_preview": redact_for_llm(sample),
-    }
-
-
-@router.get("/target-schemas")
-async def get_datasource_target_schemas(
-    current_user: User = Depends(get_current_user),
-):
-    """List target schemas available for custom datasource mapping."""
-    return {"data": list_target_schemas()}
-
-
-@router.post("/mappings/propose")
-async def propose_datasource_mapping(
-    payload: MappingProposeRequest,
-    current_user: User = Depends(get_current_user),
-    ai_client: AIProviderClient = Depends(get_ai_provider_client),
-):
-    """Generate a mapping draft for a sample payload and target schema."""
-    schema = get_target_schema(payload.target_schema)
-    redacted_sample = redact_for_llm(payload.sample_payload)
-    fallback_mapping = build_heuristic_mapping(redacted_sample, payload.target_schema)
-
-    ai_error: str | None = None
-    mapping = fallback_mapping
-    generated_by = "heuristic"
-    if payload.use_ai:
-        try:
-            response = await ai_client.analyze(
-                SituationalAnalysisRequest(
-                    title=f"Generate datasource mapping for {schema.key}",
-                    objective=(
-                        "Return only JSON for a deterministic mapping DSL. "
-                        "The JSON must contain source.items_path and fields. "
-                        "Do not include prose or code."
-                    ),
-                    context={
-                        "target_schema": schema.to_dict(),
-                        "sample_payload": redacted_sample,
-                        "mapping_dsl_example": fallback_mapping,
-                    },
-                    observations=[
-                        "Use JSONPath-like paths beginning with $.",
-                        "Never generate executable code.",
-                        "Use field types from the target schema.",
-                    ],
-                    constraints=[
-                        "Return a single JSON object.",
-                        "Do not include credentials or secrets.",
-                        "Mark uncertain optional fields with default null.",
-                    ],
-                )
-            )
-            parsed = _parse_mapping_from_ai_text(response.content)
-            if parsed:
-                mapping = parsed
-                generated_by = "ai_provider"
-            else:
-                ai_error = "AI provider did not return a valid mapping JSON object."
-        except HTTPException as exc:
-            ai_error = str(exc.detail)
-
-    mapping.setdefault("meta", {})
-    if isinstance(mapping["meta"], dict):
-        mapping["meta"].update(
+        result.append(
             {
-                "generated_by": generated_by,
-                "requires_review": True,
-                "ai_error": ai_error,
+                "name": name,
+                "default_url": yaml_url,
+                "endpoint": db_config.endpoint if db_config else yaml_url,
+                "is_overridden": db_config is not None and db_config.endpoint != yaml_url
+                if yaml_url
+                else db_config is not None,
+                "is_active": db_config.is_active if db_config else True,
+                "source_type": db_config.source_type if db_config else "http",
+                "description": db_config.description
+                if db_config
+                else f"Data source from YAML: {yaml_key}",
             }
         )
 
-    return {
-        "target_schema": schema.to_dict(),
-        "mapping_json": mapping,
-        "sample_payload_hash": stable_payload_hash(payload.sample_payload),
-        "redacted_sample_payload": redacted_sample,
-    }
-
-
-@router.post("/mappings/preview")
-async def preview_datasource_mapping(
-    payload: MappingPreviewRequest,
-    current_user: User = Depends(get_current_user),
-):
-    """Preview deterministic mapping output for a sample payload."""
-    try:
-        preview = execute_mapping(
-            payload.sample_payload,
-            payload.mapping_json,
-            payload.target_schema,
-            limit=payload.limit,
-        )
-    except (MappingError, ValueError) as exc:
-        raise HTTPException(status_code=400, detail=str(exc)) from exc
-
-    return {
-        "success": preview["failed_count"] == 0,
-        "preview": preview,
-        "sample_payload_hash": stable_payload_hash(payload.sample_payload),
-    }
-
-
-@router.get("/mappings")
-async def list_datasource_mappings(
-    datasource_config_id: Optional[int] = None,
-    active_only: bool = False,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    """List saved mapping templates."""
-    query = select(DataSourceMappingTemplate).order_by(
-        DataSourceMappingTemplate.datasource_config_id,
-        DataSourceMappingTemplate.version.desc(),
-    )
-    if datasource_config_id is not None:
-        query = query.where(DataSourceMappingTemplate.datasource_config_id == datasource_config_id)
-    if active_only:
-        query = query.where(DataSourceMappingTemplate.is_active.is_(True))
-
-    result = await db.execute(query)
-    mappings = result.scalars().all()
-    return {"total": len(mappings), "data": [serialize_mapping_template(item) for item in mappings]}
-
-
-@router.post("/mappings")
-async def create_datasource_mapping(
-    payload: MappingTemplateCreate,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    """Save a mapping template for a datasource config."""
-    get_target_schema(payload.target_schema)
-    datasource = await db.get(DataSourceConfig, payload.datasource_config_id)
-    if not datasource:
-        raise HTTPException(status_code=404, detail="Configuration not found")
-
-    if payload.sample_payload is not None:
-        try:
-            execute_mapping(payload.sample_payload, payload.mapping_json, payload.target_schema, limit=100)
-        except (MappingError, ValueError) as exc:
-            raise HTTPException(status_code=400, detail=f"Mapping validation failed: {exc}") from exc
-
-    result = await db.execute(
-        select(func.max(DataSourceMappingTemplate.version)).where(
-            DataSourceMappingTemplate.datasource_config_id == payload.datasource_config_id,
-            DataSourceMappingTemplate.target_schema == payload.target_schema,
-        )
-    )
-    next_version = int(result.scalar() or 0) + 1
-
-    if payload.is_active:
-        await db.execute(
-            DataSourceMappingTemplate.__table__.update()
-            .where(DataSourceMappingTemplate.datasource_config_id == payload.datasource_config_id)
-            .values(is_active=False)
-        )
-
-    template = DataSourceMappingTemplate(
-        datasource_config_id=payload.datasource_config_id,
-        target_schema=payload.target_schema,
-        mapping_json=payload.mapping_json,
-        sample_payload_hash=payload.sample_payload_hash
-        or (stable_payload_hash(payload.sample_payload) if payload.sample_payload is not None else None),
-        validation_status=payload.validation_status,
-        version=next_version,
-        is_active=payload.is_active,
-    )
-    db.add(template)
-    await db.commit()
-    await db.refresh(template)
-    return {"message": "Mapping template saved successfully", "data": serialize_mapping_template(template)}
-
-
-@router.put("/mappings/{mapping_id}")
-async def update_datasource_mapping(
-    mapping_id: int,
-    payload: MappingTemplateUpdate,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    """Update a mapping template in place."""
-    template = await db.get(DataSourceMappingTemplate, mapping_id)
-    if not template:
-        raise HTTPException(status_code=404, detail="Mapping template not found")
-
-    target_schema = payload.target_schema or template.target_schema
-    mapping_json = payload.mapping_json or template.mapping_json
-    get_target_schema(target_schema)
-    if payload.sample_payload is not None:
-        try:
-            execute_mapping(payload.sample_payload, mapping_json, target_schema, limit=100)
-        except (MappingError, ValueError) as exc:
-            raise HTTPException(status_code=400, detail=f"Mapping validation failed: {exc}") from exc
-
-    if payload.is_active is True:
-        await db.execute(
-            DataSourceMappingTemplate.__table__.update()
-            .where(DataSourceMappingTemplate.datasource_config_id == template.datasource_config_id)
-            .where(DataSourceMappingTemplate.id != template.id)
-            .values(is_active=False)
-        )
-
-    template.target_schema = target_schema
-    template.mapping_json = mapping_json
-    if payload.sample_payload_hash is not None:
-        template.sample_payload_hash = payload.sample_payload_hash
-    elif payload.sample_payload is not None:
-        template.sample_payload_hash = stable_payload_hash(payload.sample_payload)
-    if payload.validation_status is not None:
-        template.validation_status = payload.validation_status
-    if payload.is_active is not None:
-        template.is_active = payload.is_active
-
-    await db.commit()
-    await db.refresh(template)
-    return {"message": "Mapping template updated successfully", "data": serialize_mapping_template(template)}
-
-
-@router.post("/{config_id}/run-mapped")
-async def run_mapped_datasource(
-    config_id: int,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    """Run a saved custom datasource through its active deterministic mapping."""
-    datasource = await db.get(DataSourceConfig, config_id)
-    if not datasource:
-        raise HTTPException(status_code=404, detail="Configuration not found")
-
-    result = await db.execute(
-        select(DataSourceMappingTemplate)
-        .where(DataSourceMappingTemplate.datasource_config_id == config_id)
-        .where(DataSourceMappingTemplate.is_active.is_(True))
-        .order_by(DataSourceMappingTemplate.version.desc())
-        .limit(1)
-    )
-    mapping = result.scalar_one_or_none()
-    if not mapping:
-        raise HTTPException(status_code=404, detail="No active mapping template found")
-
-    try:
-        sample = await fetch_custom_sample_from_config(datasource, 5_000_000)
-        mapped = execute_mapping(sample, mapping.mapping_json, mapping.target_schema)
-    except httpx.HTTPStatusError as exc:
-        raise HTTPException(
-            status_code=exc.response.status_code,
-            detail=f"Datasource request failed: HTTP {exc.response.status_code}",
-        ) from exc
-    except httpx.HTTPError as exc:
-        raise HTTPException(status_code=502, detail=f"Datasource request failed: {exc}") from exc
-    except (MappingError, ValueError) as exc:
-        raise HTTPException(status_code=400, detail=f"Mapping failed: {exc}") from exc
-
-    if mapped["failed_count"] > 0:
-        return {
-            "status": "failed",
-            "datasource_config_id": config_id,
-            "mapping_id": mapping.id,
-            "mapping_version": mapping.version,
-            "target_schema": mapping.target_schema,
-            "mapped_count": mapped["mapped_count"],
-            "failed_count": mapped["failed_count"],
-            "errors": mapped["errors"][:20],
-        }
-
-    written_count = await persist_mapped_records(
-        db,
-        datasource_name=datasource.name,
-        datasource_config_id=datasource.id,
-        target_schema=mapping.target_schema,
-        records=mapped["records"],
-        mapping_version=mapping.version,
-    )
-    return {
-        "status": "success",
-        "datasource_config_id": config_id,
-        "mapping_id": mapping.id,
-        "mapping_version": mapping.version,
-        "target_schema": mapping.target_schema,
-        "fetched_count": mapped["total_items"],
-        "mapped_count": mapped["mapped_count"],
-        "written_count": written_count,
-    }
+    return {"total": len(result), "data": result}

backend/app/api/v1/datasources.py

@@ -9,7 +9,6 @@ from sqlalchemy.ext.asyncio import AsyncSession
 from app.core.time import to_iso8601_utc
 from app.core.security import get_current_user
 from app.core.data_sources import get_data_sources_config
-from app.core.datasource_defaults import DEFAULT_DATASOURCES
 from app.db.session import get_db
 from app.models.collected_data import CollectedData
 from app.models.data_snapshot import DataSnapshot
@@ -36,17 +35,6 @@ def format_frequency_label(minutes: int) -> str:
     return f"{minutes}m"
 
 
-def datasource_metadata(source: str) -> dict:
-    info = DEFAULT_DATASOURCES.get(source, {})
-    return {
-        "display_name": info.get("display_name") or info.get("name") or source,
-        "is_free": bool(info.get("is_free", True)),
-        "requires_credentials": bool(info.get("requires_credentials", False)),
-        "credential_provider": info.get("credential_provider"),
-        "credential_status": info.get("credential_status", "none"),
-    }
-
-
 def is_due_for_collection(datasource: DataSource, now: datetime) -> bool:
     if datasource.last_run_at is None:
         return True
@@ -84,6 +72,31 @@ async def _load_latest_running_tasks(
     return {task.datasource_id: task for task in result.scalars().all()}
 
 
+async def _load_latest_completed_tasks(
+    db: AsyncSession,
+    datasource_ids: list[int],
+) -> dict[int, CollectionTask]:
+    if not datasource_ids:
+        return {}
+
+    ranked_tasks = (
+        select(
+            CollectionTask.id.label("task_id"),
+            _task_rank_column(CollectionTask.completed_at),
+        )
+        .where(CollectionTask.datasource_id.in_(datasource_ids))
+        .where(CollectionTask.completed_at.isnot(None))
+        .where(CollectionTask.status.in_(("success", "failed", "cancelled")))
+        .subquery()
+    )
+    result = await db.execute(
+        select(CollectionTask)
+        .join(ranked_tasks, CollectionTask.id == ranked_tasks.c.task_id)
+        .where(ranked_tasks.c.row_num == 1)
+    )
+    return {task.datasource_id: task for task in result.scalars().all()}
+
+
 async def _load_latest_task_ids(
     db: AsyncSession,
     datasource_ids: list[int],
@@ -110,6 +123,21 @@ async def _load_latest_task_ids(
     return {datasource_id: task_id for datasource_id, task_id in result.all()}
 
 
+async def _load_datasource_data_counts(
+    db: AsyncSession,
+    sources: list[str],
+) -> dict[str, int]:
+    if not sources:
+        return {}
+
+    result = await db.execute(
+        select(CollectedData.source, func.count(CollectedData.id))
+        .where(CollectedData.source.in_(sources))
+        .group_by(CollectedData.source)
+    )
+    return {source: count for source, count in result.all()}
+
+
 async def _load_datasource_endpoint_overrides(
     db: AsyncSession,
     sources: list[str],
@@ -133,7 +161,7 @@ async def _load_datasource_endpoint_overrides(
 async def _load_datasource_list_context(
     db: AsyncSession,
     datasources: list[DataSource],
-) -> tuple[dict[int, CollectionTask], dict[str, str]]:
+) -> tuple[dict[int, CollectionTask], dict[int, CollectionTask], dict[str, int], dict[str, str]]:
     datasource_ids = [datasource.id for datasource in datasources]
     sources = [datasource.source for datasource in datasources]
 
@@ -157,8 +185,10 @@ async def _load_datasource_list_context(
     if stale_datasource_ids:
         running_tasks = await _load_latest_running_tasks(db, datasource_ids)
 
+    completed_tasks = await _load_latest_completed_tasks(db, datasource_ids)
+    data_counts = await _load_datasource_data_counts(db, sources)
     endpoint_overrides = await _load_datasource_endpoint_overrides(db, sources)
-    return running_tasks, endpoint_overrides
+    return running_tasks, completed_tasks, data_counts, endpoint_overrides
 
 
 async def get_datasource_record(db: AsyncSession, source_id: str) -> Optional[DataSource]:
@@ -371,19 +401,27 @@ async def list_datasources(
 
     collector_list = []
     config = get_data_sources_config()
-    running_tasks, endpoint_overrides = await _load_datasource_list_context(db, datasources)
+    running_tasks, completed_tasks, data_counts, endpoint_overrides = await _load_datasource_list_context(
+        db,
+        datasources,
+    )
     for datasource in datasources:
         running_task = running_tasks.get(datasource.id)
-        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(datasource.source)
-        last_run_at = datasource.last_run_at
-        last_status = datasource.last_status
+        last_task = completed_tasks.get(datasource.id)
+        endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(
+            datasource.source,
+        )
+        data_count = data_counts.get(datasource.source, 0)
+
+        last_run_at = datasource.last_run_at or (last_task.completed_at if last_task else None)
+        last_run = to_iso8601_utc(last_run_at)
+        last_status = datasource.last_status or (last_task.status if last_task else None)
 
         collector_list.append(
             {
                 "id": datasource.id,
                 "source": datasource.source,
                 "name": datasource.name,
-                **datasource_metadata(datasource.source),
                 "module": datasource.module,
                 "priority": datasource.priority,
                 "frequency": format_frequency_label(datasource.frequency_minutes),
@@ -391,18 +429,15 @@ async def list_datasources(
                 "is_active": datasource.is_active,
                 "collector_class": datasource.collector_class,
                 "endpoint": endpoint,
-                "last_run": to_iso8601_utc(last_run_at),
+                "last_run": last_run,
                 "last_run_at": to_iso8601_utc(last_run_at),
                 "last_status": last_status,
+                "last_records_processed": last_task.records_processed if last_task else None,
+                "data_count": data_count,
                 "is_running": running_task is not None,
                 "task_id": running_task.id if running_task else None,
                 "progress": running_task.progress if running_task else None,
                 "phase": running_task.phase if running_task else None,
-                "phase_progress": running_task.phase_progress if running_task else None,
-                "phase_message": running_task.phase_message if running_task else None,
-                "phase_current": running_task.phase_current if running_task else None,
-                "phase_total": running_task.phase_total if running_task else None,
-                "phase_unit": running_task.phase_unit if running_task else None,
                 "records_processed": running_task.records_processed if running_task else None,
                 "total_records": running_task.total_records if running_task else None,
             }
@@ -541,7 +576,6 @@ async def get_datasource(
     return {
         "id": datasource.id,
         "name": datasource.name,
-        **datasource_metadata(datasource.source),
         "module": datasource.module,
         "priority": datasource.priority,
         "frequency": format_frequency_label(datasource.frequency_minutes),
@@ -631,11 +665,6 @@ async def trigger_datasource(
                 "message": "当前采集任务尚未完成，重新触发会丢失本次未完成进度。是否强制重新采集？",
                 "task_id": running_task.id,
                 "phase": running_task.phase,
-                "phase_progress": running_task.phase_progress,
-                "phase_message": running_task.phase_message,
-                "phase_current": running_task.phase_current,
-                "phase_total": running_task.phase_total,
-                "phase_unit": running_task.phase_unit,
                 "progress": running_task.progress,
                 "records_processed": running_task.records_processed,
                 "total_records": running_task.total_records,
@@ -719,29 +748,13 @@ async def get_task_status(
         task = await get_running_task(db, datasource.id)
 
     if not task:
-        return {
-            "is_running": False,
-            "task_id": None,
-            "progress": None,
-            "phase": None,
-            "phase_progress": None,
-            "phase_message": None,
-            "phase_current": None,
-            "phase_total": None,
-            "phase_unit": None,
-            "status": "idle",
-        }
+        return {"is_running": False, "task_id": None, "progress": None, "phase": None, "status": "idle"}
 
     return {
         "is_running": task.status == "running",
         "task_id": task.id,
         "progress": task.progress,
         "phase": task.phase,
-        "phase_progress": task.phase_progress,
-        "phase_message": task.phase_message,
-        "phase_current": task.phase_current,
-        "phase_total": task.phase_total,
-        "phase_unit": task.phase_unit,
         "records_processed": task.records_processed,
         "total_records": task.total_records,
         "status": task.status,

backend/app/api/v1/settings.py

@@ -9,37 +9,10 @@ from sqlalchemy.ext.asyncio import AsyncSession
 
 from app.core.security import get_current_user
 from app.core.time import to_iso8601_utc
-from app.core.config import settings as app_settings
-from app.core.data_sources import get_data_sources_config
-from app.core.datasource_defaults import DEFAULT_DATASOURCES
 from app.db.session import get_db
 from app.models.datasource import DataSource
-from app.models.datasource_config import DataSourceConfig
 from app.models.system_setting import SystemSetting
 from app.models.user import User
-from app.models.vessel import AISSourceHealth
-from app.services.barentswatch import (
-    BarentsWatchConfig,
-    check_barentswatch_config,
-    check_barentswatch_connectivity,
-    get_barentswatch_datasource_record,
-    resolve_barentswatch_config,
-)
-from app.services.credential_guides import (
-    generate_credential_guide,
-    get_credential_guide,
-    reset_credential_guide,
-)
-from app.services.datasource_connectivity import (
-    build_builtin_connectivity_checksum,
-    save_connectivity_success,
-)
-from app.services.ai_client import AIProviderClient, get_ai_provider_client
-from app.services.llm_provider_catalog import (
-    get_fallback_llm_provider_preset,
-    list_fallback_llm_provider_presets,
-    refresh_llm_provider_preset,
-)
 from app.services.scheduler import sync_datasource_job
 from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
 
@@ -66,21 +39,6 @@ DEFAULT_SETTINGS = {
         "password_policy": "medium",
     },
     "tv": DEFAULT_TV_SETTINGS,
-    "external_integrations": {
-        "ai_provider": {
-            "service_url": "",
-            "service_token": "",
-            "provider": "minimax",
-            "provider_api": "anthropic-messages",
-            "base_url": "https://api.minimaxi.com/anthropic",
-            "model": "MiniMax-M2.7",
-            "api_key": "",
-            "max_tokens": 1200,
-            "anthropic_version": "2023-06-01",
-            "timeout_seconds": 60,
-            "retry_attempts": 2,
-        }
-    },
 }
 
 
@@ -138,34 +96,6 @@ class TVSettingsUpdate(BaseModel):
     sources: list[TVStreamSourceUpdate] = Field(default_factory=list)
 
 
-class AIProviderIntegrationUpdate(BaseModel):
-    service_url: str = ""
-    service_token: Optional[str] = None
-    provider: str = Field(default="minimax", max_length=80)
-    provider_api: str = Field(default="anthropic-messages", max_length=80)
-    base_url: str = Field(default="", max_length=500)
-    model: str = Field(default="", max_length=200)
-    api_key: Optional[str] = None
-    max_tokens: int = Field(default=1200, ge=1, le=200000)
-    anthropic_version: str = Field(default="2023-06-01", max_length=40)
-    timeout_seconds: int = Field(default=60, ge=5, le=600)
-    retry_attempts: int = Field(default=2, ge=1, le=10)
-    clear_service_token: bool = False
-    clear_api_key: bool = False
-
-
-class BarentsWatchIntegrationUpdate(BaseModel):
-    endpoint: str = ""
-    client_id: str = ""
-    client_secret: Optional[str] = None
-    clear_client_secret: bool = False
-
-
-class ExternalIntegrationsUpdate(BaseModel):
-    ai_provider: AIProviderIntegrationUpdate
-    barentswatch: BarentsWatchIntegrationUpdate
-
-
 def merge_with_defaults(category: str, payload: Optional[dict]) -> dict:
     merged = deepcopy(DEFAULT_SETTINGS[category])
     if payload:
@@ -216,151 +146,6 @@ async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -
     return merge_with_defaults(category, record.payload)
 
 
-def _mask_secret(value: Optional[str]) -> dict:
-    if not value:
-        return {"configured": False, "preview": ""}
-    text = str(value)
-    if "-" in text:
-        prefix = text.split("-", 1)[0] + "-"
-        preview = prefix + ("*" * max(len(text) - len(prefix), 1))
-    else:
-        prefix_len = min(4, len(text))
-        preview = text[:prefix_len] + ("*" * max(len(text) - prefix_len, 1))
-    return {"configured": True, "preview": preview}
-
-
-async def get_runtime_ai_provider_config(db: AsyncSession) -> dict:
-    runtime_record = await get_setting_record(db, "external_integrations")
-    payload = merge_with_defaults(
-        "external_integrations",
-        runtime_record.payload if runtime_record else None,
-    )
-    ai_payload = payload.get("ai_provider") or {}
-    has_runtime_llm_config = bool(
-        runtime_record
-        and isinstance(runtime_record.payload, dict)
-        and isinstance(runtime_record.payload.get("ai_provider"), dict)
-    )
-    return {
-        "service_url": ai_payload.get("service_url") or app_settings.AI_PROVIDER_SERVICE_URL,
-        "service_token": ai_payload.get("service_token") or app_settings.AI_PROVIDER_SERVICE_TOKEN,
-        "timeout_seconds": int(
-            ai_payload.get("timeout_seconds") or app_settings.AI_PROVIDER_TIMEOUT_SECONDS
-        ),
-        "retry_attempts": int(
-            ai_payload.get("retry_attempts") or app_settings.AI_PROVIDER_RETRY_ATTEMPTS
-        ),
-        "llm_config": {
-            "provider": ai_payload.get("provider") or "minimax",
-            "provider_api": ai_payload.get("provider_api") or "anthropic-messages",
-            "base_url": ai_payload.get("base_url") or "https://api.minimaxi.com/anthropic",
-            "model": ai_payload.get("model") or "MiniMax-M2.7",
-            "api_key": ai_payload.get("api_key") or "",
-            "max_tokens": int(ai_payload.get("max_tokens") or 1200),
-            "anthropic_version": ai_payload.get("anthropic_version") or "2023-06-01",
-        } if has_runtime_llm_config else {},
-    }
-
-
-async def get_barentswatch_config_record(db: AsyncSession) -> Optional[DataSourceConfig]:
-    return await get_barentswatch_datasource_record(db)
-
-
-async def serialize_external_integrations(db: AsyncSession) -> dict:
-    ai_config = await get_runtime_ai_provider_config(db)
-    runtime_setting = await get_setting_record(db, "external_integrations")
-    display_llm_config = ai_config["llm_config"] or DEFAULT_SETTINGS["external_integrations"]["ai_provider"]
-    barentswatch_record = await get_barentswatch_config_record(db)
-    barentswatch_auth = barentswatch_record.auth_config if barentswatch_record else {}
-    barentswatch_auth = barentswatch_auth or {}
-    resolved_barentswatch = await resolve_barentswatch_config(db)
-    return {
-        "ai_provider": {
-            "service_url": ai_config["service_url"],
-            "service_token": _mask_secret(ai_config["service_token"]),
-            "provider": display_llm_config.get("provider") or "minimax",
-            "provider_api": display_llm_config.get("provider_api") or "anthropic-messages",
-            "base_url": display_llm_config.get("base_url") or "https://api.minimaxi.com/anthropic",
-            "model": display_llm_config.get("model") or "MiniMax-M2.7",
-            "api_key": _mask_secret(display_llm_config.get("api_key")),
-            "max_tokens": int(display_llm_config.get("max_tokens") or 1200),
-            "anthropic_version": display_llm_config.get("anthropic_version") or "2023-06-01",
-            "timeout_seconds": ai_config["timeout_seconds"],
-            "retry_attempts": ai_config["retry_attempts"],
-            "source": "runtime" if runtime_setting else "env",
-        },
-        "barentswatch": {
-            "endpoint": resolved_barentswatch.endpoint,
-            "client_id": barentswatch_auth.get("client_id") or resolved_barentswatch.client_id,
-            "client_secret": _mask_secret(
-                barentswatch_auth.get("client_secret") or resolved_barentswatch.client_secret
-            ),
-            "source": resolved_barentswatch.credential_source,
-        },
-    }
-
-
-async def save_external_integrations_payload(
-    db: AsyncSession,
-    update: ExternalIntegrationsUpdate,
-) -> dict:
-    current_payload = await get_setting_payload(db, "external_integrations")
-    current_ai = current_payload.get("ai_provider") or {}
-    ai_payload = {
-        "service_url": update.ai_provider.service_url.strip()
-        or app_settings.AI_PROVIDER_SERVICE_URL,
-        "service_token": current_ai.get("service_token") or "",
-        "provider": update.ai_provider.provider.strip() or "minimax",
-        "provider_api": update.ai_provider.provider_api.strip() or "anthropic-messages",
-        "base_url": update.ai_provider.base_url.strip(),
-        "model": update.ai_provider.model.strip(),
-        "api_key": current_ai.get("api_key") or "",
-        "max_tokens": update.ai_provider.max_tokens,
-        "anthropic_version": update.ai_provider.anthropic_version.strip() or "2023-06-01",
-        "timeout_seconds": update.ai_provider.timeout_seconds,
-        "retry_attempts": update.ai_provider.retry_attempts,
-    }
-    if update.ai_provider.clear_service_token:
-        ai_payload["service_token"] = ""
-    elif update.ai_provider.service_token not in (None, ""):
-        ai_payload["service_token"] = update.ai_provider.service_token
-    if update.ai_provider.clear_api_key:
-        ai_payload["api_key"] = ""
-    elif update.ai_provider.api_key not in (None, ""):
-        ai_payload["api_key"] = update.ai_provider.api_key
-
-    await save_setting_payload(db, "external_integrations", {"ai_provider": ai_payload})
-
-    default_endpoint = get_data_sources_config().get_yaml_url("barentswatch_vessels")
-    barentswatch_record = await get_barentswatch_config_record(db)
-    if barentswatch_record is None:
-        barentswatch_record = DataSourceConfig(
-            name="barentswatch_vessels",
-            description="BarentsWatch Live AIS credentials",
-            source_type="api",
-            endpoint=update.barentswatch.endpoint.strip() or default_endpoint,
-            auth_type="oauth_client",
-            auth_config={},
-            headers={},
-            config={},
-            is_active=True,
-        )
-        db.add(barentswatch_record)
-
-    current_auth = dict(barentswatch_record.auth_config or {})
-    if update.barentswatch.clear_client_secret:
-        current_auth.pop("client_secret", None)
-    elif update.barentswatch.client_secret not in (None, ""):
-        current_auth["client_secret"] = update.barentswatch.client_secret
-    current_auth["client_id"] = update.barentswatch.client_id.strip()
-    barentswatch_record.endpoint = update.barentswatch.endpoint.strip() or default_endpoint
-    barentswatch_record.auth_type = "oauth_client"
-    barentswatch_record.auth_config = current_auth
-    await db.commit()
-
-    return await serialize_external_integrations(db)
-
-
 def format_frequency_label(minutes: int) -> str:
     if minutes % 1440 == 0:
         return f"{minutes // 1440}d"
@@ -369,17 +154,10 @@ def format_frequency_label(minutes: int) -> str:
     return f"{minutes}m"
 
 
-async def get_ais_source_health_by_source(db: AsyncSession) -> dict[str, dict]:
-    result = await db.execute(select(AISSourceHealth))
-    return {item.source: item.to_dict() for item in result.scalars().all()}
-
-
-def serialize_collector(datasource: DataSource, ais_health_by_source: dict[str, dict] | None = None) -> dict:
-    defaults = DEFAULT_DATASOURCES.get(datasource.source, {})
+def serialize_collector(datasource: DataSource) -> dict:
     return {
         "id": datasource.id,
         "name": datasource.name,
-        "display_name": defaults.get("display_name") or datasource.name,
         "source": datasource.source,
         "module": datasource.module,
         "priority": datasource.priority,
@@ -389,11 +167,6 @@ def serialize_collector(datasource: DataSource, ais_health_by_source: dict[str,
         "last_run_at": to_iso8601_utc(datasource.last_run_at),
         "last_status": datasource.last_status,
         "next_run_at": to_iso8601_utc(datasource.next_run_at),
-        "is_free": bool(defaults.get("is_free", True)),
-        "requires_credentials": bool(defaults.get("requires_credentials", False)),
-        "credential_provider": defaults.get("credential_provider"),
-        "credential_status": defaults.get("credential_status", "none"),
-        "ais_health": (ais_health_by_source or {}).get(datasource.source),
     }
 
 
@@ -470,135 +243,6 @@ async def update_tv_settings(
     return {"status": "updated", "tv": normalize_tv_settings(saved)}
 
 
-@router.get("/integrations")
-async def get_external_integrations(
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    return {"integrations": await serialize_external_integrations(db)}
-
-
-@router.get("/integrations/barentswatch/connectivity")
-async def get_barentswatch_connectivity(
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    return await check_barentswatch_connectivity(db)
-
-
-@router.post("/integrations/barentswatch/connect")
-async def connect_barentswatch_integration(
-    payload: BarentsWatchIntegrationUpdate,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    current = await resolve_barentswatch_config(db)
-    config = BarentsWatchConfig(
-        endpoint=payload.endpoint.strip() or current.endpoint,
-        client_id=payload.client_id.strip() or current.client_id,
-        client_secret=(
-            ""
-            if payload.clear_client_secret
-            else payload.client_secret or current.client_secret
-        ),
-        credential_source="draft",
-        endpoint_source="draft",
-    )
-    result = await check_barentswatch_config(config)
-    if result.get("success"):
-        checksum, _context = await build_builtin_connectivity_checksum(
-            "barentswatch_vessels",
-            config.endpoint,
-            "none",
-            {},
-            {},
-            db,
-            credential_override={
-                "client_id": config.client_id,
-                "client_secret": config.client_secret,
-            },
-        )
-        validation = await save_connectivity_success(
-            db,
-            "barentswatch_vessels",
-            checksum,
-            result,
-            connected_by="connection_button",
-        )
-        await db.commit()
-        return {**result, "connected": True, "validation": validation}
-    return {**result, "connected": False}
-
-
-@router.get("/credential-guides/{provider}")
-async def read_credential_guide(
-    provider: str,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    try:
-        return {"guide": await get_credential_guide(db, provider)}
-    except ValueError as exc:
-        raise HTTPException(status_code=404, detail=str(exc)) from exc
-
-
-@router.post("/credential-guides/{provider}/generate")
-async def generate_provider_credential_guide(
-    provider: str,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-    ai_client: AIProviderClient = Depends(get_ai_provider_client),
-):
-    try:
-        return {"guide": await generate_credential_guide(db, provider, ai_client)}
-    except ValueError as exc:
-        raise HTTPException(status_code=404, detail=str(exc)) from exc
-
-
-@router.post("/credential-guides/{provider}/reset")
-async def reset_provider_credential_guide(
-    provider: str,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    try:
-        return {"guide": await reset_credential_guide(db, provider)}
-    except ValueError as exc:
-        raise HTTPException(status_code=404, detail=str(exc)) from exc
-
-
-@router.get("/integrations/ai-provider/presets")
-async def get_ai_provider_presets(
-    current_user: User = Depends(get_current_user),
-):
-    return {"data": list_fallback_llm_provider_presets()}
-
-
-@router.post("/integrations/ai-provider/presets/{provider}/refresh")
-async def refresh_ai_provider_preset(
-    provider: str,
-    current_user: User = Depends(get_current_user),
-):
-    try:
-        return {"data": await refresh_llm_provider_preset(provider)}
-    except ValueError as exc:
-        raise HTTPException(status_code=404, detail=str(exc)) from exc
-    except Exception as exc:
-        fallback = get_fallback_llm_provider_preset(provider)
-        fallback["refresh_error"] = str(exc)
-        return {"data": fallback}
-
-
-@router.put("/integrations")
-async def update_external_integrations(
-    payload: ExternalIntegrationsUpdate,
-    current_user: User = Depends(get_current_user),
-    db: AsyncSession = Depends(get_db),
-):
-    saved = await save_external_integrations_payload(db, payload)
-    return {"status": "updated", "integrations": saved}
-
-
 @router.get("/collectors")
 async def get_collector_settings(
     current_user: User = Depends(get_current_user),
@@ -606,8 +250,7 @@ async def get_collector_settings(
 ):
     result = await db.execute(select(DataSource).order_by(DataSource.module, DataSource.id))
     datasources = result.scalars().all()
-    ais_health_by_source = await get_ais_source_health_by_source(db)
-    return {"collectors": [serialize_collector(datasource, ais_health_by_source) for datasource in datasources]}
+    return {"collectors": [serialize_collector(datasource) for datasource in datasources]}
 
 
 @router.put("/collectors/{datasource_id}")
@@ -627,8 +270,7 @@ async def update_collector_settings(
     await db.commit()
     await db.refresh(datasource)
     await sync_datasource_job(datasource.id)
-    ais_health_by_source = await get_ais_source_health_by_source(db)
-    return {"status": "updated", "collector": serialize_collector(datasource, ais_health_by_source)}
+    return {"status": "updated", "collector": serialize_collector(datasource)}
 
 
 @router.get("")
@@ -642,13 +284,11 @@ async def get_all_settings(
         db,
         ["system", "notifications", "security"],
     )
-    ais_health_by_source = await get_ais_source_health_by_source(db)
     return {
         "system": setting_payloads["system"],
         "notifications": setting_payloads["notifications"],
         "security": setting_payloads["security"],
         "tv": await get_tv_settings_payload(db),
-        "integrations": await serialize_external_integrations(db),
-        "collectors": [serialize_collector(datasource, ais_health_by_source) for datasource in datasources],
+        "collectors": [serialize_collector(datasource) for datasource in datasources],
         "generated_at": to_iso8601_utc(datetime.now(UTC)),
     }

backend/app/api/v1/system_control.py

@@ -4,15 +4,12 @@ import os
 import subprocess
 import sys
 
-from datetime import datetime
-
-from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
+from fastapi import APIRouter, Depends, HTTPException, status
 from pydantic import BaseModel
 
 from app.core.config import ROOT_DIR
 from app.core.security import get_current_user
 from app.models.user import User
-from app.services.persistent_logs import record_audit_log, record_system_log
 from app.services.system_control import (
     build_task_id,
     clear_active_task_id,
@@ -26,15 +23,6 @@ from app.services.system_control import (
     set_active_task_id,
     upsert_task_state,
 )
-from app.services.system_logs import (
-    DEFAULT_LOG_LINE_LIMIT,
-    MAX_LOG_LINE_LIMIT,
-    SUPPORTED_LOG_LEVELS,
-    append_buffer_log,
-    list_log_sources,
-    normalize_log_level,
-    read_log_snapshot,
-)
 
 router = APIRouter()
 
@@ -59,59 +47,6 @@ class RestartTaskLogsResponse(BaseModel):
     lines: list[str]
 
 
-class SystemLogSourceSummary(BaseModel):
-    source_id: str
-    name: str
-    kind: str
-    location: str
-    description: str
-    category: str
-    status: str
-
-
-class SystemLogSourcesResponse(BaseModel):
-    items: list[SystemLogSourceSummary]
-
-
-class SystemLogDailyMarker(BaseModel):
-    date_token: str
-    total: int
-    dominant_level: str
-
-
-class SystemLogSnapshotResponse(BaseModel):
-    source_id: str
-    name: str
-    kind: str
-    location: str
-    description: str
-    category: str
-    status: str
-    level: str
-    selected_levels: list[str] = []
-    search_query: str = ""
-    available_levels: list[str]
-    daily_markers: list[SystemLogDailyMarker] = []
-    line_limit: int
-    line_count: int
-    lines: list[str]
-
-
-class EarthClientLogEventCreate(BaseModel):
-    level: str = "error"
-    message: str
-    category: str | None = None
-    url: str | None = None
-    module: str | None = None
-    detail: str | None = None
-
-
-class EarthClientLogEventResponse(BaseModel):
-    accepted: bool
-    source_id: str
-    level: str
-
-
 def ensure_super_admin(current_user: User) -> None:
     if not require_super_admin(current_user.role):
         raise HTTPException(
@@ -120,22 +55,9 @@ def ensure_super_admin(current_user: User) -> None:
         )
 
 
-def validate_log_date(raw_value: str | None, field_name: str) -> str | None:
-    if raw_value in {None, ""}:
-        return None
-    try:
-        return datetime.strptime(raw_value, "%Y-%m-%d").date().isoformat()
-    except ValueError as exc:
-        raise HTTPException(
-            status_code=status.HTTP_400_BAD_REQUEST,
-            detail=f"{field_name} must be in YYYY-MM-DD format",
-        ) from exc
-
-
 @router.post("/restart-tasks", response_model=RestartTaskResponse)
 async def create_restart_task(
     payload: RestartTaskCreate,
-    request: Request,
     current_user: User = Depends(get_current_user),
 ):
     ensure_super_admin(current_user)
@@ -211,31 +133,11 @@ async def create_restart_task(
             requested_by=requested_by,
         )
         clear_active_task_id(task_id)
-        await record_audit_log(
-            action="system.restart_task.requested",
-            actor_id=current_user.id,
-            actor_name=current_user.username,
-            target_type="restart_task",
-            target_id=task_id,
-            result="failed",
-            ip=request.client.host if request.client else None,
-            details={"action": payload.action, "message": task_state["message"]},
-        )
         raise HTTPException(
             status_code=status.HTTP_500_INTERNAL_SERVER_ERROR,
             detail=task_state["message"],
         ) from exc
 
-    await record_audit_log(
-        action="system.restart_task.requested",
-        actor_id=current_user.id,
-        actor_name=current_user.username,
-        target_type="restart_task",
-        target_id=task_id,
-        result="accepted",
-        ip=request.client.host if request.client else None,
-        details={"action": payload.action},
-    )
     return task_state
 
 
@@ -263,92 +165,3 @@ async def get_restart_task_logs(
     if task is None:
         raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Restart task not found")
     return {"task_id": task_id, "lines": get_task_logs(task_id)}
-
-
-@router.get("/logs/sources", response_model=SystemLogSourcesResponse)
-async def get_system_log_sources(
-    current_user: User = Depends(get_current_user),
-):
-    ensure_super_admin(current_user)
-    return {"items": list_log_sources()}
-
-
-@router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
-async def get_system_log_snapshot(
-    source_id: str,
-    limit: int = DEFAULT_LOG_LINE_LIMIT,
-    level: str = "all",
-    levels: str | None = Query(None, description="Comma-separated log levels"),
-    start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
-    end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
-    search: str | None = Query(None, description="Case-insensitive substring search"),
-    current_user: User = Depends(get_current_user),
-):
-    ensure_super_admin(current_user)
-
-    if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
-        raise HTTPException(
-            status_code=status.HTTP_400_BAD_REQUEST,
-            detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}",
-        )
-    if str(level).strip().lower() not in SUPPORTED_LOG_LEVELS and normalize_log_level(level) == "all" and str(level).strip().lower() not in {"", "all"}:
-        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Unsupported log level")
-    if levels:
-        for raw_level in str(levels).split(","):
-            normalized_level = str(raw_level).strip().lower()
-            if not normalized_level:
-                continue
-            if normalized_level not in SUPPORTED_LOG_LEVELS and normalize_log_level(normalized_level) == "all":
-                raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="Unsupported log level")
-    normalized_start_date = validate_log_date(start_date, "start_date")
-    normalized_end_date = validate_log_date(end_date, "end_date")
-    if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
-        raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
-
-    snapshot = read_log_snapshot(
-        source_id,
-        limit,
-        level=level,
-        levels=levels,
-        start_date=normalized_start_date,
-        end_date=normalized_end_date,
-        search=search,
-    )
-    if snapshot is None:
-        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Log source not found")
-    return snapshot
-
-
-@router.post("/logs/earth-client", response_model=EarthClientLogEventResponse)
-async def ingest_earth_client_log(
-    payload: EarthClientLogEventCreate,
-    request: Request,
-):
-    normalized_level = normalize_log_level(payload.level)
-    append_buffer_log(
-        "earth-client",
-        level=normalized_level,
-        message=payload.message,
-        context={
-            "category": payload.category or "",
-            "url": payload.url or "",
-            "module": payload.module or "",
-            "detail": payload.detail or "",
-        },
-    )
-    await record_system_log(
-        source="earth-client",
-        service="earth",
-        module=payload.module or "earth-client",
-        event="earth.client.runtime_log",
-        level=normalized_level,
-        message=payload.message,
-        category=payload.category or "client-runtime",
-        context={
-            "url": payload.url or "",
-            "detail": payload.detail or "",
-            "module": payload.module or "",
-            "client_ip": request.client.host if request.client else "",
-        },
-    )
-    return {"accepted": True, "source_id": "earth-client", "level": normalized_level}

backend/app/api/v1/tasks.py

@@ -27,9 +27,7 @@ async def list_tasks(
     offset = (page - 1) * page_size
     query = """
         SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
-               ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
-               ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
-               ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
+               ct.started_at, ct.completed_at, ct.records_processed, ct.error_message
         FROM collection_tasks ct
         JOIN data_sources ds ON ct.datasource_id = ds.id
         WHERE 1=1
@@ -68,14 +66,6 @@ async def list_tasks(
                 "completed_at": to_iso8601_utc(t[5]),
                 "records_processed": t[6],
                 "error_message": t[7],
-                "phase": t[8],
-                "phase_progress": t[9],
-                "phase_message": t[10],
-                "phase_current": t[11],
-                "phase_total": t[12],
-                "phase_unit": t[13],
-                "total_records": t[14],
-                "progress": t[15],
             }
             for t in tasks
         ],
@@ -91,9 +81,7 @@ async def get_task(
     result = await db.execute(
         text("""
             SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
-                   ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
-                   ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
-                   ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
+                   ct.started_at, ct.completed_at, ct.records_processed, ct.error_message
             FROM collection_tasks ct
             JOIN data_sources ds ON ct.datasource_id = ds.id
             WHERE ct.id = :id
@@ -117,14 +105,6 @@ async def get_task(
         "completed_at": to_iso8601_utc(task[5]),
         "records_processed": task[6],
         "error_message": task[7],
-        "phase": task[8],
-        "phase_progress": task[9],
-        "phase_message": task[10],
-        "phase_current": task[11],
-        "phase_total": task[12],
-        "phase_unit": task[13],
-        "total_records": task[14],
-        "progress": task[15],
     }
 
 

backend/app/api/v1/visualization.py

@@ -4,7 +4,7 @@ Unified API for all visualization data sources.
 Returns GeoJSON format compatible with Three.js, CesiumJS, and Unreal Cesium.
 """
 
-from datetime import UTC, datetime, timedelta
+from datetime import UTC, datetime
 import math
 import httpx
 from fastapi import APIRouter, HTTPException, Depends, Query, Response
@@ -13,30 +13,17 @@ from sqlalchemy import select, func
 from typing import List, Dict, Any, Optional
 
 from app.core.collected_data_fields import get_record_field
-from app.core.countries import get_country_centroid
 from app.core.satellite_tle import build_tle_lines_from_elements
 from app.core.time import to_iso8601_utc
 from app.db.session import get_db
 from app.models.bgp_anomaly import BGPAnomaly
 from app.models.bgp_incident import BGPIncident
 from app.models.collected_data import CollectedData
-from app.models.vessel import VesselPosition, VesselStatic
 from app.services.bgp_collectors import build_bgp_collector_coverage
 from app.services.cable_graph import build_graph_from_data, CableGraph, haversine_distance
 from app.services.collectors.bgp_common import RIPE_RIS_COLLECTOR_COORDS
-from app.services.persistent_logs import record_system_log
-from app.services.vessel_ais_aggregation import (
-    build_field_conflict_candidates,
-    get_aggregated_vessel,
-    get_aggregated_vessel_track,
-    get_aggregated_vessels,
-    get_vessel_conflict_records,
-    get_vessel_raw_observations,
-)
-from app.core.logging import get_logger
 
 router = APIRouter()
-logger = get_logger(__name__, service="api")
 TERRAIN_TILE_URL_TEMPLATE = (
     "https://s3.amazonaws.com/elevation-tiles-prod/terrarium/{z}/{x}/{y}.png"
 )
@@ -193,12 +180,6 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
                 mean_motion=metadata.get("mean_motion"),
             )
 
-        constellation_group = _normalize_satellite_constellation_group(
-            metadata.get("constellation_group"),
-            record.name,
-        )
-        footprint_policy = _get_satellite_footprint_policy(constellation_group)
-
         features.append(
             {
                 "type": "Feature",
@@ -208,8 +189,6 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
                     "id": record.id,
                     "norad_cat_id": norad_id,
                     "name": record.name,
-                    "constellation_group": constellation_group,
-                    "footprint_policy": footprint_policy,
                     "international_designator": metadata.get("international_designator"),
                     "epoch": metadata.get("epoch"),
                     "inclination": metadata.get("inclination"),
@@ -230,31 +209,6 @@ def convert_satellite_to_geojson(records: List[CollectedData]) -> Dict[str, Any]
     return {"type": "FeatureCollection", "features": features}
 
 
-def _normalize_satellite_constellation_group(
-    raw_group: Any,
-    name: Optional[str],
-) -> Optional[str]:
-    normalized_group = str(raw_group or "").strip().lower()
-    if normalized_group:
-        return normalized_group
-
-    normalized_name = str(name or "").strip().upper()
-    if normalized_name.startswith("STARLINK"):
-        return "starlink"
-    if normalized_name.startswith("IRIDIUM"):
-        return "iridium-next"
-
-    return None
-
-
-def _get_satellite_footprint_policy(constellation_group: Optional[str]) -> str:
-    if constellation_group == "starlink":
-        return "starlink_ground_footprint"
-    if constellation_group == "iridium-next":
-        return "iridium_coverage_ring"
-    return "none"
-
-
 def _current_collected_data_stmt(source: str):
     return (
         select(CollectedData)
@@ -409,365 +363,6 @@ def convert_gpu_cluster_to_geojson(records: List[CollectedData]) -> Dict[str, An
     return {"type": "FeatureCollection", "features": features}
 
 
-def _parse_float(value: Any) -> Optional[float]:
-    try:
-        if value in (None, ""):
-            return None
-        return float(value)
-    except (TypeError, ValueError):
-        return None
-
-
-COMPUTE_CENTER_COORDINATE_HINTS = (
-    ("el capitan", 37.6819, -121.7681),
-    ("livermore", 37.6819, -121.7681),
-    ("llnl", 37.6819, -121.7681),
-    ("lawrence livermore", 37.6819, -121.7681),
-    ("frontier", 35.9319, -84.3107),
-    ("oak ridge", 35.9319, -84.3107),
-    ("ornl", 35.9319, -84.3107),
-    ("aurora", 41.7130, -87.9820),
-    ("argonne", 41.7130, -87.9820),
-    ("anl", 41.7130, -87.9820),
-    ("fugaku", 34.6953, 135.1974),
-    ("kobe", 34.6953, 135.1974),
-    ("riken", 34.6953, 135.1974),
-    ("summit", 35.9319, -84.3107),
-    ("leonardo", 44.4949, 11.3426),
-    ("bologna", 44.4949, 11.3426),
-    ("alps", 46.0037, 8.9511),
-    ("lugano", 46.0037, 8.9511),
-    ("sunway taihulight", 31.4912, 120.3119),
-    ("wuxi", 31.4912, 120.3119),
-    ("tianhe-2", 23.1291, 113.2644),
-    ("tianhe-2a", 23.1291, 113.2644),
-    ("guangzhou", 23.1291, 113.2644),
-    ("colossus", 35.1495, -90.0490),
-    ("memphis", 35.1495, -90.0490),
-    ("xai", 35.1495, -90.0490),
-)
-
-
-def _normalize_hint_text(*parts: Any) -> str:
-    return " ".join(
-        str(part).strip().lower()
-        for part in parts
-        if part not in (None, "")
-    )
-
-
-def _resolve_compute_center_coordinates(
-    record: CollectedData,
-    metadata: Dict[str, Any],
-) -> Dict[str, Any]:
-    latitude = _parse_float(get_record_field(record, "latitude"))
-    longitude = _parse_float(get_record_field(record, "longitude"))
-    if latitude not in (None, 0.0) and longitude not in (None, 0.0):
-        return {
-            "latitude": latitude,
-            "longitude": longitude,
-            "location_precision": "precise",
-            "geography_mode": "source_coordinates",
-            "is_estimated": False,
-            "estimated_reason": None,
-        }
-
-    hint_text = _normalize_hint_text(
-        record.name,
-        get_record_field(record, "city"),
-        get_record_field(record, "country"),
-        metadata.get("site"),
-        metadata.get("organization"),
-        metadata.get("operator"),
-    )
-    for needle, resolved_latitude, resolved_longitude in COMPUTE_CENTER_COORDINATE_HINTS:
-        if needle in hint_text:
-            return {
-                "latitude": resolved_latitude,
-                "longitude": resolved_longitude,
-                "location_precision": "estimated_site",
-                "geography_mode": "site_hint",
-                "is_estimated": True,
-                "estimated_reason": f"Matched known site hint: {needle}",
-            }
-
-    centroid = get_country_centroid(get_record_field(record, "country"))
-    if centroid:
-        return {
-            "latitude": centroid.get("latitude"),
-            "longitude": centroid.get("longitude"),
-            "location_precision": "estimated_country",
-            "geography_mode": "country_centroid",
-            "is_estimated": True,
-            "estimated_reason": "Estimated from country centroid",
-        }
-
-    return {
-        "latitude": latitude,
-        "longitude": longitude,
-        "location_precision": "unknown",
-        "geography_mode": "unknown",
-        "is_estimated": True,
-        "estimated_reason": "No resolvable location hints",
-    }
-
-
-def _normalize_capacity_band(capacity_value: Optional[float], capacity_unit: str) -> str:
-    if capacity_value is None:
-        return "unknown"
-
-    unit = str(capacity_unit or "").strip().lower()
-    if unit in {"pflop/s", "pflops", "pflop"}:
-        normalized_tflops = capacity_value * 1000
-    elif unit in {"gflop/s", "gflops", "gflop"}:
-        normalized_tflops = capacity_value
-    else:
-        normalized_tflops = capacity_value
-
-    if normalized_tflops >= 1_000_000:
-        return "exascale"
-    if normalized_tflops >= 100_000:
-        return "ultra"
-    if normalized_tflops >= 10_000:
-        return "large"
-    if normalized_tflops > 0:
-        return "regional"
-    return "unknown"
-
-
-def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str, Any]:
-    """Convert compute infrastructure records into a unified GeoJSON layer."""
-    features = []
-
-    for record in records:
-        metadata = record.extra_data or {}
-        coordinate_info = _resolve_compute_center_coordinates(record, metadata)
-        latitude = coordinate_info.get("latitude")
-        longitude = coordinate_info.get("longitude")
-        site_type = (
-            "supercomputer"
-            if record.source == "top500" or record.data_type == "supercomputer"
-            else "gpu_cluster"
-        )
-        if latitude in (None, 0.0) or longitude in (None, 0.0):
-            continue
-
-        if site_type == "supercomputer":
-            capacity_value = _parse_float(get_record_field(record, "rmax"))
-            capacity_unit = "GFlops"
-        else:
-            capacity_value = _parse_float(get_record_field(record, "value"))
-            capacity_unit = str(get_record_field(record, "unit") or "TFlop/s")
-
-        vendor = (
-            metadata.get("manufacturer")
-            or metadata.get("vendor")
-            or metadata.get("gpu_type")
-        )
-        operator = (
-            metadata.get("organization")
-            or metadata.get("operator")
-            or metadata.get("owner")
-        )
-        rank = metadata.get("rank")
-        if rank in (None, "") and site_type == "supercomputer":
-            rank = get_record_field(record, "rank")
-
-        updated_at = to_iso8601_utc(record.reference_date or record.collected_at)
-
-        features.append(
-            {
-                "type": "Feature",
-                "id": record.id,
-                "geometry": {
-                    "type": "Point",
-                    "coordinates": [longitude or 0, latitude or 0],
-                },
-                "properties": {
-                    "id": record.id,
-                    "source_id": record.source_id,
-                    "name": record.name,
-                    "site_type": site_type,
-                    "country": get_record_field(record, "country"),
-                    "city": get_record_field(record, "city"),
-                    "latitude": latitude,
-                    "longitude": longitude,
-                    "operator": operator,
-                    "vendor": vendor,
-                    "capacity_value": capacity_value,
-                    "capacity_unit": capacity_unit,
-                    "capacity_band": _normalize_capacity_band(capacity_value, capacity_unit),
-                    "rank": rank,
-                    "gpu_count": metadata.get("gpu_count"),
-                    "gpu_type": metadata.get("gpu_type"),
-                    "cores": get_record_field(record, "cores"),
-                    "power": get_record_field(record, "power"),
-                    "source": record.source,
-                    "updated_at": updated_at,
-                    "status": "observed",
-                    "location_precision": coordinate_info.get("location_precision"),
-                    "geography_mode": coordinate_info.get("geography_mode"),
-                    "is_estimated": coordinate_info.get("is_estimated", False),
-                    "estimated_reason": coordinate_info.get("estimated_reason"),
-                    "data_type": "compute_center",
-                    "metadata": metadata,
-                },
-            }
-        )
-
-    return {"type": "FeatureCollection", "features": features}
-
-
-VESSEL_TYPE_FILTERS = {
-    "cargo": lambda props: str(props.get("vessel_type_name", "")).lower() == "cargo"
-    or 70 <= int(props.get("vessel_type") or -1) <= 79,
-    "tanker": lambda props: str(props.get("vessel_type_name", "")).lower() == "tanker"
-    or 80 <= int(props.get("vessel_type") or -1) <= 89,
-    "passenger": lambda props: str(props.get("vessel_type_name", "")).lower() == "passenger"
-    or 60 <= int(props.get("vessel_type") or -1) <= 69,
-    "fishing": lambda props: str(props.get("vessel_type_name", "")).lower() == "fishing"
-    or int(props.get("vessel_type") or -1) == 30,
-    "military": lambda props: str(props.get("vessel_type_name", "")).lower() == "military"
-    or int(props.get("vessel_type") or -1) == 35,
-    "other": lambda props: str(props.get("vessel_type_name", "")).lower()
-    not in {"cargo", "tanker", "passenger", "fishing", "military"},
-}
-
-
-def convert_vessels_to_geojson(rows: List[Any]) -> Dict[str, Any]:
-    features = []
-    for position, static in rows:
-        if position.lat is None or position.lon is None:
-            continue
-        props = {
-            "mmsi": position.mmsi,
-            "name": getattr(static, "name", None) or f"MMSI {position.mmsi}",
-            "callsign": getattr(static, "callsign", None),
-            "imo": getattr(static, "imo", None),
-            "vessel_type": getattr(static, "vessel_type", None),
-            "vessel_type_name": getattr(static, "vessel_type_name", None) or "Other",
-            "flag": getattr(static, "flag", None),
-            "length": getattr(static, "length", None),
-            "width": getattr(static, "width", None),
-            "draught": getattr(static, "draught", None),
-            "sog": position.sog,
-            "cog": position.cog,
-            "heading": position.heading,
-            "nav_status": position.nav_status,
-            "received_at": to_iso8601_utc(position.received_at),
-            "data_type": "vessel",
-        }
-        features.append(
-            {
-                "type": "Feature",
-                "id": position.mmsi,
-                "geometry": {
-                    "type": "Point",
-                    "coordinates": [position.lon, position.lat],
-                },
-                "properties": props,
-            }
-        )
-
-    return {"type": "FeatureCollection", "features": features}
-
-
-def convert_aggregated_vessels_to_geojson(vessels: List[dict[str, Any]]) -> Dict[str, Any]:
-    features = []
-    for vessel in vessels:
-        if vessel.get("lat") is None or vessel.get("lon") is None:
-            continue
-        source_summary = {}
-        for source, summary in (vessel.get("source_summary") or {}).items():
-            source_summary[source] = {
-                **summary,
-                "latest_observed_at": to_iso8601_utc(summary.get("latest_observed_at")),
-            }
-        props = {
-            "mmsi": vessel["mmsi"],
-            "name": vessel.get("name") or f"MMSI {vessel['mmsi']}",
-            "callsign": vessel.get("callsign"),
-            "imo": vessel.get("imo"),
-            "vessel_type": vessel.get("vessel_type"),
-            "vessel_type_name": vessel.get("vessel_type_name") or "Other",
-            "flag": vessel.get("flag"),
-            "length": vessel.get("length"),
-            "width": vessel.get("width"),
-            "draught": vessel.get("draught"),
-            "sog": vessel.get("sog"),
-            "cog": vessel.get("cog"),
-            "heading": vessel.get("heading"),
-            "nav_status": vessel.get("nav_status"),
-            "received_at": to_iso8601_utc(vessel.get("received_at")),
-            "field_sources": vessel.get("field_sources") or {},
-            "selected_reasons": vessel.get("selected_reasons") or {},
-            "source_summary": source_summary,
-            "quality_flags": vessel.get("quality_flags") or [],
-            "conflict_count": vessel.get("conflict_count", 0),
-            "data_type": "vessel",
-        }
-        features.append(
-            {
-                "type": "Feature",
-                "id": vessel["mmsi"],
-                "geometry": {
-                    "type": "Point",
-                    "coordinates": [vessel["lon"], vessel["lat"]],
-                },
-                "properties": props,
-            }
-        )
-    return {"type": "FeatureCollection", "features": features}
-
-
-def _parse_bbox(value: Optional[str]) -> tuple[float, float, float, float] | None:
-    if not value:
-        return None
-    parts = [part.strip() for part in value.split(",")]
-    if len(parts) != 4:
-        raise HTTPException(status_code=400, detail="bbox must be lon_min,lat_min,lon_max,lat_max")
-    try:
-        lon_min, lat_min, lon_max, lat_max = [float(part) for part in parts]
-    except ValueError as exc:
-        raise HTTPException(status_code=400, detail="bbox values must be numbers") from exc
-    if lat_min > lat_max:
-        lat_min, lat_max = lat_max, lat_min
-    if lon_min > lon_max:
-        lon_min, lon_max = lon_max, lon_min
-    return lon_min, lat_min, lon_max, lat_max
-
-
-def _matches_vessel_type(props: dict[str, Any], requested_types: set[str]) -> bool:
-    if not requested_types:
-        return True
-    for requested_type in requested_types:
-        predicate = VESSEL_TYPE_FILTERS.get(requested_type)
-        if predicate and predicate(props):
-            return True
-    return False
-
-
-def _build_vessel_stats(features: List[dict[str, Any]]) -> dict[str, Any]:
-    by_type: dict[str, int] = {}
-    underway = 0
-    anchored_or_moored = 0
-    for feature in features:
-        props = feature.get("properties", {})
-        vessel_type = str(props.get("vessel_type_name") or "Other")
-        by_type[vessel_type] = by_type.get(vessel_type, 0) + 1
-        nav_status = props.get("nav_status")
-        if nav_status in (1, 5):
-            anchored_or_moored += 1
-        else:
-            underway += 1
-    return {
-        "total": len(features),
-        "by_type": by_type,
-        "underway": underway,
-        "anchored_or_moored": anchored_or_moored,
-    }
-
-
 def convert_bgp_anomalies_to_geojson(
     records: List[BGPAnomaly],
     geography_hints: Optional[Dict[str, Dict[str, Any]]] = None,
@@ -1185,21 +780,6 @@ async def get_cables_geojson(db: AsyncSession = Depends(get_db)):
     except HTTPException:
         raise
     except Exception as e:
-        logger.exception_event(
-            "Failed to build cables GeoJSON response",
-            event="visualization.cables.load_failed",
-            context={"error": str(e)},
-        )
-        await record_system_log(
-            source="backend",
-            service="api",
-            module=__name__,
-            event="visualization.cables.load_failed",
-            level="error",
-            message="Failed to build cables GeoJSON response",
-            category="visualization",
-            context={"error": str(e)},
-        )
         raise HTTPException(status_code=500, detail=f"Internal error: {str(e)}")
 
 
@@ -1236,21 +816,6 @@ async def get_landing_points_geojson(db: AsyncSession = Depends(get_db)):
     except HTTPException:
         raise
     except Exception as e:
-        logger.exception_event(
-            "Failed to build landing points GeoJSON response",
-            event="visualization.landing_points.load_failed",
-            context={"error": str(e)},
-        )
-        await record_system_log(
-            source="backend",
-            service="api",
-            module=__name__,
-            event="visualization.landing_points.load_failed",
-            level="error",
-            message="Failed to build landing points GeoJSON response",
-            category="visualization",
-            context={"error": str(e)},
-        )
         raise HTTPException(status_code=500, detail=f"Internal error: {str(e)}")
 
 
@@ -1410,275 +975,6 @@ async def get_gpu_clusters_geojson(
     }
 
 
-@router.get("/geo/compute-centers")
-async def get_compute_centers_geojson(
-    limit: int = Query(200, ge=1, le=1000),
-    db: AsyncSession = Depends(get_db),
-):
-    """获取统一算力中心 GeoJSON 数据"""
-    records_by_source = await _load_current_collected_data_by_sources(
-        db,
-        ["top500", "epoch_ai_gpu"],
-    )
-    records = _filter_known_records(
-        records_by_source.get("top500", []) + records_by_source.get("epoch_ai_gpu", []),
-    )
-    if limit is not None:
-        records = records[:limit]
-
-    if not records:
-        return {
-            "type": "FeatureCollection",
-            "features": [],
-            "count": 0,
-            "stats": {
-                "total": 0,
-                "supercomputers": 0,
-                "gpu_clusters": 0,
-            },
-        }
-
-    geojson = convert_compute_centers_to_geojson(records)
-    features = geojson.get("features", [])
-    return {
-        **geojson,
-        "count": len(features),
-        "stats": {
-            "total": len(features),
-            "supercomputers": sum(
-                1 for feature in features
-                if feature.get("properties", {}).get("site_type") == "supercomputer"
-            ),
-            "gpu_clusters": sum(
-                1 for feature in features
-                if feature.get("properties", {}).get("site_type") == "gpu_cluster"
-            ),
-        },
-    }
-
-
-@router.get("/geo/vessels")
-async def get_vessels_geojson(
-    bbox: Optional[str] = Query(
-        None,
-        description="Viewport bbox as lon_min,lat_min,lon_max,lat_max",
-    ),
-    type: Optional[str] = Query(
-        None,
-        description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",
-    ),
-    limit: Optional[int] = Query(
-        None,
-        ge=0,
-        description="Maximum vessel features to return. Omit or pass 0 for no limit.",
-    ),
-    db: AsyncSession = Depends(get_db),
-):
-    """Return latest vessel positions as GeoJSON points."""
-    parsed_bbox = _parse_bbox(bbox)
-    aggregated_vessels = await get_aggregated_vessels(db, bbox=parsed_bbox, limit=limit)
-    if aggregated_vessels:
-        geojson = convert_aggregated_vessels_to_geojson(aggregated_vessels)
-        requested_types = {
-            item.strip().lower()
-            for item in (type or "").split(",")
-            if item.strip()
-        }
-        if requested_types:
-            geojson["features"] = [
-                feature
-                for feature in geojson.get("features", [])
-                if _matches_vessel_type(feature.get("properties", {}), requested_types)
-            ]
-
-        features = geojson.get("features", [])
-        return {
-            **geojson,
-            "count": len(features),
-            "stats": _build_vessel_stats(features),
-        }
-
-    latest_times = (
-        select(
-            VesselPosition.mmsi.label("mmsi"),
-            func.max(VesselPosition.received_at).label("received_at"),
-        )
-        .group_by(VesselPosition.mmsi)
-        .subquery()
-    )
-    stmt = (
-        select(VesselPosition, VesselStatic)
-        .join(
-            latest_times,
-            (VesselPosition.mmsi == latest_times.c.mmsi)
-            & (VesselPosition.received_at == latest_times.c.received_at),
-        )
-        .outerjoin(VesselStatic, VesselStatic.mmsi == VesselPosition.mmsi)
-        .order_by(VesselPosition.received_at.desc())
-    )
-    if limit and limit > 0:
-        stmt = stmt.limit(limit)
-
-    if parsed_bbox is not None:
-        lon_min, lat_min, lon_max, lat_max = parsed_bbox
-        stmt = stmt.where(
-            VesselPosition.lon >= lon_min,
-            VesselPosition.lon <= lon_max,
-            VesselPosition.lat >= lat_min,
-            VesselPosition.lat <= lat_max,
-        )
-
-    result = await db.execute(stmt)
-    rows = list(result.all())
-    geojson = convert_vessels_to_geojson(rows)
-    requested_types = {
-        item.strip().lower()
-        for item in (type or "").split(",")
-        if item.strip()
-    }
-    if requested_types:
-        geojson["features"] = [
-            feature
-            for feature in geojson.get("features", [])
-            if _matches_vessel_type(feature.get("properties", {}), requested_types)
-        ]
-
-    features = geojson.get("features", [])
-    return {
-        **geojson,
-        "count": len(features),
-        "stats": _build_vessel_stats(features),
-    }
-
-
-@router.get("/vessels/{mmsi}")
-async def get_vessel_detail(mmsi: int, db: AsyncSession = Depends(get_db)):
-    aggregated = await get_aggregated_vessel(db, mmsi)
-    if aggregated is not None:
-        return {
-            **aggregated,
-            "received_at": to_iso8601_utc(aggregated.get("received_at")),
-            "latitude": aggregated["lat"],
-            "longitude": aggregated["lon"],
-        }
-
-    latest_position_stmt = (
-        select(VesselPosition)
-        .where(VesselPosition.mmsi == mmsi)
-        .order_by(VesselPosition.received_at.desc())
-        .limit(1)
-    )
-    static = await db.get(VesselStatic, mmsi)
-    result = await db.execute(latest_position_stmt)
-    position = result.scalar_one_or_none()
-    if position is None:
-        raise HTTPException(status_code=404, detail="Vessel not found")
-    geojson = convert_vessels_to_geojson([(position, static)])
-    return {
-        **(geojson["features"][0]["properties"]),
-        "latitude": position.lat,
-        "longitude": position.lon,
-    }
-
-
-@router.get("/vessels/{mmsi}/track")
-async def get_vessel_track(
-    mmsi: int,
-    hours: int = Query(6, ge=1, le=24),
-    db: AsyncSession = Depends(get_db),
-):
-    cutoff = datetime.now(UTC) - timedelta(hours=hours)
-    aggregated_points = await get_aggregated_vessel_track(db, mmsi, cutoff=cutoff)
-    if aggregated_points:
-        return {
-            "type": "FeatureCollection",
-            "features": [
-                {
-                    "type": "Feature",
-                    "geometry": {
-                        "type": "LineString",
-                        "coordinates": [[point["lon"], point["lat"]] for point in aggregated_points],
-                    },
-                    "properties": {
-                        "mmsi": mmsi,
-                        "hours": hours,
-                        "point_count": len(aggregated_points),
-                        "start_at": to_iso8601_utc(aggregated_points[0]["observed_at"]),
-                        "end_at": to_iso8601_utc(aggregated_points[-1]["observed_at"]),
-                        "point_sources": [point["source"] for point in aggregated_points],
-                    },
-                }
-            ],
-            "count": 1,
-        }
-
-    result = await db.execute(
-        select(VesselPosition)
-        .where(VesselPosition.mmsi == mmsi)
-        .where(VesselPosition.received_at >= cutoff)
-        .order_by(VesselPosition.received_at.asc())
-    )
-    positions = list(result.scalars().all())
-    if not positions:
-        return {
-            "type": "FeatureCollection",
-            "features": [],
-            "count": 0,
-        }
-
-    return {
-        "type": "FeatureCollection",
-        "features": [
-            {
-                "type": "Feature",
-                "geometry": {
-                    "type": "LineString",
-                    "coordinates": [[position.lon, position.lat] for position in positions],
-                },
-                "properties": {
-                    "mmsi": mmsi,
-                    "hours": hours,
-                    "point_count": len(positions),
-                    "start_at": to_iso8601_utc(positions[0].received_at),
-                    "end_at": to_iso8601_utc(positions[-1].received_at),
-                },
-            }
-        ],
-        "count": 1,
-    }
-
-
-@router.get("/vessels/{mmsi}/observations")
-async def get_vessel_observations(
-    mmsi: int,
-    limit: int = Query(100, ge=1, le=500),
-    db: AsyncSession = Depends(get_db),
-):
-    """Return raw AIS observations for debugging source-level collector facts."""
-
-    observations = await get_vessel_raw_observations(db, mmsi, limit=limit)
-    return {
-        "mmsi": mmsi,
-        "count": len(observations),
-        "observations": [item.to_dict() for item in observations],
-        "conflict_candidates": build_field_conflict_candidates(observations),
-    }
-
-
-@router.get("/vessels/{mmsi}/conflicts")
-async def get_vessel_conflicts(mmsi: int, db: AsyncSession = Depends(get_db)):
-    """Return recorded AIS conflicts plus current raw-observation candidates."""
-
-    records = await get_vessel_conflict_records(db, mmsi)
-    observations = await get_vessel_raw_observations(db, mmsi, limit=500)
-    return {
-        "mmsi": mmsi,
-        "count": len(records),
-        "conflicts": [item.to_dict() for item in records],
-        "candidates": build_field_conflict_candidates(observations),
-    }
-
-
 @router.get("/geo/bgp-anomalies")
 async def get_bgp_anomalies_geojson(
     severity: Optional[str] = Query(None),
@@ -1734,76 +1030,6 @@ async def get_bgp_collectors_geojson(db: AsyncSession = Depends(get_db)):
     return {**geojson, "count": len(geojson.get("features", []))}
 
 
-@router.get("/geo/summary")
-async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db)):
-    """Return lightweight Earth HUD counts without loading layer GeoJSON payloads."""
-    records_by_source = await _load_current_collected_data_by_sources(
-        db,
-        [
-            "arcgis_cables",
-            "arcgis_landing_points",
-            "celestrak_tle",
-            "top500",
-            "epoch_ai_gpu",
-        ],
-    )
-
-    cables = convert_cable_to_geojson(records_by_source.get("arcgis_cables", []))
-    landing_points = convert_landing_point_to_geojson(
-        records_by_source.get("arcgis_landing_points", []),
-    )
-    satellites = convert_satellite_to_geojson(
-        _filter_known_records(records_by_source.get("celestrak_tle", [])),
-    )
-    compute_centers = convert_compute_centers_to_geojson(
-        _filter_known_records(
-            records_by_source.get("top500", [])
-            + records_by_source.get("epoch_ai_gpu", []),
-        ),
-    )
-    compute_features = compute_centers.get("features", [])
-
-    active_incident_result = await db.execute(
-        select(func.count(BGPIncident.id)).where(BGPIncident.status == "active"),
-    )
-    active_anomaly_result = await db.execute(
-        select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == "active"),
-    )
-    active_incident_count = int(active_incident_result.scalar() or 0)
-    active_anomaly_count = int(active_anomaly_result.scalar() or 0)
-    bgp_collectors = await build_bgp_collector_coverage(
-        db,
-        source_filter=("ris_live_bgp", "bgpstream_bgp"),
-    )
-    vessel_count_result = await db.execute(
-        select(func.count(func.distinct(VesselPosition.mmsi))),
-    )
-    vessel_count = int(vessel_count_result.scalar() or 0)
-
-    return {
-        "generated_at": to_iso8601_utc(datetime.now(UTC)),
-        "stats": {
-            "cable_count": len(cables.get("features", [])),
-            "landing_point_count": len(landing_points.get("features", [])),
-            "satellite_count": len(satellites.get("features", [])),
-            "compute_center_count": len(compute_features),
-            "vessel_count": vessel_count,
-            "supercomputer_count": sum(
-                1 for feature in compute_features
-                if feature.get("properties", {}).get("site_type") == "supercomputer"
-            ),
-            "gpu_cluster_count": sum(
-                1 for feature in compute_features
-                if feature.get("properties", {}).get("site_type") == "gpu_cluster"
-            ),
-            "bgp_event_count": active_incident_count or active_anomaly_count,
-            "bgp_incident_count": active_incident_count,
-            "bgp_anomaly_count": active_anomaly_count,
-            "bgp_collector_count": len([item for item in bgp_collectors if item.get("collector")]),
-        },
-    }
-
-
 @router.get("/all")
 async def get_all_visualization_data(db: AsyncSession = Depends(get_db)):
     """获取所有可视化数据的统一端点

backend/app/api/v1/websocket.py

@@ -1,7 +1,7 @@
 """WebSocket API endpoints"""
 
 import asyncio
-import json
+import logging
 from datetime import UTC, datetime
 from typing import Optional
 
@@ -9,12 +9,22 @@ from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
 from jose import jwt, JWTError
 
 from app.core.config import settings
-from app.core.logging import get_logger
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
+from app.core.websocket.ue_scene import expand_scene_payload_for_transport, ue_scene_state_store
+from app.db.session import async_session_factory
 
-logger = get_logger(__name__, service="api")
+logger = logging.getLogger(__name__)
 router = APIRouter()
+SUPPORTED_CHANNELS = [
+    "gpu_clusters",
+    "submarine_cables",
+    "ixp_nodes",
+    "alerts",
+    "dashboard",
+    "datasource_tasks",
+    "ue_scene",
+]
 
 
 async def authenticate_token(token: str) -> Optional[dict]:
@@ -22,18 +32,11 @@ async def authenticate_token(token: str) -> Optional[dict]:
     try:
         payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
         if payload.get("type") != "access":
-            logger.warning_event(
-                "WebSocket auth failed: wrong token type",
-                event="auth.websocket.invalid_token_type",
-            )
+            logger.warning(f"WebSocket auth failed: wrong token type")
             return None
         return payload
     except JWTError as e:
-        logger.warning_event(
-            "WebSocket auth failed",
-            event="auth.websocket.decode_failed",
-            context={"error": str(e)},
-        )
+        logger.warning(f"WebSocket auth failed: {e}")
         return None
 
 
@@ -43,17 +46,10 @@ async def websocket_endpoint(
     token: str = Query(...),
 ):
     """WebSocket endpoint for real-time data"""
-    logger.info_event(
-        "WebSocket connection attempt",
-        event="auth.websocket.connection_attempt",
-        context={"token_preview": f"{token[:8]}..."},
-    )
+    logger.info(f"WebSocket connection attempt with token: {token[:20]}...")
     payload = await authenticate_token(token)
     if payload is None:
-        logger.warning_event(
-            "WebSocket authentication failed, closing connection",
-            event="auth.websocket.connection_rejected",
-        )
+        logger.warning("WebSocket authentication failed, closing connection")
         await websocket.close(code=4001)
         return
 
@@ -64,18 +60,12 @@ async def websocket_endpoint(
         await websocket.send_json(
             {
                 "type": "connection_established",
+                "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "data": {
                     "connection_id": f"conn_{user_id}",
                     "server_version": settings.VERSION,
                     "heartbeat_interval": 30,
-                    "supported_channels": [
-                        "gpu_clusters",
-                        "submarine_cables",
-                        "ixp_nodes",
-                        "alerts",
-                        "dashboard",
-                        "datasource_tasks",
-                    ],
+                    "supported_channels": SUPPORTED_CHANNELS,
                 },
             }
         )
@@ -88,26 +78,79 @@ async def websocket_endpoint(
                     await websocket.send_json(
                         {
                             "type": "heartbeat",
+                            "timestamp": to_iso8601_utc(datetime.now(UTC)),
                             "data": {"action": "pong", "timestamp": to_iso8601_utc(datetime.now(UTC))},
                         }
                     )
                 elif data.get("type") == "subscribe":
-                    channels = data.get("data", {}).get("channels", [])
+                    requested_channels = data.get("data", {}).get("channels", [])
+                    channels = manager.subscribe(websocket, requested_channels)
                     await websocket.send_json(
                         {
                             "type": "subscription_confirmed",
+                            "timestamp": to_iso8601_utc(datetime.now(UTC)),
                             "data": {"action": "subscribe", "channels": channels},
                         }
                     )
+                elif data.get("type") == "sync_request":
+                    sync_data = data.get("data", {})
+                    channel = sync_data.get("channel")
+                    if channel == "ue_scene":
+                        async with async_session_factory() as session:
+                            scene_payloads = await ue_scene_state_store.get_sync_payloads(
+                                session,
+                                last_sequence=sync_data.get("last_sequence"),
+                                reason=sync_data.get("reason"),
+                            )
+                        for scene_payload in scene_payloads:
+                            for transport_payload in expand_scene_payload_for_transport(scene_payload):
+                                await websocket.send_json(
+                                    {
+                                        "type": "data_frame",
+                                        "channel": "ue_scene",
+                                        "timestamp": to_iso8601_utc(datetime.now(UTC)),
+                                        "data": transport_payload,
+                                    }
+                                )
+                    else:
+                        await websocket.send_json(
+                            {
+                                "type": "error",
+                                "timestamp": to_iso8601_utc(datetime.now(UTC)),
+                                "data": {
+                                    "message": f"Unsupported sync channel: {channel}",
+                                },
+                            }
+                        )
                 elif data.get("type") == "control_frame":
                     await websocket.send_json(
-                        {"type": "control_acknowledged", "data": {"received": True}}
+                        {
+                            "type": "control_acknowledged",
+                            "timestamp": to_iso8601_utc(datetime.now(UTC)),
+                            "data": {
+                                "target": data.get("data", {}).get("target"),
+                                "command": data.get("data", {}).get("command"),
+                                "accepted": True,
+                            },
+                        }
                     )
                 else:
-                    await websocket.send_json({"type": "ack", "data": {"received": True}})
+                    await websocket.send_json(
+                        {
+                            "type": "ack",
+                            "timestamp": to_iso8601_utc(datetime.now(UTC)),
+                            "data": {"received": True},
+                        }
+                    )
 
             except asyncio.TimeoutError:
-                await websocket.send_json({"type": "heartbeat", "data": {"action": "ping"}})
+                await websocket.send_json(
+                    {
+                        "type": "heartbeat",
+                        "timestamp": to_iso8601_utc(datetime.now(UTC)),
+                        "data": {"action": "ping"},
+                    }
+                )
 
     except WebSocketDisconnect:
         pass

backend/app/core/cache.py

@@ -1,15 +1,15 @@
 """Redis caching service"""
 
 import json
+import logging
 from datetime import timedelta
 from typing import Optional, Any
 
 import redis
 
 from app.core.config import settings
-from app.core.logging import get_logger
 
-logger = get_logger(__name__)
+logger = logging.getLogger(__name__)
 
 
 # Lazy Redis client initialization
@@ -47,7 +47,7 @@ class CacheService:
                 return json.loads(value)
             return None
         except Exception as e:
-            logger.warning_event("Cache get error", event="cache.get.failed", context={"error": str(e)})
+            logger.warning(f"Cache get error: {e}")
             return None
 
     def set(
@@ -61,7 +61,7 @@ class CacheService:
             serialized = json.dumps(value, default=str)
             return self.client.setex(key, expire_seconds, serialized)
         except Exception as e:
-            logger.warning_event("Cache set error", event="cache.set.failed", context={"error": str(e)})
+            logger.warning(f"Cache set error: {e}")
             return False
 
     def delete(self, key: str) -> bool:
@@ -69,7 +69,7 @@ class CacheService:
         try:
             return self.client.delete(key) > 0
         except Exception as e:
-            logger.warning_event("Cache delete error", event="cache.delete.failed", context={"error": str(e)})
+            logger.warning(f"Cache delete error: {e}")
             return False
 
     def delete_pattern(self, pattern: str) -> int:
@@ -80,7 +80,7 @@ class CacheService:
                 return self.client.delete(*keys)
             return 0
         except Exception as e:
-            logger.warning_event("Cache delete_pattern error", event="cache.delete_pattern.failed", context={"error": str(e)})
+            logger.warning(f"Cache delete_pattern error: {e}")
             return 0
 
     def get_or_set(

backend/app/core/data_sources.py

@@ -1,6 +1,7 @@
 import os
 import yaml
 from functools import lru_cache
+from typing import Optional
 
 
 COLLECTOR_URL_KEYS = {
@@ -30,8 +31,6 @@ COLLECTOR_URL_KEYS = {
     "opengeofeed_prefix_geo": "opengeofeed.public_csv_url",
     "nro_delegated_prefix_geo": "nro.delegated_stats_url",
     "news_live_streams": "news_live_streams.channels_url",
-    "barentswatch_vessels": "barentswatch_vessels.url",
-    "aisstream_vessels": "aisstream_vessels.url",
 }
 
 
@@ -74,7 +73,7 @@ class DataSourcesConfig:
             from app.models.datasource_config import DataSourceConfig
 
             query = select(DataSourceConfig).where(
-                DataSourceConfig.name == collector_name, DataSourceConfig.is_active
+                DataSourceConfig.name == collector_name, DataSourceConfig.is_active == True
             )
             result = await db.execute(query)
             db_config = result.scalar_one_or_none()

backend/app/core/data_sources.yaml

@@ -94,11 +94,3 @@ news_live_streams:
   streams_url: "https://iptv-org.github.io/api/streams.json"
   # IPTV-org 台标 JSON
   logos_url: "https://iptv-org.github.io/api/logos.json"
-
-barentswatch_vessels:
-  # BarentsWatch Live AIS latest combined endpoint. Requires an AIS bearer token.
-  url: "https://live.ais.barentswatch.no/v1/latest/combined"
-
-aisstream_vessels:
-  # AISStream realtime WebSocket endpoint. Requires an AISStream API key.
-  url: "wss://stream.aisstream.io/v0/stream"

backend/app/core/datasource_defaults.py

@@ -4,258 +4,163 @@ DEFAULT_DATASOURCES = {
     "top500": {
         "id": 1,
         "name": "TOP500 Supercomputers",
-        "display_name": "TOP500 超算榜单",
         "module": "L1",
         "priority": "P0",
         "frequency_minutes": 240,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "epoch_ai_gpu": {
         "id": 2,
         "name": "Epoch AI GPU Clusters",
-        "display_name": "Epoch AI GPU 集群",
         "module": "L1",
         "priority": "P0",
         "frequency_minutes": 360,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "huggingface_models": {
         "id": 3,
         "name": "HuggingFace Models",
-        "display_name": "Hugging Face 模型",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 720,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "huggingface_datasets": {
         "id": 4,
         "name": "HuggingFace Datasets",
-        "display_name": "Hugging Face 数据集",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 720,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "huggingface_spaces": {
         "id": 5,
         "name": "HuggingFace Spaces",
-        "display_name": "Hugging Face Spaces",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 1440,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "peeringdb_ixp": {
         "id": 6,
         "name": "PeeringDB IXP",
-        "display_name": "PeeringDB 交换中心",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 1440,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "peeringdb_network": {
         "id": 7,
         "name": "PeeringDB Networks",
-        "display_name": "PeeringDB 网络",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 2880,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "peeringdb_facility": {
         "id": 8,
         "name": "PeeringDB Facilities",
-        "display_name": "PeeringDB 设施",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 2880,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "telegeography_cables": {
         "id": 9,
         "name": "Submarine Cables",
-        "display_name": "海底光缆",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "telegeography_landing": {
         "id": 10,
         "name": "Cable Landing Points",
-        "display_name": "光缆登陆点",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 10080,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "telegeography_systems": {
         "id": 11,
         "name": "Cable Systems",
-        "display_name": "光缆系统",
         "module": "L2",
         "priority": "P2",
         "frequency_minutes": 10080,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "arcgis_cables": {
         "id": 15,
         "name": "ArcGIS Submarine Cables",
-        "display_name": "ArcGIS 海底光缆",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "arcgis_landing_points": {
         "id": 16,
         "name": "ArcGIS Landing Points",
-        "display_name": "ArcGIS 登陆点",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "arcgis_cable_landing_relation": {
         "id": 17,
         "name": "ArcGIS Cable-Landing Relations",
-        "display_name": "ArcGIS 光缆登陆关系",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "fao_landing_points": {
         "id": 18,
         "name": "FAO Landing Points",
-        "display_name": "FAO 登陆点",
         "module": "L2",
         "priority": "P1",
         "frequency_minutes": 10080,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "spacetrack_tle": {
         "id": 19,
         "name": "Space-Track TLE",
-        "display_name": "Space-Track 轨道根数",
         "module": "L3",
         "priority": "P2",
         "frequency_minutes": 1440,
-        "is_free": True,
-        "requires_credentials": True,
-        "credential_provider": "spacetrack",
-        "credential_status": "planned",
     },
     "celestrak_tle": {
         "id": 20,
         "name": "CelesTrak TLE",
-        "display_name": "CelesTrak 轨道根数",
         "module": "L3",
         "priority": "P2",
         "frequency_minutes": 1440,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "ris_live_bgp": {
         "id": 21,
         "name": "RIPE RIS Live BGP",
-        "display_name": "RIPE RIS 实时 BGP",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 15,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "bgpstream_bgp": {
         "id": 22,
         "name": "CAIDA BGPStream Backfill",
-        "display_name": "CAIDA BGPStream 回填",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 360,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "iptoasn_prefix_geo": {
         "id": 23,
         "name": "IPtoASN Prefix Geography",
-        "display_name": "IPtoASN 前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "opengeofeed_prefix_geo": {
         "id": 24,
         "name": "OpenGeoFeed Prefix Geography",
-        "display_name": "OpenGeoFeed 前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "nro_delegated_prefix_geo": {
         "id": 25,
         "name": "NRO Delegated Prefix Geography",
-        "display_name": "NRO 分配前缀地理",
         "module": "L3",
         "priority": "P1",
         "frequency_minutes": 1440,
-        "is_free": True,
-        "requires_credentials": False,
     },
     "news_live_streams": {
         "id": 26,
         "name": "News Live Streams",
-        "display_name": "新闻直播源",
         "module": "L4",
         "priority": "P2",
         "frequency_minutes": 720,
-        "is_free": True,
-        "requires_credentials": False,
-    },
-    "barentswatch_vessels": {
-        "id": 27,
-        "name": "BarentsWatch AIS Vessels",
-        "display_name": "BarentsWatch AIS 船舶",
-        "module": "L4",
-        "priority": "P1",
-        "frequency_minutes": 1,
-        "is_free": True,
-        "requires_credentials": True,
-        "credential_provider": "barentswatch",
-        "credential_status": "supported",
-    },
-    "aisstream_vessels": {
-        "id": 28,
-        "name": "AISStream Vessels",
-        "display_name": "AISStream 实时船舶",
-        "module": "L4",
-        "priority": "P1",
-        "frequency_minutes": 1,
-        "is_free": True,
-        "requires_credentials": True,
-        "credential_provider": "aisstream",
-        "credential_status": "supported",
     },
 }
 

backend/app/core/logging.py

@@ -1,161 +0,0 @@
-from __future__ import annotations
-
-import json
-import logging
-import os
-import re
-
-from collections.abc import Mapping, Sequence
-from typing import Any
-
-from app.core.request_context import get_request_id
-
-DEFAULT_SERVICE = "backend"
-DEFAULT_EVENT = "app.log"
-DEFAULT_LOG_LEVEL = os.getenv("PLANET_LOG_LEVEL", "INFO").upper()
-REDACTED = "[REDACTED]"
-SENSITIVE_FIELD_NAMES = {
-    "access_token",
-    "api_key",
-    "authorization",
-    "cookie",
-    "password",
-    "refresh_token",
-    "secret",
-    "token",
-}
-SENSITIVE_TEXT_PATTERNS = (
-    re.compile(r"(?i)(authorization\s*[:=]\s*)(.+)"),
-    re.compile(r"(?i)(bearer\s+)([A-Za-z0-9._\-]+)"),
-    re.compile(r"(?i)(token\s*[:=]\s*)(.+)"),
-    re.compile(r"(?i)(password\s*[:=]\s*)(.+)"),
-    re.compile(r"(?i)(cookie\s*[:=]\s*)(.+)"),
-)
-
-
-def sanitize_log_value(value: Any) -> Any:
-    if isinstance(value, Mapping):
-        return {
-            str(key): (REDACTED if str(key).lower() in SENSITIVE_FIELD_NAMES else sanitize_log_value(item))
-            for key, item in value.items()
-        }
-    if isinstance(value, Sequence) and not isinstance(value, (str, bytes, bytearray)):
-        return [sanitize_log_value(item) for item in value]
-    if isinstance(value, str):
-        sanitized = value
-        for pattern in SENSITIVE_TEXT_PATTERNS:
-            sanitized = pattern.sub(lambda match: f"{match.group(1)}{REDACTED}", sanitized)
-        return sanitized
-    return value
-
-
-def _normalize_context(context: Any) -> dict[str, Any]:
-    if context is None:
-        return {}
-    if isinstance(context, Mapping):
-        sanitized = sanitize_log_value(context)
-        return {str(key): value for key, value in sanitized.items()}
-    return {"value": sanitize_log_value(context)}
-
-
-class PlanetContextFilter(logging.Filter):
-    def filter(self, record: logging.LogRecord) -> bool:
-        record.request_id = getattr(record, "request_id", None) or get_request_id() or "-"
-        record.service = getattr(record, "service", None) or DEFAULT_SERVICE
-        record.event = getattr(record, "event", None) or DEFAULT_EVENT
-        record.context = _normalize_context(getattr(record, "context", None))
-        record.message = sanitize_log_value(record.getMessage())
-        return True
-
-
-class PlanetFormatter(logging.Formatter):
-    def format(self, record: logging.LogRecord) -> str:
-        timestamp = self.formatTime(record, self.datefmt)
-        level = record.levelname
-        service = getattr(record, "service", DEFAULT_SERVICE)
-        module_name = record.name
-        event = getattr(record, "event", DEFAULT_EVENT)
-        request_id = getattr(record, "request_id", "-")
-        message = sanitize_log_value(record.getMessage())
-        context = _normalize_context(getattr(record, "context", None))
-        context_suffix = ""
-        if context:
-            context_suffix = f" context={json.dumps(context, ensure_ascii=False, sort_keys=True)}"
-        rendered = (
-            f"{timestamp} {level} service={service} module={module_name} "
-            f"event={event} request_id={request_id} message={message}{context_suffix}"
-        )
-        if record.exc_info:
-            rendered = f"{rendered}\n{self.formatException(record.exc_info)}"
-        return rendered
-
-
-class PlanetLoggerAdapter(logging.LoggerAdapter):
-    def process(self, msg: Any, kwargs: dict[str, Any]) -> tuple[Any, dict[str, Any]]:
-        extra = dict(self.extra)
-        extra.update(kwargs.get("extra", {}))
-        if "context" in extra:
-            extra["context"] = _normalize_context(extra.get("context"))
-        kwargs["extra"] = extra
-        return sanitize_log_value(msg), kwargs
-
-    def log_event(
-        self,
-        level: int,
-        message: str,
-        *,
-        event: str,
-        context: Mapping[str, Any] | None = None,
-        **extra: Any,
-    ) -> None:
-        self.log(level, message, extra={"event": event, "context": context or {}, **extra})
-
-    def debug_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
-        self.log_event(logging.DEBUG, message, event=event, context=context, **extra)
-
-    def info_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
-        self.log_event(logging.INFO, message, event=event, context=context, **extra)
-
-    def warning_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
-        self.log_event(logging.WARNING, message, event=event, context=context, **extra)
-
-    def error_event(self, message: str, *, event: str, context: Mapping[str, Any] | None = None, **extra: Any) -> None:
-        self.log_event(logging.ERROR, message, event=event, context=context, **extra)
-
-    def exception_event(
-        self,
-        message: str,
-        *,
-        event: str,
-        context: Mapping[str, Any] | None = None,
-        **extra: Any,
-    ) -> None:
-        self.error(message, exc_info=True, extra={"event": event, "context": context or {}, **extra})
-
-
-def get_logger(name: str, *, service: str = DEFAULT_SERVICE) -> PlanetLoggerAdapter:
-    return PlanetLoggerAdapter(logging.getLogger(name), {"service": service})
-
-
-def configure_logging(level: str | None = None) -> None:
-    root_logger = logging.getLogger()
-    if getattr(configure_logging, "_configured", False):
-        if level:
-            root_logger.setLevel(level.upper())
-        return
-
-    handler = logging.StreamHandler()
-    handler.setFormatter(PlanetFormatter(datefmt="%Y-%m-%d %H:%M:%S"))
-    handler.addFilter(PlanetContextFilter())
-
-    root_logger.handlers.clear()
-    root_logger.addHandler(handler)
-    root_logger.setLevel((level or DEFAULT_LOG_LEVEL).upper())
-
-    for logger_name in ("uvicorn", "uvicorn.error", "uvicorn.access"):
-        target_logger = logging.getLogger(logger_name)
-        target_logger.handlers.clear()
-        target_logger.propagate = True
-
-    logging.captureWarnings(True)
-    configure_logging._configured = True

backend/app/core/request_context.py

@@ -1,14 +0,0 @@
-from __future__ import annotations
-
-from contextvars import ContextVar
-
-
-request_id_context: ContextVar[str | None] = ContextVar("request_id", default=None)
-
-
-def set_request_id(request_id: str | None) -> None:
-    request_id_context.set(request_id)
-
-
-def get_request_id() -> str | None:
-    return request_id_context.get()

backend/app/core/target_schema_registry.py

@@ -1,151 +0,0 @@
-"""Registry of target schemas supported by mapped custom data sources."""
-
-from __future__ import annotations
-
-from dataclasses import dataclass
-from datetime import datetime
-from typing import Any
-
-from pydantic import BaseModel, Field, ValidationError, field_validator
-
-
-class VesselAISRecord(BaseModel):
-    mmsi: int = Field(ge=100000000, le=999999999)
-    lat: float = Field(ge=-90, le=90)
-    lon: float = Field(ge=-180, le=180)
-    sog: float | None = None
-    cog: float | None = Field(default=None, ge=0, le=360)
-    heading: int | None = Field(default=None, ge=0, le=511)
-    name: str | None = None
-    vessel_type: str | int | None = None
-    received_at: datetime | None = None
-
-
-class GeoPointRecord(BaseModel):
-    lat: float = Field(ge=-90, le=90)
-    lon: float = Field(ge=-180, le=180)
-    name: str | None = None
-    type: str | None = None
-    source_id: str | None = None
-    observed_at: datetime | None = None
-    metadata: dict[str, Any] = Field(default_factory=dict)
-
-
-class GenericRecord(BaseModel):
-    data: dict[str, Any] = Field(default_factory=dict)
-    source_id: str | None = None
-    observed_at: datetime | None = None
-
-    @field_validator("data")
-    @classmethod
-    def require_payload(cls, value: dict[str, Any]) -> dict[str, Any]:
-        if not value:
-            raise ValueError("generic_records requires a non-empty data object")
-        return value
-
-
-@dataclass(frozen=True)
-class TargetField:
-    name: str
-    type: str
-    required: bool = False
-    description: str = ""
-    example: Any = None
-
-    def to_dict(self) -> dict[str, Any]:
-        return {
-            "name": self.name,
-            "type": self.type,
-            "required": self.required,
-            "description": self.description,
-            "example": self.example,
-        }
-
-
-@dataclass(frozen=True)
-class TargetSchema:
-    key: str
-    label: str
-    description: str
-    fields: tuple[TargetField, ...]
-    model: type[BaseModel]
-    destination: str
-
-    def to_dict(self) -> dict[str, Any]:
-        return {
-            "key": self.key,
-            "label": self.label,
-            "description": self.description,
-            "destination": self.destination,
-            "fields": [field.to_dict() for field in self.fields],
-        }
-
-    def validate_record(self, record: dict[str, Any]) -> tuple[dict[str, Any] | None, list[str]]:
-        try:
-            return self.model.model_validate(record).model_dump(mode="json"), []
-        except ValidationError as exc:
-            return None, [
-                ".".join(str(part) for part in error["loc"]) + f": {error['msg']}"
-                for error in exc.errors()
-            ]
-
-
-TARGET_SCHEMAS: dict[str, TargetSchema] = {
-    "vessel_ais": TargetSchema(
-        key="vessel_ais",
-        label="船舶 AIS",
-        description="船只位置、航速、航向、MMSI 等 AIS 数据。",
-        destination="vessel_position",
-        model=VesselAISRecord,
-        fields=(
-            TargetField("mmsi", "integer", True, "MMSI 九位船舶标识", 257123000),
-            TargetField("lat", "float", True, "纬度", 59.91),
-            TargetField("lon", "float", True, "经度", 10.75),
-            TargetField("sog", "float", False, "对地航速，单位节", 12.4),
-            TargetField("cog", "float", False, "对地航向，0-360 度", 184.5),
-            TargetField("heading", "integer", False, "船首向，0-511", 186),
-            TargetField("name", "string", False, "船名", "OSLO EXPRESS"),
-            TargetField("vessel_type", "string", False, "船型", "cargo"),
-            TargetField("received_at", "datetime", False, "数据接收时间", "2026-04-28T00:00:00Z"),
-        ),
-    ),
-    "geo_points": TargetSchema(
-        key="geo_points",
-        label="通用地理点",
-        description="带经纬度的通用实体或事件点位。",
-        destination="generic_geo_points",
-        model=GeoPointRecord,
-        fields=(
-            TargetField("lat", "float", True, "纬度", 1.3),
-            TargetField("lon", "float", True, "经度", 103.8),
-            TargetField("name", "string", False, "点位名称", "Singapore"),
-            TargetField("type", "string", False, "点位类型", "datacenter"),
-            TargetField("source_id", "string", False, "来源侧 ID", "sg-1"),
-            TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
-            TargetField("metadata", "object", False, "扩展字段", {"provider": "example"}),
-        ),
-    ),
-    "generic_records": TargetSchema(
-        key="generic_records",
-        label="通用结构化记录",
-        description="未知结构数据沉淀，不直接进入 Earth 图层。",
-        destination="collected_data",
-        model=GenericRecord,
-        fields=(
-            TargetField("data", "object", True, "结构化记录主体", {"raw": "value"}),
-            TargetField("source_id", "string", False, "来源侧 ID", "record-1"),
-            TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
-        ),
-    ),
-}
-
-
-def list_target_schemas() -> list[dict[str, Any]]:
-    return [schema.to_dict() for schema in TARGET_SCHEMAS.values()]
-
-
-def get_target_schema(key: str) -> TargetSchema:
-    try:
-        return TARGET_SCHEMAS[key]
-    except KeyError as exc:
-        raise ValueError(f"Unsupported target schema: {key}") from exc

backend/app/core/websocket/broadcaster.py

@@ -1,11 +1,12 @@
-"""Data broadcaster for WebSocket connections"""
+"""Data broadcaster for WebSocket connections."""
 
 import asyncio
 from datetime import UTC, datetime
-from typing import Dict, Any, Optional
+from typing import Any, Dict
 
 from app.core.time import to_iso8601_utc
 from app.core.websocket.manager import manager
+from app.db.session import async_session_factory
 
 
 
@@ -45,6 +46,30 @@ class DataBroadcaster:
                 pass
             await asyncio.sleep(interval)
 
+    async def broadcast_ue_scene(self, interval: int = 5) -> None:
+        """Broadcast UE scene updates for nDisplay primary nodes."""
+        from app.core.websocket.ue_scene import expand_scene_payload_for_transport, ue_scene_state_store
+
+        while self.running:
+            try:
+                async with async_session_factory() as session:
+                    scene_payloads = await ue_scene_state_store.get_broadcast_payloads(session)
+
+                for scene_data in scene_payloads:
+                    for transport_payload in expand_scene_payload_for_transport(scene_data):
+                        await manager.broadcast(
+                            {
+                                "type": "data_frame",
+                                "channel": "ue_scene",
+                                "timestamp": to_iso8601_utc(datetime.now(UTC)),
+                                "data": transport_payload,
+                            },
+                            channel="ue_scene",
+                        )
+            except Exception:
+                pass
+            await asyncio.sleep(interval)
+
     async def broadcast_alert(self, alert: Dict[str, Any]):
         """Broadcast an alert to all connected clients"""
         await manager.broadcast(
@@ -75,7 +100,7 @@ class DataBroadcaster:
                 "timestamp": to_iso8601_utc(datetime.now(UTC)),
                 "payload": data,
             },
-            channel=channel if channel in manager.active_connections else "all",
+            channel=channel,
         )
 
     async def broadcast_datasource_task_update(self, data: Dict[str, Any]):
@@ -95,6 +120,7 @@ class DataBroadcaster:
         if not self.running:
             self.running = True
             self.tasks["dashboard"] = asyncio.create_task(self.broadcast_stats(5))
+            self.tasks["ue_scene"] = asyncio.create_task(self.broadcast_ue_scene(5))
 
     def stop(self):
         """Stop all broadcasters"""

backend/app/core/websocket/manager.py

@@ -1,9 +1,7 @@
-"""WebSocket Connection Manager"""
+"""WebSocket connection manager with channel subscriptions."""
+
+from typing import Dict, Optional, Set
 
-import json
-import asyncio
-from typing import Dict, Set, Optional
-from datetime import datetime
 from fastapi import WebSocket
 import redis.asyncio as redis
 
@@ -11,17 +9,25 @@ from app.core.config import settings
 
 
 class ConnectionManager:
-    """Manages WebSocket connections"""
+    """Manage user connections and channel subscriptions."""
 
-    def __init__(self):
-        self.active_connections: Dict[str, Set[WebSocket]] = {}  # user_id -> connections
+    def __init__(self) -> None:
+        self.user_connections: Dict[str, Set[WebSocket]] = {}
+        self.socket_users: Dict[WebSocket, str] = {}
+        self.channel_connections: Dict[str, Set[WebSocket]] = {}
+        self.socket_channels: Dict[WebSocket, Set[str]] = {}
         self.redis_client: Optional[redis.Redis] = None
 
-    async def connect(self, websocket: WebSocket, user_id: str):
+    @property
+    def active_connections(self) -> Dict[str, Set[WebSocket]]:
+        """Compatibility alias for existing callers."""
+        return self.user_connections
+
+    async def connect(self, websocket: WebSocket, user_id: str) -> None:
         await websocket.accept()
-        if user_id not in self.active_connections:
-            self.active_connections[user_id] = set()
-        self.active_connections[user_id].add(websocket)
+        self.user_connections.setdefault(user_id, set()).add(websocket)
+        self.socket_users[websocket] = user_id
+        self.socket_channels.setdefault(websocket, set())
 
         if self.redis_client is None:
             redis_url = settings.REDIS_URL
@@ -35,32 +41,69 @@ class ConnectionManager:
                     decode_responses=True,
                 )
 
-    def disconnect(self, websocket: WebSocket, user_id: str):
-        if user_id in self.active_connections:
-            self.active_connections[user_id].discard(websocket)
-            if not self.active_connections[user_id]:
-                del self.active_connections[user_id]
+    def disconnect(self, websocket: WebSocket, user_id: str) -> None:
+        self.unsubscribe(websocket, list(self.socket_channels.get(websocket, set())))
 
-    async def send_personal_message(self, message: dict, user_id: str):
-        if user_id in self.active_connections:
-            for connection in self.active_connections[user_id]:
-                try:
-                    await connection.send_json(message)
-                except Exception:
-                    pass
+        if user_id in self.user_connections:
+            self.user_connections[user_id].discard(websocket)
+            if not self.user_connections[user_id]:
+                del self.user_connections[user_id]
 
-    async def broadcast(self, message: dict, channel: str = "all"):
+        self.socket_users.pop(websocket, None)
+        self.socket_channels.pop(websocket, None)
+
+    def subscribe(self, websocket: WebSocket, channels: list[str]) -> list[str]:
+        subscribed_channels: list[str] = []
+        socket_channel_set = self.socket_channels.setdefault(websocket, set())
+
+        for channel in channels:
+            normalized_channel = channel.strip()
+            if not normalized_channel:
+                continue
+            self.channel_connections.setdefault(normalized_channel, set()).add(websocket)
+            socket_channel_set.add(normalized_channel)
+            subscribed_channels.append(normalized_channel)
+
+        return subscribed_channels
+
+    def unsubscribe(self, websocket: WebSocket, channels: list[str]) -> None:
+        socket_channel_set = self.socket_channels.setdefault(websocket, set())
+
+        for channel in channels:
+            normalized_channel = channel.strip()
+            if not normalized_channel:
+                continue
+            if normalized_channel in self.channel_connections:
+                self.channel_connections[normalized_channel].discard(websocket)
+                if not self.channel_connections[normalized_channel]:
+                    del self.channel_connections[normalized_channel]
+            socket_channel_set.discard(normalized_channel)
+
+    async def send_personal_message(self, message: dict, user_id: str) -> None:
+        for connection in list(self.user_connections.get(user_id, set())):
+            try:
+                await connection.send_json(message)
+            except Exception:
+                self.disconnect(connection, user_id)
+
+    async def broadcast(self, message: dict, channel: str = "all") -> None:
         if channel == "all":
-            for user_id in self.active_connections:
-                await self.send_personal_message(message, user_id)
+            targets = list(self.socket_users.keys())
         else:
-            await self.send_personal_message(message, channel)
+            targets = list(self.channel_connections.get(channel, set()))
 
-    async def close_all(self):
-        for user_id in self.active_connections:
-            for connection in self.active_connections[user_id]:
-                await connection.close()
-        self.active_connections.clear()
+        for connection in targets:
+            try:
+                await connection.send_json(message)
+            except Exception:
+                user_id = self.socket_users.get(connection)
+                if user_id is not None:
+                    self.disconnect(connection, user_id)
+
+    async def close_all(self) -> None:
+        for websocket, user_id in list(self.socket_users.items()):
+            await websocket.close()
+            self.disconnect(websocket, user_id)
 
 
 manager = ConnectionManager()

backend/app/core/websocket/ue_scene.py

@@ -0,0 +1,562 @@
+"""UE scene state built from visualization aggregate output."""
+
+import asyncio
+import json
+import zlib
+from datetime import UTC, datetime
+from typing import TYPE_CHECKING, Any, Dict, List
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.core.time import to_iso8601_utc
+from app.models.bgp_anomaly import BGPAnomaly
+from app.models.bgp_incident import BGPIncident
+
+if TYPE_CHECKING:
+    from app.models.collected_data import CollectedData
+
+DEFAULT_LAYER_ORDER = (
+    "satellites",
+    "supercomputers",
+    "gpu_clusters",
+    "submarine_cables",
+    "landing_points",
+    "bgp_anomalies",
+    "bgp_incidents",
+    "bgp_collectors",
+    "alerts",
+)
+
+FULL_RESYNC_REASONS = {"initial_connect", "manual_resync", "profile_changed"}
+UE_SCENE_LAYER_CHUNK_ITEM_LIMITS = {
+    "satellites": 500,
+    "supercomputers": 200,
+    "gpu_clusters": 100,
+    "submarine_cables": 25,
+    "landing_points": 250,
+    "bgp_anomalies": 100,
+    "bgp_incidents": 100,
+    "bgp_collectors": 100,
+    "alerts": 100,
+}
+
+
+def _default_display_profile() -> Dict[str, Any]:
+    return {
+        "profile_id": "polarized-wall-a",
+        "stereo_mode": "polarized",
+        "screen_width_m": 3.0,
+        "screen_height_m": 2.0,
+        "target_refresh_hz": 120,
+    }
+
+
+def _default_camera_state() -> Dict[str, Any]:
+    return {
+        "mode": "auto_cruise",
+        "path_id": "global_overview",
+        "fov": 42.0,
+    }
+
+
+def _stable_json(value: Any) -> str:
+    return json.dumps(value, ensure_ascii=True, sort_keys=True, separators=(",", ":"))
+
+
+def _entity_type_for_layer(layer_name: str) -> str:
+    return {
+        "satellites": "satellite",
+        "supercomputers": "supercomputer",
+        "gpu_clusters": "gpu_cluster",
+        "submarine_cables": "submarine_cable",
+        "landing_points": "landing_point",
+        "bgp_anomalies": "bgp_anomaly",
+        "bgp_incidents": "bgp_incident",
+        "bgp_collectors": "bgp_collector",
+        "alerts": "alert",
+    }[layer_name]
+
+
+def _default_visual_for_layer(layer_name: str) -> Dict[str, Any]:
+    visuals = {
+        "satellites": {"style": "satellite_marker", "size": 0.7, "color": "#9BDBFF"},
+        "supercomputers": {"style": "supercomputer_marker", "size": 1.2, "color": "#FF6B6B"},
+        "gpu_clusters": {"style": "pulse_marker", "size": 1.0, "color": "#FF8C42"},
+        "submarine_cables": {"style": "cable_arc", "width": 2.0, "color": "#4ECDC4"},
+        "landing_points": {"style": "landing_point_marker", "size": 0.8, "color": "#45B7D1"},
+        "bgp_anomalies": {"style": "anomaly_marker", "size": 1.0, "color": "#FFB703"},
+        "bgp_incidents": {"style": "incident_marker", "size": 1.2, "color": "#E63946"},
+        "bgp_collectors": {"style": "collector_marker", "size": 0.9, "color": "#7B9ACC"},
+        "alerts": {"style": "alert_marker", "size": 1.0, "color": "#FFD166"},
+    }
+    return visuals[layer_name]
+
+
+def _empty_scene_layers() -> Dict[str, Dict[str, Any]]:
+    return {
+        layer_name: {"revision": 0, "items": {}}
+        for layer_name in DEFAULT_LAYER_ORDER
+    }
+
+
+def _empty_changes() -> Dict[str, Dict[str, List[Any]]]:
+    return {
+        layer_name: {"added": [], "updated": [], "removed": []}
+        for layer_name in DEFAULT_LAYER_ORDER
+    }
+
+
+def _point_geo(coordinates: List[Any]) -> Dict[str, Any]:
+    lng = coordinates[0] if len(coordinates) > 0 else None
+    lat = coordinates[1] if len(coordinates) > 1 else None
+    alt = coordinates[2] if len(coordinates) > 2 else 0.0
+    return {"lat": lat, "lng": lng, "alt": alt}
+
+
+def _path_geo(geometry_type: str, coordinates: Any) -> Dict[str, Any]:
+    if geometry_type == "LineString":
+        return {
+            "path": [
+                {"lat": point[1], "lng": point[0], "alt": point[2] if len(point) > 2 else 0.0}
+                for point in coordinates
+                if isinstance(point, list) and len(point) >= 2
+            ]
+        }
+
+    if geometry_type == "MultiLineString":
+        return {
+            "segments": [
+                [
+                    {"lat": point[1], "lng": point[0], "alt": point[2] if len(point) > 2 else 0.0}
+                    for point in line
+                    if isinstance(point, list) and len(point) >= 2
+                ]
+                for line in coordinates
+                if isinstance(line, list)
+            ]
+        }
+
+    return {}
+
+
+def _item_identifier(layer_name: str, feature: Dict[str, Any], index: int) -> str:
+    properties = feature.get("properties") or {}
+    feature_id = feature.get("id") or properties.get("id") or properties.get("source_id")
+    return f"{layer_name}:{feature_id or index}"
+
+
+def _item_revision(item: Dict[str, Any]) -> int:
+    return zlib.crc32(_stable_json(item).encode("utf-8")) & 0xFFFFFFFF
+
+
+def _layer_revision(items: Dict[str, Dict[str, Any]]) -> int:
+    if not items:
+        return 0
+    joined = "|".join(
+        f"{item_id}:{items[item_id]['revision']}"
+        for item_id in sorted(items)
+    )
+    return zlib.crc32(joined.encode("utf-8")) & 0xFFFFFFFF
+
+
+def _serialize_feature(layer_name: str, feature: Dict[str, Any], index: int) -> Dict[str, Any]:
+    properties = dict(feature.get("properties") or {})
+    geometry = feature.get("geometry") or {}
+    geometry_type = geometry.get("type", "")
+    coordinates = geometry.get("coordinates") or []
+    item_id = _item_identifier(layer_name, feature, index)
+
+    geo: Dict[str, Any] = {}
+    if geometry_type == "Point":
+        geo = _point_geo(coordinates)
+    elif geometry_type in {"LineString", "MultiLineString"}:
+        geo = _path_geo(geometry_type, coordinates)
+
+    title = (
+        properties.get("name")
+        or properties.get("Name")
+        or properties.get("title")
+        or item_id
+    )
+    subtitle_parts = [
+        properties.get("city"),
+        properties.get("country"),
+        properties.get("region"),
+    ]
+    item = {
+        "id": item_id,
+        "entity_type": _entity_type_for_layer(layer_name),
+        "geo": geo,
+        "visual": {
+            **_default_visual_for_layer(layer_name),
+            **({"color": properties["color"]} if properties.get("color") else {}),
+        },
+        "metrics": properties,
+        "labels": {
+            "title": title,
+            "subtitle": ", ".join([part for part in subtitle_parts if part]),
+        },
+        "status": {
+            "health": properties.get("status", "normal"),
+            "alert_level": properties.get("severity", "none"),
+        },
+    }
+    item["revision"] = _item_revision(item)
+    return item
+
+
+async def build_visualization_scene_state(db: AsyncSession) -> Dict[str, Any]:
+    from app.api.v1.visualization import (
+        _build_landing_point_cable_maps,
+        _filter_known_records,
+        _load_current_collected_data_by_sources,
+        build_anomaly_geography_hints,
+        build_incident_geography_hints,
+        convert_bgp_anomalies_to_geojson,
+        convert_bgp_collectors_to_geojson,
+        convert_bgp_incidents_to_geojson,
+        convert_cable_to_geojson,
+        convert_gpu_cluster_to_geojson,
+        convert_landing_point_to_geojson,
+        convert_satellite_to_geojson,
+        convert_supercomputer_to_geojson,
+    )
+    from app.services.bgp_collectors import build_bgp_collector_coverage
+
+    records_by_source = await _load_current_collected_data_by_sources(
+        db,
+        [
+            "arcgis_cables",
+            "arcgis_landing_points",
+            "arcgis_cable_landing_relation",
+            "celestrak_tle",
+            "top500",
+            "epoch_ai_gpu",
+        ],
+    )
+
+    cables_records = records_by_source.get("arcgis_cables", [])
+    landing_point_records = records_by_source.get("arcgis_landing_points", [])
+    relation_records = records_by_source.get("arcgis_cable_landing_relation", [])
+    satellites_records = _filter_known_records(records_by_source.get("celestrak_tle", []))
+    supercomputer_records = _filter_known_records(records_by_source.get("top500", []))
+    gpu_records = _filter_known_records(records_by_source.get("epoch_ai_gpu", []))
+    bgp_anomalies_result = await db.execute(
+        select(BGPAnomaly)
+        .where(BGPAnomaly.status == "active")
+        .order_by(BGPAnomaly.created_at.desc())
+        .limit(200)
+    )
+    bgp_anomalies = list(bgp_anomalies_result.scalars().all())
+    bgp_anomaly_geography_hints = await build_anomaly_geography_hints(db, bgp_anomalies)
+    bgp_incidents_result = await db.execute(
+        select(BGPIncident)
+        .where(BGPIncident.status == "active")
+        .order_by(BGPIncident.created_at.desc())
+        .limit(100)
+    )
+    bgp_incidents = list(bgp_incidents_result.scalars().all())
+    bgp_incident_geography_hints = await build_incident_geography_hints(db, bgp_incidents)
+    bgp_collector_coverage = await build_bgp_collector_coverage(
+        db,
+        source_filter=("ris_live_bgp", "bgpstream_bgp"),
+    )
+    bgp_coverage_by_collector = {
+        item["collector"]: item
+        for item in bgp_collector_coverage
+        if item.get("collector")
+    }
+
+    city_to_cable_ids_map, cable_id_to_name_map = _build_landing_point_cable_maps(
+        relation_records,
+        cables_records,
+    )
+
+    aggregate = {
+        "satellites": convert_satellite_to_geojson(satellites_records),
+        "supercomputers": convert_supercomputer_to_geojson(supercomputer_records),
+        "gpu_clusters": convert_gpu_cluster_to_geojson(gpu_records),
+        "submarine_cables": convert_cable_to_geojson(cables_records),
+        "landing_points": convert_landing_point_to_geojson(
+            landing_point_records,
+            city_to_cable_ids_map,
+            cable_id_to_name_map,
+        ),
+        "bgp_anomalies": convert_bgp_anomalies_to_geojson(
+            bgp_anomalies,
+            bgp_anomaly_geography_hints,
+        ),
+        "bgp_incidents": convert_bgp_incidents_to_geojson(
+            bgp_incidents,
+            bgp_incident_geography_hints,
+        ),
+        "bgp_collectors": convert_bgp_collectors_to_geojson(bgp_coverage_by_collector),
+        "alerts": {"type": "FeatureCollection", "features": []},
+    }
+
+    layers = _empty_scene_layers()
+    total_records = 0
+    for layer_name in DEFAULT_LAYER_ORDER:
+        feature_collection = aggregate.get(layer_name) or {"features": []}
+        items = {
+            item["id"]: item
+            for index, feature in enumerate(feature_collection.get("features", []), start=1)
+            for item in [_serialize_feature(layer_name, feature, index)]
+        }
+        layers[layer_name]["items"] = items
+        layers[layer_name]["revision"] = _layer_revision(items)
+        total_records += len(items)
+
+    timestamp = to_iso8601_utc(datetime.now(UTC))
+    state_hash = zlib.crc32(
+        _stable_json(
+            {
+                layer_name: {
+                    item_id: item["revision"]
+                    for item_id, item in layers[layer_name]["items"].items()
+                }
+                for layer_name in DEFAULT_LAYER_ORDER
+            }
+        ).encode("utf-8")
+    ) & 0xFFFFFFFF
+
+    return {
+        "generated_at": timestamp,
+        "state_hash": state_hash,
+        "total_records": total_records,
+        "layers": layers,
+    }
+
+
+def _changes_exist(changes: Dict[str, Dict[str, List[Any]]]) -> bool:
+    return any(
+        layer_changes["added"] or layer_changes["updated"] or layer_changes["removed"]
+        for layer_changes in changes.values()
+    )
+
+
+def build_incremental_changes(
+    previous_state: Dict[str, Any],
+    current_state: Dict[str, Any],
+) -> Dict[str, Dict[str, List[Any]]]:
+    changes = _empty_changes()
+
+    for layer_name in DEFAULT_LAYER_ORDER:
+        previous_items = previous_state["layers"][layer_name]["items"]
+        current_items = current_state["layers"][layer_name]["items"]
+        previous_ids = set(previous_items)
+        current_ids = set(current_items)
+
+        for added_id in sorted(current_ids - previous_ids):
+            changes[layer_name]["added"].append(current_items[added_id])
+
+        for removed_id in sorted(previous_ids - current_ids):
+            changes[layer_name]["removed"].append(removed_id)
+
+        for common_id in sorted(previous_ids & current_ids):
+            if _stable_json(previous_items[common_id]) != _stable_json(current_items[common_id]):
+                changes[layer_name]["updated"].append(current_items[common_id])
+
+    return changes
+
+
+def _full_payload_from_state(state: Dict[str, Any], sequence: int) -> Dict[str, Any]:
+    return {
+        "update_type": "full",
+        "sequence": sequence,
+        "cluster_time": state["generated_at"],
+        "display_profile": _default_display_profile(),
+        "camera_state": _default_camera_state(),
+        "payload": {
+            "meta": {
+                "generated_at": state["generated_at"],
+                "total_records": state["total_records"],
+                "state_hash": state["state_hash"],
+            },
+            "layers": {
+                layer_name: {
+                    "revision": layer_data["revision"],
+                    "items": [
+                        layer_data["items"][item_id]
+                        for item_id in sorted(layer_data["items"])
+                    ],
+                }
+                for layer_name, layer_data in state["layers"].items()
+            },
+        },
+    }
+
+
+def _incremental_payload(
+    *,
+    current_state: Dict[str, Any],
+    sequence: int,
+    base_sequence: int,
+    changes: Dict[str, Dict[str, List[Any]]],
+) -> Dict[str, Any]:
+    return {
+        "update_type": "incremental",
+        "sequence": sequence,
+        "base_sequence": base_sequence,
+        "cluster_time": current_state["generated_at"],
+        "changes": changes,
+        "meta": {
+            "generated_at": current_state["generated_at"],
+            "total_records": current_state["total_records"],
+            "state_hash": current_state["state_hash"],
+        },
+    }
+
+
+def _noop_incremental_payload(state: Dict[str, Any], sequence: int) -> Dict[str, Any]:
+    return _incremental_payload(
+        current_state=state,
+        sequence=sequence,
+        base_sequence=sequence,
+        changes=_empty_changes(),
+    )
+
+
+def expand_scene_payload_for_transport(scene_payload: Dict[str, Any]) -> List[Dict[str, Any]]:
+    """Split oversized scene payloads into smaller transport-safe chunks."""
+    update_type = scene_payload.get("update_type")
+    if update_type != "full":
+        return [scene_payload]
+
+    payload = scene_payload.get("payload") or {}
+    layers = payload.get("layers") or {}
+    if not layers:
+        return [scene_payload]
+
+    chunks: List[Dict[str, Any]] = []
+    for layer_name in DEFAULT_LAYER_ORDER:
+        layer_data = layers.get(layer_name) or {}
+        items = list(layer_data.get("items") or [])
+        if not items:
+            continue
+
+        chunk_size = UE_SCENE_LAYER_CHUNK_ITEM_LIMITS.get(layer_name, 100)
+        total_layer_chunks = max(1, (len(items) + chunk_size - 1) // chunk_size)
+        for chunk_index, offset in enumerate(range(0, len(items), chunk_size), start=1):
+            chunk_items = items[offset : offset + chunk_size]
+            chunks.append(
+                {
+                    **scene_payload,
+                    "payload": {
+                        "meta": {
+                            **(payload.get("meta") or {}),
+                            "chunked": True,
+                            "layer_name": layer_name,
+                            "layer_chunk_index": chunk_index,
+                            "layer_chunk_count": total_layer_chunks,
+                        },
+                        "layers": {
+                            layer_name: {
+                                "revision": layer_data.get("revision", 0),
+                                "items": chunk_items,
+                            }
+                        },
+                    },
+                }
+            )
+
+    if not chunks:
+        return [scene_payload]
+
+    total_chunks = len(chunks)
+    for transport_index, chunk in enumerate(chunks, start=1):
+        chunk_payload = chunk.setdefault("payload", {})
+        chunk_meta = chunk_payload.setdefault("meta", {})
+        chunk_meta["transport_chunk_index"] = transport_index
+        chunk_meta["transport_chunk_count"] = total_chunks
+
+    return chunks
+
+
+class UeSceneStateStore:
+    """Maintain cached ue_scene state and a bounded incremental replay history."""
+
+    def __init__(self, history_limit: int = 20) -> None:
+        self.history_limit = history_limit
+        self.sequence = 0
+        self.state: Dict[str, Any] | None = None
+        self.history: List[Dict[str, Any]] = []
+        self.lock = asyncio.Lock()
+
+    async def _refresh_locked(self, db: AsyncSession) -> List[Dict[str, Any]]:
+        current_state = await build_visualization_scene_state(db)
+
+        if self.state is None:
+            self.sequence = 1
+            self.state = current_state
+            return [_full_payload_from_state(self.state, self.sequence)]
+
+        if current_state["state_hash"] == self.state["state_hash"]:
+            self.state = current_state
+            return []
+
+        previous_sequence = self.sequence
+        previous_state = self.state
+        self.sequence += 1
+        self.state = current_state
+        changes = build_incremental_changes(previous_state, current_state)
+        incremental = _incremental_payload(
+            current_state=current_state,
+            sequence=self.sequence,
+            base_sequence=previous_sequence,
+            changes=changes,
+        )
+        self.history.append(incremental)
+        if len(self.history) > self.history_limit:
+            self.history = self.history[-self.history_limit :]
+        return [incremental]
+
+    async def get_broadcast_payloads(self, db: AsyncSession) -> List[Dict[str, Any]]:
+        async with self.lock:
+            payloads = await self._refresh_locked(db)
+            return [payload for payload in payloads if payload["update_type"] == "full" or _changes_exist(payload.get("changes", {}))]
+
+    async def get_sync_payloads(
+        self,
+        db: AsyncSession,
+        *,
+        last_sequence: int | None,
+        reason: str | None,
+    ) -> List[Dict[str, Any]]:
+        async with self.lock:
+            await self._refresh_locked(db)
+            if self.state is None:
+                return []
+
+            normalized_reason = (reason or "").strip()
+            if last_sequence is None or normalized_reason in FULL_RESYNC_REASONS:
+                return [_full_payload_from_state(self.state, self.sequence)]
+
+            if last_sequence == self.sequence:
+                return [_noop_incremental_payload(self.state, self.sequence)]
+
+            if last_sequence > self.sequence:
+                return [_full_payload_from_state(self.state, self.sequence)]
+
+            replay_payloads = [
+                payload
+                for payload in self.history
+                if payload["base_sequence"] >= last_sequence
+                and payload["sequence"] > last_sequence
+            ]
+            if replay_payloads:
+                expected_base = last_sequence
+                ordered_payloads: List[Dict[str, Any]] = []
+                for payload in replay_payloads:
+                    if payload["base_sequence"] != expected_base:
+                        return [_full_payload_from_state(self.state, self.sequence)]
+                    ordered_payloads.append(payload)
+                    expected_base = payload["sequence"]
+                if ordered_payloads and ordered_payloads[-1]["sequence"] == self.sequence:
+                    return ordered_payloads
+
+            return [_full_payload_from_state(self.state, self.sequence)]
+
+
+ue_scene_state_store = UeSceneStateStore()

backend/app/db/session.py

@@ -5,22 +5,10 @@ from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sess
 from sqlalchemy.orm import declarative_base
 
 from app.core.config import settings
-from app.core.logging import get_logger
-
-logger = get_logger(__name__)
-
-DB_POOL_CONFIG = {
-    "pool_pre_ping": True,
-    "pool_recycle": 1800,
-    "pool_size": 10,
-    "max_overflow": 20,
-    "pool_timeout": 30,
-}
 
 engine = create_async_engine(
     settings.DATABASE_URL,
     echo=settings.DEBUG if hasattr(settings, "DEBUG") else False,
-    **DB_POOL_CONFIG,
 )
 
 async_session_factory = async_sessionmaker(engine, class_=AsyncSession, expire_on_commit=False)
@@ -109,21 +97,6 @@ async def init_db():
     import app.models.system_setting  # noqa: F401
     import app.models.playground_session  # noqa: F401
     import app.models.playground_message  # noqa: F401
-    import app.models.system_log  # noqa: F401
-    import app.models.vessel  # noqa: F401
-    import app.models.datasource_mapping  # noqa: F401
-
-    logger.warning_event(
-        "Database pool settings active",
-        event="database.pool.initialized",
-        context={
-            "pool_pre_ping": DB_POOL_CONFIG["pool_pre_ping"],
-            "pool_recycle": DB_POOL_CONFIG["pool_recycle"],
-            "pool_size": DB_POOL_CONFIG["pool_size"],
-            "max_overflow": DB_POOL_CONFIG["max_overflow"],
-            "pool_timeout": DB_POOL_CONFIG["pool_timeout"],
-        },
-    )
 
     async with engine.begin() as conn:
         await conn.run_sync(Base.metadata.create_all)
@@ -146,12 +119,7 @@ async def init_db():
             text(
                 """
                 ALTER TABLE collection_tasks
-                ADD COLUMN IF NOT EXISTS phase VARCHAR(30) DEFAULT 'queued',
-                ADD COLUMN IF NOT EXISTS phase_progress DOUBLE PRECISION,
-                ADD COLUMN IF NOT EXISTS phase_message VARCHAR(255),
-                ADD COLUMN IF NOT EXISTS phase_current BIGINT,
-                ADD COLUMN IF NOT EXISTS phase_total BIGINT,
-                ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30)
+                ADD COLUMN IF NOT EXISTS phase VARCHAR(30) DEFAULT 'queued'
                 """
             )
         )

backend/app/main.py

@@ -1,5 +1,4 @@
 from contextlib import asynccontextmanager
-from uuid import uuid4
 
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
@@ -8,8 +7,6 @@ from starlette.middleware.base import BaseHTTPMiddleware
 from app.api.main import api_router
 from app.api.v1 import websocket
 from app.core.config import settings
-from app.core.logging import configure_logging
-from app.core.request_context import set_request_id
 from app.core.websocket.broadcaster import broadcaster
 from app.db.session import init_db
 from app.services.scheduler import (
@@ -20,9 +17,6 @@ from app.services.scheduler import (
 )
 
 
-configure_logging()
-
-
 class WebSocketCORSMiddleware(BaseHTTPMiddleware):
     async def dispatch(self, request, call_next):
         if request.url.path.startswith("/ws") and request.method == "GET":
@@ -34,18 +28,6 @@ class WebSocketCORSMiddleware(BaseHTTPMiddleware):
         return await call_next(request)
 
 
-class RequestContextMiddleware(BaseHTTPMiddleware):
-    async def dispatch(self, request, call_next):
-        request_id = request.headers.get("X-Request-ID") or uuid4().hex
-        set_request_id(request_id)
-        try:
-            response = await call_next(request)
-        finally:
-            set_request_id(None)
-        response.headers["X-Request-ID"] = request_id
-        return response
-
-
 @asynccontextmanager
 async def lifespan(app: FastAPI):
     await init_db()
@@ -76,7 +58,6 @@ app.add_middleware(
     allow_headers=["*"],
 )
 
-app.add_middleware(RequestContextMiddleware)
 app.add_middleware(WebSocketCORSMiddleware)
 
 app.include_router(api_router, prefix="/api/v1")

backend/app/models/__init__.py

@@ -11,9 +11,6 @@ from app.models.bgp_observation import BGPObservation
 from app.models.system_setting import SystemSetting
 from app.models.playground_session import PlaygroundSession
 from app.models.playground_message import PlaygroundMessage
-from app.models.system_log import SystemLog, AuditLog
-from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
-from app.models.datasource_mapping import DataSourceMappingTemplate
 
 __all__ = [
     "User",
@@ -29,14 +26,4 @@ __all__ = [
     "BGPAnomaly",
     "BGPIncident",
     "BGPObservation",
-    "SystemLog",
-    "AuditLog",
-    "PlaygroundSession",
-    "PlaygroundMessage",
-    "VesselPosition",
-    "VesselStatic",
-    "AISRawObservation",
-    "AISConflictRecord",
-    "AISSourceHealth",
-    "DataSourceMappingTemplate",
 ]

backend/app/models/datasource_mapping.py

@@ -1,32 +0,0 @@
-"""Mapping templates for user-defined data source payloads."""
-
-from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
-from sqlalchemy.sql import func
-
-from app.db.session import Base
-
-
-class DataSourceMappingTemplate(Base):
-    __tablename__ = "datasource_mapping_templates"
-
-    id = Column(Integer, primary_key=True, autoincrement=True)
-    datasource_config_id = Column(
-        Integer,
-        ForeignKey("datasource_configs.id"),
-        nullable=False,
-        index=True,
-    )
-    target_schema = Column(String(80), nullable=False, index=True)
-    mapping_json = Column(JSON, nullable=False, default={})
-    sample_payload_hash = Column(String(64), nullable=True)
-    validation_status = Column(String(30), nullable=False, default="draft")
-    version = Column(Integer, nullable=False, default=1)
-    is_active = Column(Boolean, nullable=False, default=False, index=True)
-    created_at = Column(DateTime(timezone=True), server_default=func.now())
-    updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
-
-    def __repr__(self):
-        return (
-            f"<DataSourceMappingTemplate {self.id}: "
-            f"{self.datasource_config_id}/{self.target_schema}/v{self.version}>"
-        )

backend/app/models/system_log.py

@@ -1,40 +0,0 @@
-from sqlalchemy import JSON, Column, DateTime, Integer, String, Text
-from sqlalchemy.sql import func
-
-from app.db.session import Base
-
-
-class SystemLog(Base):
-    __tablename__ = "system_logs"
-
-    id = Column(Integer, primary_key=True, autoincrement=True)
-    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
-    source = Column(String(50), nullable=False, index=True)
-    service = Column(String(50), nullable=True)
-    module = Column(String(120), nullable=True)
-    event = Column(String(160), nullable=True, index=True)
-    level = Column(String(20), nullable=False, index=True)
-    message = Column(Text, nullable=False)
-    request_id = Column(String(64), nullable=True, index=True)
-    trace_id = Column(String(64), nullable=True)
-    user_id = Column(Integer, nullable=True, index=True)
-    category = Column(String(80), nullable=True, index=True)
-    context = Column(JSON, nullable=False, default=dict)
-    created_at = Column(DateTime(timezone=True), server_default=func.now())
-
-
-class AuditLog(Base):
-    __tablename__ = "audit_logs"
-
-    id = Column(Integer, primary_key=True, autoincrement=True)
-    occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
-    actor_id = Column(Integer, nullable=True, index=True)
-    actor_name = Column(String(255), nullable=True)
-    action = Column(String(120), nullable=False, index=True)
-    target_type = Column(String(80), nullable=True)
-    target_id = Column(String(120), nullable=True)
-    result = Column(String(40), nullable=True, index=True)
-    request_id = Column(String(64), nullable=True, index=True)
-    ip = Column(String(64), nullable=True)
-    details = Column(JSON, nullable=False, default=dict)
-    created_at = Column(DateTime(timezone=True), server_default=func.now())

backend/app/models/task.py

@@ -1,6 +1,6 @@
 """Collection Task model"""
 
-from sqlalchemy import BigInteger, Column, DateTime, Integer, String, Text, Float
+from sqlalchemy import Column, DateTime, Integer, String, Text, Float
 from sqlalchemy.sql import func
 
 from app.db.session import Base
@@ -13,11 +13,6 @@ class CollectionTask(Base):
     datasource_id = Column(Integer, nullable=False, index=True)
     status = Column(String(20), nullable=False)  # pending, running, success, failed, cancelled
     phase = Column(String(30), default="queued")
-    phase_progress = Column(Float)
-    phase_message = Column(String(255))
-    phase_current = Column(BigInteger)
-    phase_total = Column(BigInteger)
-    phase_unit = Column(String(30))
     started_at = Column(DateTime(timezone=True))
     completed_at = Column(DateTime(timezone=True))
     records_processed = Column(Integer, default=0)

backend/app/models/vessel.py

@@ -1,185 +0,0 @@
-"""Vessel AIS models for live maritime tracking."""
-
-from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, JSON, SmallInteger, String
-from sqlalchemy.sql import func
-
-from app.core.time import to_iso8601_utc
-from app.db.session import Base
-
-
-class VesselStatic(Base):
-    """Slow-changing vessel identity and dimensions."""
-
-    __tablename__ = "vessel_static"
-
-    mmsi = Column(BigInteger, primary_key=True)
-    name = Column(String(128), nullable=True)
-    callsign = Column(String(16), nullable=True)
-    vessel_type = Column(SmallInteger, nullable=True, index=True)
-    vessel_type_name = Column(String(64), nullable=True, index=True)
-    flag = Column(String(4), nullable=True, index=True)
-    length = Column(Float, nullable=True)
-    width = Column(Float, nullable=True)
-    draught = Column(Float, nullable=True)
-    imo = Column(BigInteger, nullable=True)
-    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
-
-    def to_dict(self) -> dict:
-        return {
-            "mmsi": self.mmsi,
-            "name": self.name,
-            "callsign": self.callsign,
-            "vessel_type": self.vessel_type,
-            "vessel_type_name": self.vessel_type_name,
-            "flag": self.flag,
-            "length": self.length,
-            "width": self.width,
-            "draught": self.draught,
-            "imo": self.imo,
-            "updated_at": to_iso8601_utc(self.updated_at),
-        }
-
-
-class VesselPosition(Base):
-    """Append-only AIS positions retained for short history windows."""
-
-    __tablename__ = "vessel_position"
-
-    id = Column(Integer, primary_key=True, autoincrement=True)
-    mmsi = Column(BigInteger, nullable=False, index=True)
-    lat = Column(Float, nullable=False)
-    lon = Column(Float, nullable=False)
-    sog = Column(Float, nullable=True)
-    cog = Column(Float, nullable=True)
-    heading = Column(SmallInteger, nullable=True)
-    nav_status = Column(SmallInteger, nullable=True, index=True)
-    received_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
-
-    __table_args__ = (
-        Index("idx_vessel_pos_mmsi_time", "mmsi", "received_at"),
-        Index("idx_vessel_pos_time", "received_at"),
-        Index("idx_vessel_pos_lat_lon", "lat", "lon"),
-    )
-
-    def to_dict(self) -> dict:
-        return {
-            "id": self.id,
-            "mmsi": self.mmsi,
-            "lat": self.lat,
-            "lon": self.lon,
-            "sog": self.sog,
-            "cog": self.cog,
-            "heading": self.heading,
-            "nav_status": self.nav_status,
-            "received_at": to_iso8601_utc(self.received_at),
-        }
-
-
-class AISRawObservation(Base):
-    """Source-level AIS fact before aggregation and conflict resolution."""
-
-    __tablename__ = "ais_raw_observations"
-
-    id = Column(Integer, primary_key=True, autoincrement=True)
-    target_schema = Column(String(64), nullable=False, default="vessel_ais", index=True)
-    source = Column(String(100), nullable=False, index=True)
-    entity_key = Column(String(64), nullable=False, index=True)
-    delivery_mode = Column(String(32), nullable=False, index=True)
-    transport = Column(String(32), nullable=False, index=True)
-    message_type = Column(String(64), nullable=True, index=True)
-    source_message_id = Column(String(128), nullable=True, index=True)
-    observation_hash = Column(String(64), nullable=False, unique=True, index=True)
-    observed_at = Column(DateTime(timezone=True), nullable=False, index=True)
-    collected_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
-    normalized_payload = Column(JSON, default=dict)
-    raw_payload = Column(JSON, default=dict)
-    quality_flags = Column(JSON, default=list)
-
-    __table_args__ = (
-        Index("idx_ais_raw_entity_observed", "target_schema", "entity_key", "observed_at"),
-        Index("idx_ais_raw_source_entity", "source", "entity_key"),
-    )
-
-    def to_dict(self) -> dict:
-        return {
-            "id": self.id,
-            "target_schema": self.target_schema,
-            "source": self.source,
-            "entity_key": self.entity_key,
-            "delivery_mode": self.delivery_mode,
-            "transport": self.transport,
-            "message_type": self.message_type,
-            "source_message_id": self.source_message_id,
-            "observation_hash": self.observation_hash,
-            "observed_at": to_iso8601_utc(self.observed_at),
-            "collected_at": to_iso8601_utc(self.collected_at),
-            "normalized_payload": self.normalized_payload or {},
-            "raw_payload": self.raw_payload or {},
-            "quality_flags": self.quality_flags or [],
-        }
-
-
-class AISConflictRecord(Base):
-    """Recorded field-level disagreement between AIS sources."""
-
-    __tablename__ = "ais_conflict_records"
-
-    id = Column(Integer, primary_key=True, autoincrement=True)
-    target_schema = Column(String(64), nullable=False, default="vessel_ais", index=True)
-    entity_key = Column(String(64), nullable=False, index=True)
-    field = Column(String(64), nullable=False, index=True)
-    candidates = Column(JSON, default=dict)
-    selected_source = Column(String(100), nullable=True, index=True)
-    selected_value = Column(JSON, nullable=True)
-    selected_reason = Column(String(64), nullable=True, index=True)
-    resolved_by = Column(String(32), nullable=False, default="system", index=True)
-    status = Column(String(32), nullable=False, default="open", index=True)
-    created_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
-    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
-
-    __table_args__ = (
-        Index("idx_ais_conflict_entity_field", "target_schema", "entity_key", "field"),
-    )
-
-    def to_dict(self) -> dict:
-        return {
-            "id": self.id,
-            "target_schema": self.target_schema,
-            "entity_key": self.entity_key,
-            "field": self.field,
-            "candidates": self.candidates or {},
-            "selected_source": self.selected_source,
-            "selected_value": self.selected_value,
-            "selected_reason": self.selected_reason,
-            "resolved_by": self.resolved_by,
-            "status": self.status,
-            "created_at": to_iso8601_utc(self.created_at),
-            "updated_at": to_iso8601_utc(self.updated_at),
-        }
-
-
-class AISSourceHealth(Base):
-    """Runtime health signal for an AIS collector source."""
-
-    __tablename__ = "ais_source_health"
-
-    source = Column(String(100), primary_key=True)
-    connection_state = Column(String(32), nullable=False, default="disconnected", index=True)
-    last_seen_at = Column(DateTime(timezone=True), nullable=True, index=True)
-    last_success_at = Column(DateTime(timezone=True), nullable=True, index=True)
-    last_error = Column(String(500), nullable=True)
-    message_rate = Column(Float, nullable=True)
-    lag_seconds = Column(Float, nullable=True)
-    updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
-
-    def to_dict(self) -> dict:
-        return {
-            "source": self.source,
-            "connection_state": self.connection_state,
-            "last_seen_at": to_iso8601_utc(self.last_seen_at),
-            "last_success_at": to_iso8601_utc(self.last_success_at),
-            "last_error": self.last_error,
-            "message_rate": self.message_rate,
-            "lag_seconds": self.lag_seconds,
-            "updated_at": to_iso8601_utc(self.updated_at),
-        }

backend/app/services/ai_client.py

@@ -3,11 +3,9 @@ from __future__ import annotations
 import asyncio
 
 import httpx
-from fastapi import Depends, HTTPException, status
-from sqlalchemy.ext.asyncio import AsyncSession
+from fastapi import HTTPException, status
 
 from app.core.config import settings
-from app.db.session import get_db
 from app.schemas.ai import (
     AIProviderStatusResponse,
     SituationalAnalysisRequest,
@@ -16,27 +14,11 @@ from app.schemas.ai import (
 
 
 class AIProviderClient:
-    def __init__(
-        self,
-        *,
-        service_url: str | None = None,
-        service_token: str | None = None,
-        timeout: int | None = None,
-        retry_attempts: int | None = None,
-        llm_config: dict | None = None,
-    ) -> None:
-        self.service_url = (
-            service_url if service_url is not None else settings.AI_PROVIDER_SERVICE_URL
-        ).rstrip("/")
-        self.service_token = (
-            service_token if service_token is not None else settings.AI_PROVIDER_SERVICE_TOKEN
-        )
-        self.timeout = timeout if timeout is not None else settings.AI_PROVIDER_TIMEOUT_SECONDS
-        self.retry_attempts = max(
-            retry_attempts if retry_attempts is not None else settings.AI_PROVIDER_RETRY_ATTEMPTS,
-            1,
-        )
-        self.llm_config = llm_config or {}
+    def __init__(self) -> None:
+        self.service_url = settings.AI_PROVIDER_SERVICE_URL.rstrip("/")
+        self.service_token = settings.AI_PROVIDER_SERVICE_TOKEN
+        self.timeout = settings.AI_PROVIDER_TIMEOUT_SECONDS
+        self.retry_attempts = max(settings.AI_PROVIDER_RETRY_ATTEMPTS, 1)
 
     def _headers(self, request_id: str | None = None) -> dict[str, str]:
         headers = {"Content-Type": "application/json"}
@@ -44,19 +26,6 @@ class AIProviderClient:
             headers["X-Provider-Token"] = self.service_token
         if request_id:
             headers["X-Request-ID"] = request_id
-        llm_header_map = {
-            "provider": "X-AI-Provider",
-            "provider_api": "X-AI-Provider-API",
-            "base_url": "X-AI-Base-URL",
-            "api_key": "X-AI-API-Key",
-            "model": "X-AI-Model",
-            "max_tokens": "X-AI-Max-Tokens",
-            "anthropic_version": "X-AI-Anthropic-Version",
-        }
-        for key, header_name in llm_header_map.items():
-            value = self.llm_config.get(key)
-            if value not in (None, ""):
-                headers[header_name] = str(value)
         return headers
 
     async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
@@ -136,14 +105,5 @@ class AIProviderClient:
         )
 
 
-async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
-    from app.api.v1.settings import get_runtime_ai_provider_config
-
-    runtime_config = await get_runtime_ai_provider_config(db)
-    return AIProviderClient(
-        service_url=runtime_config["service_url"],
-        service_token=runtime_config["service_token"],
-        timeout=runtime_config["timeout_seconds"],
-        retry_attempts=runtime_config["retry_attempts"],
-        llm_config=runtime_config.get("llm_config") or {},
-    )
+def get_ai_provider_client() -> AIProviderClient:
+    return AIProviderClient()

backend/app/services/barentswatch.py

@@ -1,209 +0,0 @@
-"""BarentsWatch AIS credential resolution and connectivity checks."""
-
-from __future__ import annotations
-
-import os
-import shlex
-from dataclasses import dataclass
-from pathlib import Path
-from typing import Any
-
-import httpx
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from app.core.data_sources import get_data_sources_config
-from app.models.datasource_config import DataSourceConfig
-
-
-BARENTSWATCH_LATEST_URL = "https://live.ais.barentswatch.no/v1/latest/combined"
-BARENTSWATCH_TOKEN_URL = "https://id.barentswatch.no/connect/token"
-BARENTSWATCH_DATASOURCE_NAME = "barentswatch_vessels"
-
-
-@dataclass(frozen=True)
-class BarentsWatchConfig:
-    endpoint: str
-    client_id: str
-    client_secret: str
-    credential_source: str
-    endpoint_source: str
-
-
-def _read_zshrc_env(path: Path | None = None) -> dict[str, str]:
-    zshrc_path = path or Path.home() / ".zshrc"
-    if not zshrc_path.exists():
-        return {}
-
-    values: dict[str, str] = {}
-    for raw_line in zshrc_path.read_text(encoding="utf-8", errors="ignore").splitlines():
-        line = raw_line.strip()
-        if not line or line.startswith("#"):
-            continue
-        if line.startswith("export "):
-            line = line[len("export ") :].strip()
-        if "=" not in line:
-            continue
-
-        key, value = line.split("=", 1)
-        key = key.strip()
-        if not key or not key.replace("_", "").isalnum() or not key[0].isalpha():
-            continue
-
-        try:
-            parsed = shlex.split(value, comments=True, posix=True)
-        except ValueError:
-            parsed = [value.strip().strip("'\"")]
-        if parsed:
-            values[key] = parsed[0]
-    return values
-
-
-def _first_env_value(zshrc_env: dict[str, str], *keys: str) -> tuple[str, str]:
-    for key in keys:
-        value = os.getenv(key)
-        if value:
-            return value, "environment"
-    for key in keys:
-        value = zshrc_env.get(key)
-        if value:
-            return value, "~/.zshrc"
-    return "", ""
-
-
-async def get_barentswatch_datasource_record(db: AsyncSession) -> DataSourceConfig | None:
-    result = await db.execute(
-        select(DataSourceConfig)
-        .where(DataSourceConfig.name == BARENTSWATCH_DATASOURCE_NAME)
-        .where(DataSourceConfig.is_active.is_(True))
-    )
-    return result.scalar_one_or_none()
-
-
-async def resolve_barentswatch_config(db: AsyncSession | None = None) -> BarentsWatchConfig:
-    record = await get_barentswatch_datasource_record(db) if db else None
-    auth_config = dict(record.auth_config or {}) if record else {}
-    config = dict(record.config or {}) if record else {}
-    zshrc_env = _read_zshrc_env()
-
-    env_client_id, env_source = _first_env_value(
-        zshrc_env,
-        "BARENTSWATCH_CLIENT_ID",
-        "BARRENTSWATCH_CLIENT_ID",
-    )
-    env_client_secret, secret_env_source = _first_env_value(
-        zshrc_env,
-        "BARENTSWATCH_CLIENT_SECRET",
-        "BARRENTSWATCH_CLIENT_SECRET",
-    )
-    client_id = auth_config.get("client_id") or config.get("client_id") or env_client_id
-    client_secret = (
-        auth_config.get("client_secret") or config.get("client_secret") or env_client_secret
-    )
-
-    credential_source = ""
-    if auth_config.get("client_id") or auth_config.get("client_secret"):
-        credential_source = "datasource_config"
-    elif config.get("client_id") or config.get("client_secret"):
-        credential_source = "datasource_runtime_config"
-    elif env_source or secret_env_source:
-        credential_source = env_source or secret_env_source
-
-    yaml_endpoint = get_data_sources_config().get_yaml_url(BARENTSWATCH_DATASOURCE_NAME)
-    endpoint = record.endpoint if record and record.endpoint else yaml_endpoint
-    return BarentsWatchConfig(
-        endpoint=endpoint or BARENTSWATCH_LATEST_URL,
-        client_id=str(client_id or ""),
-        client_secret=str(client_secret or ""),
-        credential_source=credential_source or "missing",
-        endpoint_source="datasource_config" if record and record.endpoint else "default",
-    )
-
-
-async def fetch_barentswatch_access_token(
-    client: httpx.AsyncClient,
-    config: BarentsWatchConfig,
-) -> str | None:
-    if not config.client_id or not config.client_secret:
-        return None
-
-    response = await client.post(
-        BARENTSWATCH_TOKEN_URL,
-        data={
-            "client_id": config.client_id,
-            "client_secret": config.client_secret,
-            "scope": "ais",
-            "grant_type": "client_credentials",
-        },
-        headers={"Content-Type": "application/x-www-form-urlencoded"},
-    )
-    response.raise_for_status()
-    payload = response.json()
-    token = payload.get("access_token")
-    return str(token) if token else None
-
-
-async def check_barentswatch_connectivity(db: AsyncSession) -> dict[str, Any]:
-    config = await resolve_barentswatch_config(db)
-    return await check_barentswatch_config(config)
-
-
-async def check_barentswatch_config(config: BarentsWatchConfig) -> dict[str, Any]:
-    if not config.client_id or not config.client_secret:
-        return {
-            "success": False,
-            "stage": "credentials",
-            "message": "未找到 BarentsWatch client id/client secret，请先配置采集器凭证。",
-            "endpoint": config.endpoint,
-            "credential_source": config.credential_source,
-            "settings_tab": "collector_credentials",
-        }
-
-    try:
-        async with httpx.AsyncClient(timeout=20.0) as client:
-            token = await fetch_barentswatch_access_token(client, config)
-            if not token:
-                return {
-                    "success": False,
-                    "stage": "token",
-                    "message": "BarentsWatch token 响应中没有 access_token，请检查凭证。",
-                    "endpoint": config.endpoint,
-                    "credential_source": config.credential_source,
-                    "settings_tab": "collector_credentials",
-                }
-
-            async with client.stream(
-                "GET",
-                config.endpoint,
-                headers={"Authorization": f"Bearer {token}"},
-            ) as response:
-                response.raise_for_status()
-
-        return {
-            "success": True,
-            "stage": "endpoint",
-            "message": "BarentsWatch AIS token 和数据接口均可连通。",
-            "endpoint": config.endpoint,
-            "credential_source": config.credential_source,
-            "endpoint_source": config.endpoint_source,
-        }
-    except httpx.HTTPStatusError as exc:
-        status_code = exc.response.status_code
-        stage = "token" if str(exc.request.url) == BARENTSWATCH_TOKEN_URL else "endpoint"
-        return {
-            "success": False,
-            "stage": stage,
-            "message": f"BarentsWatch {stage} 请求返回 HTTP {status_code}，请检查凭证或接口地址。",
-            "endpoint": config.endpoint,
-            "credential_source": config.credential_source,
-            "settings_tab": "collector_credentials",
-        }
-    except httpx.HTTPError as exc:
-        return {
-            "success": False,
-            "stage": "network",
-            "message": f"BarentsWatch 链路检查失败：{exc.__class__.__name__}",
-            "endpoint": config.endpoint,
-            "credential_source": config.credential_source,
-            "settings_tab": "collector_credentials",
-        }

backend/app/services/collectors/__init__.py

@@ -36,8 +36,6 @@ from app.services.collectors.iptoasn import IPtoASNPrefixGeoCollector
 from app.services.collectors.opengeofeed import OpenGeoFeedPrefixGeoCollector
 from app.services.collectors.nro_delegated import NRODelegatedPrefixGeoCollector
 from app.services.collectors.news_live_streams import NewsLiveStreamsCollector
-from app.services.collectors.aisstream import AISStreamCollector
-from app.services.collectors.vessel_ais import VesselAISCollector
 
 collector_registry.register(TOP500Collector())
 collector_registry.register(EpochAIGPUCollector())
@@ -65,41 +63,3 @@ collector_registry.register(IPtoASNPrefixGeoCollector())
 collector_registry.register(OpenGeoFeedPrefixGeoCollector())
 collector_registry.register(NRODelegatedPrefixGeoCollector())
 collector_registry.register(NewsLiveStreamsCollector())
-collector_registry.register(VesselAISCollector())
-collector_registry.register(AISStreamCollector())
-
-__all__ = [
-    "BaseCollector",
-    "HTTPCollector",
-    "IntervalCollector",
-    "collector_registry",
-    "CollectorRegistry",
-    "TOP500Collector",
-    "EpochAIGPUCollector",
-    "HuggingFaceModelCollector",
-    "HuggingFaceDatasetCollector",
-    "HuggingFaceSpacesCollector",
-    "PeeringDBIXPCollector",
-    "PeeringDBNetworkCollector",
-    "PeeringDBFacilityCollector",
-    "TeleGeographyCableCollector",
-    "TeleGeographyLandingPointCollector",
-    "TeleGeographyCableSystemCollector",
-    "CloudflareRadarDeviceCollector",
-    "CloudflareRadarTrafficCollector",
-    "CloudflareRadarTopASCollector",
-    "ArcGISCableCollector",
-    "FAOLandingPointCollector",
-    "ArcGISLandingPointCollector",
-    "ArcGISCableLandingRelationCollector",
-    "SpaceTrackTLECollector",
-    "CelesTrakTLECollector",
-    "RISLiveCollector",
-    "BGPStreamBackfillCollector",
-    "IPtoASNPrefixGeoCollector",
-    "OpenGeoFeedPrefixGeoCollector",
-    "NRODelegatedPrefixGeoCollector",
-    "NewsLiveStreamsCollector",
-    "VesselAISCollector",
-    "AISStreamCollector",
-]

backend/app/services/collectors/aisstream.py

@@ -1,276 +0,0 @@
-"""AISStream WebSocket collector for realtime vessel AIS observations."""
-
-from datetime import UTC, datetime
-import asyncio
-import json
-import os
-from typing import Any
-
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from app.core.data_sources import get_data_sources_config
-from app.models.datasource_config import DataSourceConfig
-from app.services.collectors.base import BaseCollector
-from app.services.vessel_ais_aggregation import (
-    AISSTREAM_DELIVERY_MODE,
-    AISSTREAM_TRANSPORT,
-    record_vessel_ais_observation,
-    update_ais_source_health,
-)
-from app.services.vessel_types import normalize_vessel_type_name
-
-DEFAULT_AISSTREAM_URL = "wss://stream.aisstream.io/v0/stream"
-DEFAULT_BOUNDING_BOXES = [[[-90, -180], [90, 180]]]
-DEFAULT_MESSAGE_TYPES = ["PositionReport", "ShipStaticData"]
-
-
-class AISStreamCollector(BaseCollector):
-    """Collect AISStream WebSocket messages into the raw AIS observation layer."""
-
-    name = "aisstream_vessels"
-    priority = "P1"
-    module = "L4"
-    frequency_hours = 1
-    data_type = "vessel_ais"
-    fail_on_empty = False
-
-    async def _load_datasource_config(self) -> DataSourceConfig | None:
-        if self._db_session is None:
-            return None
-        result = await self._db_session.execute(
-            select(DataSourceConfig)
-            .where(DataSourceConfig.name == self.name)
-            .where(DataSourceConfig.is_active.is_(True))
-        )
-        return result.scalar_one_or_none()
-
-    async def _get_effective_config(self) -> dict[str, Any]:
-        datasource_config = await self._load_datasource_config()
-        config = dict(datasource_config.config or {}) if datasource_config else {}
-        auth_config = dict(datasource_config.auth_config or {}) if datasource_config else {}
-        endpoint = (
-            (datasource_config.endpoint if datasource_config else None)
-            or self._resolved_url
-            or get_data_sources_config().get_yaml_url(self.name)
-            or DEFAULT_AISSTREAM_URL
-        )
-        api_key = (
-            auth_config.get("api_key")
-            or config.get("api_key")
-            or os.getenv("AISSTREAM_API_KEY")
-        )
-        return {
-            "endpoint": endpoint,
-            "api_key": api_key,
-            "bounding_boxes": config.get("bounding_boxes") or DEFAULT_BOUNDING_BOXES,
-            "message_types": config.get("message_types") or DEFAULT_MESSAGE_TYPES,
-            "max_messages": int(config.get("max_messages") or 500),
-            "receive_timeout_seconds": float(config.get("receive_timeout_seconds") or 30),
-        }
-
-    async def fetch(self) -> list[dict[str, Any]]:
-        config = await self._get_effective_config()
-        if not config["api_key"]:
-            raise RuntimeError("AISStream API key is not configured")
-
-        try:
-            import websockets
-        except ImportError as exc:
-            raise RuntimeError("Python package 'websockets' is required for AISStream") from exc
-
-        subscription = {
-            "APIKey": config["api_key"],
-            "BoundingBoxes": config["bounding_boxes"],
-            "FilterMessageTypes": config["message_types"],
-        }
-
-        messages: list[dict[str, Any]] = []
-        try:
-            async with websockets.connect(config["endpoint"]) as websocket:
-                await websocket.send(json.dumps(subscription))
-                while len(messages) < config["max_messages"]:
-                    try:
-                        raw_message = await asyncio.wait_for(
-                            websocket.recv(),
-                            timeout=config["receive_timeout_seconds"],
-                        )
-                    except TimeoutError:
-                        break
-                    payload = json.loads(raw_message)
-                    if isinstance(payload, dict):
-                        messages.append(payload)
-        except Exception as exc:
-            if self._db_session is not None:
-                await update_ais_source_health(
-                    self._db_session,
-                    source=self.name,
-                    connection_state="disconnected",
-                    last_error=f"{exc.__class__.__name__}: {exc}",
-                )
-                await self._db_session.commit()
-            raise
-
-        return messages
-
-    def transform(self, raw_data: list[dict[str, Any]]) -> list[dict[str, Any]]:
-        records = []
-        for item in raw_data:
-            record = self._normalize_message(item)
-            if record:
-                records.append(record)
-        return records
-
-    async def _save_data(
-        self,
-        db: AsyncSession,
-        data: list[dict[str, Any]],
-        task_id: int | None = None,
-        snapshot_id: int | None = None,
-    ) -> int:
-        now = datetime.now(UTC)
-        records_added = 0
-        latest_observed_at = now
-        for index, item in enumerate(data):
-            observed_at = item.get("received_at") or now
-            observation = await record_vessel_ais_observation(
-                db,
-                source=self.name,
-                normalized_payload=item,
-                raw_payload=item.get("_raw_payload") or item,
-                delivery_mode=AISSTREAM_DELIVERY_MODE,
-                transport=AISSTREAM_TRANSPORT,
-                message_type=item.get("_message_type") or "PositionReport",
-                source_message_id=item.get("_source_message_id"),
-                observed_at=observed_at,
-                collected_at=now,
-            )
-            if observation is not None:
-                records_added += 1
-            if isinstance(observed_at, datetime) and observed_at > latest_observed_at:
-                latest_observed_at = observed_at
-            if (index + 1) % 1000 == 0:
-                await self.update_progress(index + 1, commit=True)
-
-        await update_ais_source_health(
-            db,
-            source=self.name,
-            connection_state="connected",
-            observed_count=len(data),
-            last_seen_at=latest_observed_at,
-            last_success_at=now if data else None,
-            lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
-        )
-        await db.commit()
-        await self.update_progress(records_added, force=True)
-        return records_added
-
-    def _normalize_message(self, item: dict[str, Any]) -> dict[str, Any] | None:
-        message_type = str(item.get("MessageType") or item.get("message_type") or "")
-        metadata = item.get("MetaData") if isinstance(item.get("MetaData"), dict) else {}
-        message = item.get("Message") if isinstance(item.get("Message"), dict) else {}
-        body = message.get(message_type) if isinstance(message.get(message_type), dict) else message
-        if not isinstance(body, dict):
-            body = {}
-
-        mmsi = _as_int(_pick(metadata, "MMSI", "mmsi") or _pick(body, "MMSI", "mmsi"))
-        if mmsi is None:
-            return None
-
-        received_at = _parse_datetime(
-            _pick(metadata, "time_utc", "Time_UTC", "timestamp")
-            or _pick(body, "Timestamp", "timestamp", "time")
-        )
-        ship_name = _clean_text(
-            _pick(body, "Name", "ShipName", "name")
-            or _pick(metadata, "ShipName", "ship_name", "name")
-        )
-        record: dict[str, Any] = {
-            "mmsi": mmsi,
-            "received_at": received_at,
-            "_message_type": message_type or None,
-            "_source_message_id": item.get("MessageID") or item.get("message_id"),
-            "_raw_payload": item,
-        }
-
-        lat = _as_float(_pick(body, "Latitude", "lat", "latitude"))
-        lon = _as_float(_pick(body, "Longitude", "lon", "lng", "longitude"))
-        if lat is not None and lon is not None:
-            if not (-90 <= lat <= 90 and -180 <= lon <= 180):
-                return None
-            record.update(
-                {
-                    "lat": lat,
-                    "lon": lon,
-                    "sog": _as_float(_pick(body, "Sog", "SOG", "speedOverGround")),
-                    "cog": _as_float(_pick(body, "Cog", "COG", "courseOverGround")),
-                    "heading": _as_int(_pick(body, "TrueHeading", "Heading", "heading")),
-                    "nav_status": _as_int(_pick(body, "NavigationalStatus", "nav_status")),
-                }
-            )
-
-        vessel_type = _as_int(_pick(body, "Type", "ShipType", "vessel_type"))
-        record.update(
-            {
-                "name": ship_name,
-                "callsign": _pick(body, "CallSign", "callsign"),
-                "imo": _as_int(_pick(body, "ImoNumber", "IMO", "imo")),
-                "vessel_type": vessel_type,
-                "vessel_type_name": _pick(body, "TypeName", "ShipTypeName", "vessel_type_name")
-                or normalize_vessel_type_name(vessel_type),
-                "length": _as_float(_pick(body, "DimensionToBow", "Length", "length")),
-                "width": _as_float(_pick(body, "DimensionToPort", "Width", "width")),
-            }
-        )
-        return record
-
-
-def _pick(item: dict[str, Any], *keys: str) -> Any:
-    for key in keys:
-        if key in item and item[key] not in (None, ""):
-            return item[key]
-    return None
-
-
-def _clean_text(value: Any) -> str | None:
-    if value in (None, ""):
-        return None
-    text = str(value).strip()
-    return text or None
-
-
-def _as_float(value: Any) -> float | None:
-    try:
-        if value in (None, ""):
-            return None
-        return float(value)
-    except (TypeError, ValueError):
-        return None
-
-
-def _as_int(value: Any) -> int | None:
-    try:
-        if value in (None, ""):
-            return None
-        return int(float(value))
-    except (TypeError, ValueError):
-        return None
-
-
-def _parse_datetime(value: Any) -> datetime | None:
-    if isinstance(value, datetime):
-        return value if value.tzinfo else value.replace(tzinfo=UTC)
-    if not value:
-        return None
-    if isinstance(value, (int, float)):
-        timestamp = float(value)
-        if timestamp > 10_000_000_000:
-            timestamp /= 1000
-        return datetime.fromtimestamp(timestamp, UTC)
-    if isinstance(value, str):
-        try:
-            parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
-            return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
-        except ValueError:
-            return None
-    return None

backend/app/services/collectors/base.py

@@ -54,11 +54,6 @@ class BaseCollector(ABC):
                 "task_id": self._current_task.id,
                 "status": self._current_task.status,
                 "phase": self._current_task.phase,
-                "phase_progress": self._current_task.phase_progress,
-                "phase_message": self._current_task.phase_message,
-                "phase_current": self._current_task.phase_current,
-                "phase_total": self._current_task.phase_total,
-                "phase_unit": self._current_task.phase_unit,
                 "progress": progress,
                 "records_processed": self._current_task.records_processed,
                 "total_records": self._current_task.total_records,
@@ -85,52 +80,12 @@ class BaseCollector(ABC):
 
             await self._publish_task_update(force=force)
 
-    async def set_phase(self, phase: str, *, message: str | None = None, reset_progress: bool = True):
+    async def set_phase(self, phase: str):
         if self._current_task and self._db_session:
             self._current_task.phase = phase
-            self._current_task.phase_message = message
-            if reset_progress:
-                self._current_task.phase_progress = None
-                self._current_task.phase_current = None
-                self._current_task.phase_total = None
-                self._current_task.phase_unit = None
             await self._db_session.commit()
             await self._publish_task_update(force=True)
 
-    async def update_phase_progress(
-        self,
-        *,
-        current: int | None = None,
-        total: int | None = None,
-        unit: str | None = None,
-        message: str | None = None,
-        progress: float | None = None,
-        commit: bool = False,
-        force: bool = False,
-    ):
-        """Update progress for the current phase without changing task totals."""
-        if not self._current_task or not self._db_session:
-            return
-
-        if progress is None and current is not None and total and total > 0:
-            progress = (current / total) * 100
-
-        if progress is not None:
-            self._current_task.phase_progress = max(0.0, min(float(progress), 100.0))
-        if current is not None:
-            self._current_task.phase_current = max(0, int(current))
-        if total is not None:
-            self._current_task.phase_total = max(0, int(total))
-        if unit is not None:
-            self._current_task.phase_unit = unit
-        if message is not None:
-            self._current_task.phase_message = message
-
-        if commit:
-            await self._db_session.commit()
-
-        await self._publish_task_update(force=force)
-
     @abstractmethod
     async def fetch(self) -> List[Dict[str, Any]]:
         """Fetch raw data from source"""
@@ -296,7 +251,7 @@ class BaseCollector(ABC):
         await self._publish_task_update(force=True)
 
         try:
-            await self.set_phase("fetching", message="正在拉取原始数据")
+            await self.set_phase("fetching")
             raw_data = await self.fetch()
             task.total_records = len(raw_data)
             await db.commit()
@@ -305,20 +260,15 @@ class BaseCollector(ABC):
             if self.fail_on_empty and not raw_data:
                 raise RuntimeError(f"Collector {self.name} returned no data")
 
-            await self.set_phase("transforming", message="正在转换采集数据")
+            await self.set_phase("transforming")
             data = self.transform(raw_data)
             snapshot_id = await self._create_snapshot(db, task_id, data, start_time)
 
-            await self.set_phase("saving", message="正在保存采集数据")
+            await self.set_phase("saving")
             records_count = await self._save_data(db, data, task_id=task_id, snapshot_id=snapshot_id)
 
             task.status = "success"
             task.phase = "completed"
-            task.phase_progress = 100.0
-            task.phase_message = "采集完成"
-            task.phase_current = records_count
-            task.phase_total = records_count
-            task.phase_unit = "records"
             task.records_processed = records_count
             task.progress = 100.0
             task.completed_at = datetime.now(UTC)
@@ -335,7 +285,6 @@ class BaseCollector(ABC):
             await db.rollback()
             task.status = "cancelled"
             task.phase = "cancelled"
-            task.phase_message = "采集已取消"
             task.error_message = "Collection cancelled by operator and rolled back"
             task.completed_at = datetime.now(UTC)
             if snapshot_id is not None:
@@ -352,7 +301,6 @@ class BaseCollector(ABC):
             await db.rollback()
             task.status = "failed"
             task.phase = "failed"
-            task.phase_message = str(e)
             task.error_message = str(e)
             task.completed_at = datetime.now(UTC)
             if snapshot_id is not None:

backend/app/services/collectors/celestrak.py

@@ -46,9 +46,6 @@ class CelesTrakTLECollector(BaseCollector):
                     if response.status_code == 200:
                         data = response.json()
                         if isinstance(data, list):
-                            for item in data:
-                                if isinstance(item, dict):
-                                    item["_celestrak_group"] = group
                             all_satellites.extend(data)
                             print(f"CelesTrak: Fetched {len(data)} satellites from group '{group}'")
                 except Exception as e:
@@ -81,7 +78,6 @@ class CelesTrakTLECollector(BaseCollector):
                     "name": item.get("OBJECT_NAME", "Unknown"),
                     "reference_date": item.get("EPOCH", ""),
                     "metadata": {
-                        "constellation_group": item.get("_celestrak_group"),
                         "norad_cat_id": item.get("NORAD_CAT_ID"),
                         "international_designator": item.get("OBJECT_ID"),
                         "epoch": item.get("EPOCH"),

backend/app/services/collectors/iptoasn.py

@@ -108,11 +108,6 @@ class IPtoASNPrefixGeoCollector(BaseCollector):
                 self._current_task.total_records = total_expected
                 self._current_task.records_processed = 0
                 self._current_task.progress = 0.0
-                self._current_task.phase_progress = 0.0
-                self._current_task.phase_message = "正在下载 IPtoASN 数据"
-                self._current_task.phase_current = 0
-                self._current_task.phase_total = total_expected
-                self._current_task.phase_unit = "bytes"
                 await self._db_session.commit()
                 await self._publish_task_update(force=True)
 
@@ -140,14 +135,7 @@ class IPtoASNPrefixGeoCollector(BaseCollector):
                         return
                     last_emit["value"] = aggregated
                     last_emit["t"] = now
-                current = min(aggregated, total_expected)
-                await self.update_phase_progress(
-                    current=current,
-                    total=total_expected,
-                    unit="bytes",
-                    message="正在下载 IPtoASN 数据",
-                )
-                await self.update_progress(current, commit=True)
+                await self.update_progress(min(aggregated, total_expected), commit=True)
 
             batches = await asyncio.gather(
                 *(
@@ -160,12 +148,6 @@ class IPtoASNPrefixGeoCollector(BaseCollector):
                 )
             )
             if total_expected > 0:
-                await self.update_phase_progress(
-                    current=total_expected,
-                    total=total_expected,
-                    unit="bytes",
-                    message="IPtoASN 数据下载完成",
-                )
                 await self.update_progress(total_expected, commit=True, force=True)
 
         rows: list[dict[str, Any]] = []

backend/app/services/collectors/nro_delegated.py

@@ -39,23 +39,12 @@ class NRODelegatedPrefixGeoCollector(BaseCollector):
                 self._current_task.total_records = total_expected
                 self._current_task.records_processed = 0
                 self._current_task.progress = 0.0
-                self._current_task.phase_progress = 0.0
-                self._current_task.phase_message = "正在下载 NRO delegated 数据"
-                self._current_task.phase_current = 0
-                self._current_task.phase_total = total_expected
-                self._current_task.phase_unit = "bytes"
                 await self._db_session.commit()
                 await self._publish_task_update(force=True)
 
             async def on_progress(downloaded: int, total: int | None) -> None:
                 if not total or total <= 0:
                     return
-                await self.update_phase_progress(
-                    current=min(downloaded, total),
-                    total=total,
-                    unit="bytes",
-                    message="正在下载 NRO delegated 数据",
-                )
                 await self.update_progress(min(downloaded, total), commit=True)
 
             body_path = await self._downloader.download_file(

backend/app/services/collectors/opengeofeed.py

@@ -40,23 +40,12 @@ class OpenGeoFeedPrefixGeoCollector(BaseCollector):
                 self._current_task.total_records = total_expected
                 self._current_task.records_processed = 0
                 self._current_task.progress = 0.0
-                self._current_task.phase_progress = 0.0
-                self._current_task.phase_message = "正在下载 OpenGeoFeed 数据"
-                self._current_task.phase_current = 0
-                self._current_task.phase_total = total_expected
-                self._current_task.phase_unit = "bytes"
                 await self._db_session.commit()
                 await self._publish_task_update(force=True)
 
             async def on_progress(downloaded: int, total: int | None) -> None:
                 if not total or total <= 0:
                     return
-                await self.update_phase_progress(
-                    current=min(downloaded, total),
-                    total=total,
-                    unit="bytes",
-                    message="正在下载 OpenGeoFeed 数据",
-                )
                 await self.update_progress(min(downloaded, total), commit=True)
 
             body_path = await self._downloader.download_file(

backend/app/services/collectors/vessel_ais.py

@@ -1,280 +0,0 @@
-"""BarentsWatch AIS collector for vessel tracking."""
-
-from datetime import UTC, datetime, timedelta
-from typing import Any
-
-import httpx
-from sqlalchemy import delete
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from app.models.vessel import VesselPosition, VesselStatic
-from app.services.barentswatch import (
-    BARENTSWATCH_LATEST_URL,
-    fetch_barentswatch_access_token,
-    resolve_barentswatch_config,
-)
-from app.services.collectors.base import BaseCollector
-from app.services.vessel_ais_aggregation import (
-    BARENTSWATCH_DELIVERY_MODE,
-    BARENTSWATCH_TRANSPORT,
-    record_vessel_ais_observation,
-    update_ais_source_health,
-)
-from app.services.vessel_types import normalize_vessel_type_name
-
-
-class VesselAISCollector(BaseCollector):
-    """Collect latest AIS positions and append them to vessel tables."""
-
-    name = "barentswatch_vessels"
-    priority = "P1"
-    module = "L4"
-    frequency_hours = 1
-    data_type = "vessel_ais"
-
-    @property
-    def base_url(self) -> str:
-        return self._resolved_url or BARENTSWATCH_LATEST_URL
-
-    async def _get_access_token(self, client: httpx.AsyncClient) -> str | None:
-        config = await resolve_barentswatch_config(self._db_session)
-        return await fetch_barentswatch_access_token(client, config)
-
-    async def fetch(self) -> list[dict[str, Any]]:
-        async with httpx.AsyncClient(timeout=60.0) as client:
-            headers: dict[str, str] = {}
-            token = await self._get_access_token(client)
-            if token:
-                headers["Authorization"] = f"Bearer {token}"
-
-            response = await client.get(self.base_url, headers=headers)
-            if response.status_code == 401 and not token:
-                return self._get_sample_data()
-            response.raise_for_status()
-            payload = response.json()
-
-        if isinstance(payload, list):
-            return [item for item in payload if isinstance(item, dict)]
-        if isinstance(payload, dict):
-            for key in ("features", "data", "items", "vessels"):
-                value = payload.get(key)
-                if isinstance(value, list):
-                    if key == "features":
-                        return [
-                            {
-                                **(item.get("properties") or {}),
-                                "geometry": item.get("geometry"),
-                            }
-                            for item in value
-                            if isinstance(item, dict)
-                        ]
-                    return [item for item in value if isinstance(item, dict)]
-        return self._get_sample_data()
-
-    def transform(self, raw_data: list[dict[str, Any]]) -> list[dict[str, Any]]:
-        transformed = []
-        for item in raw_data:
-            record = self._normalize_record(item)
-            if record:
-                transformed.append(record)
-        return transformed
-
-    async def _save_data(
-        self,
-        db: AsyncSession,
-        data: list[dict[str, Any]],
-        task_id: int | None = None,
-        snapshot_id: int | None = None,
-    ) -> int:
-        now = datetime.now(UTC)
-        records_added = 0
-
-        for index, item in enumerate(data):
-            observed_at = item.get("received_at") or now
-            await record_vessel_ais_observation(
-                db,
-                source=self.name,
-                normalized_payload=item,
-                raw_payload=item,
-                delivery_mode=BARENTSWATCH_DELIVERY_MODE,
-                transport=BARENTSWATCH_TRANSPORT,
-                observed_at=observed_at,
-                collected_at=now,
-            )
-
-            static = await db.get(VesselStatic, item["mmsi"])
-            if static is None:
-                static = VesselStatic(mmsi=item["mmsi"])
-                db.add(static)
-
-            for field in (
-                "name",
-                "callsign",
-                "vessel_type",
-                "vessel_type_name",
-                "flag",
-                "length",
-                "width",
-                "draught",
-                "imo",
-            ):
-                value = item.get(field)
-                if value not in (None, ""):
-                    setattr(static, field, value)
-            static.updated_at = now
-
-            db.add(
-                VesselPosition(
-                    mmsi=item["mmsi"],
-                    lat=item["lat"],
-                    lon=item["lon"],
-                    sog=item.get("sog"),
-                    cog=item.get("cog"),
-                    heading=item.get("heading"),
-                    nav_status=item.get("nav_status"),
-                    received_at=observed_at,
-                )
-            )
-            records_added += 1
-
-            if (index + 1) % 1000 == 0:
-                await self.update_progress(index + 1, commit=True)
-
-        latest_observed_at = max(
-            (item.get("received_at") for item in data if item.get("received_at")),
-            default=now,
-        )
-        await update_ais_source_health(
-            db,
-            source=self.name,
-            connection_state="connected",
-            observed_count=len(data),
-            last_seen_at=latest_observed_at,
-            last_success_at=now if data else None,
-            lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
-        )
-        await db.execute(
-            delete(VesselPosition).where(VesselPosition.received_at < now - timedelta(hours=24))
-        )
-        await db.commit()
-        await self.update_progress(records_added, force=True)
-        return records_added
-
-    def _normalize_record(self, item: dict[str, Any]) -> dict[str, Any] | None:
-        mmsi = _as_int(_pick(item, "mmsi", "MMSI", "Mmsi"))
-        lat = _as_float(_pick(item, "lat", "latitude", "Latitude"))
-        lon = _as_float(_pick(item, "lon", "lng", "longitude", "Longitude"))
-
-        geometry = item.get("geometry")
-        coordinates = geometry.get("coordinates") if isinstance(geometry, dict) else None
-        if (lat is None or lon is None) and isinstance(coordinates, list) and len(coordinates) >= 2:
-            lon = _as_float(coordinates[0])
-            lat = _as_float(coordinates[1])
-
-        if mmsi is None or lat is None or lon is None:
-            return None
-        if not (-90 <= lat <= 90 and -180 <= lon <= 180):
-            return None
-
-        vessel_type = _as_int(_pick(item, "vessel_type", "shipType", "ship_type", "ShipType"))
-        vessel_type_name = (
-            _pick(item, "vessel_type_name", "shipTypeName", "ship_type_name", "VesselTypeName")
-            or normalize_vessel_type_name(vessel_type)
-        )
-        received_at = _parse_datetime(_pick(item, "received_at", "timestamp", "time", "msgtime"))
-
-        return {
-            "mmsi": mmsi,
-            "name": _pick(item, "name", "shipName", "ship_name", "Name"),
-            "callsign": _pick(item, "callsign", "callSign", "CallSign"),
-            "vessel_type": vessel_type,
-            "vessel_type_name": vessel_type_name,
-            "flag": _pick(item, "flag", "country", "Flag"),
-            "length": _as_float(_pick(item, "length", "shipLength", "Length")),
-            "width": _as_float(_pick(item, "width", "shipWidth", "Width")),
-            "draught": _as_float(_pick(item, "draught", "draft", "Draught")),
-            "imo": _as_int(_pick(item, "imo", "IMO", "imoNumber")),
-            "lat": lat,
-            "lon": lon,
-            "sog": _as_float(_pick(item, "sog", "speedOverGround", "SOG")),
-            "cog": _as_float(_pick(item, "cog", "courseOverGround", "COG")),
-            "heading": _as_int(_pick(item, "heading", "trueHeading", "Heading")),
-            "nav_status": _as_int(_pick(item, "nav_status", "navStatus", "NavigationalStatus")),
-            "received_at": received_at,
-        }
-
-    def _get_sample_data(self) -> list[dict[str, Any]]:
-        return [
-            {
-                "mmsi": 257123000,
-                "name": "OSLO TRADER",
-                "lat": 59.91,
-                "lon": 10.73,
-                "sog": 12.4,
-                "cog": 214,
-                "heading": 215,
-                "nav_status": 0,
-                "vessel_type": 70,
-                "vessel_type_name": "Cargo",
-                "flag": "NO",
-                "length": 185,
-            },
-            {
-                "mmsi": 257456000,
-                "name": "NORDIC FJORD",
-                "lat": 60.39,
-                "lon": 5.32,
-                "sog": 0.2,
-                "cog": 82,
-                "heading": 80,
-                "nav_status": 1,
-                "vessel_type": 60,
-                "vessel_type_name": "Passenger",
-                "flag": "NO",
-                "length": 126,
-            },
-        ]
-
-
-def _pick(item: dict[str, Any], *keys: str) -> Any:
-    for key in keys:
-        if key in item and item[key] not in (None, ""):
-            return item[key]
-    return None
-
-
-def _as_float(value: Any) -> float | None:
-    try:
-        if value in (None, ""):
-            return None
-        return float(value)
-    except (TypeError, ValueError):
-        return None
-
-
-def _as_int(value: Any) -> int | None:
-    try:
-        if value in (None, ""):
-            return None
-        return int(float(value))
-    except (TypeError, ValueError):
-        return None
-
-
-def _parse_datetime(value: Any) -> datetime | None:
-    if isinstance(value, datetime):
-        return value if value.tzinfo else value.replace(tzinfo=UTC)
-    if not value:
-        return None
-    if isinstance(value, (int, float)):
-        timestamp = float(value)
-        if timestamp > 10_000_000_000:
-            timestamp /= 1000
-        return datetime.fromtimestamp(timestamp, UTC)
-    if isinstance(value, str):
-        try:
-            parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
-            return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
-        except ValueError:
-            return None
-    return None

backend/app/services/credential_guides.py

@@ -1,224 +0,0 @@
-"""Credential setup guides for collector integrations."""
-
-from __future__ import annotations
-
-from dataclasses import dataclass
-from typing import Any
-
-from sqlalchemy import select
-
-from app.models.system_setting import SystemSetting
-from app.schemas.ai import SituationalAnalysisRequest
-from app.services.ai_client import AIProviderClient
-
-
-CREDENTIAL_GUIDES_CATEGORY = "collector_credential_guides"
-
-
-@dataclass(frozen=True)
-class CredentialGuideDefault:
-    provider: str
-    title: str
-    prompt: str
-    markdown: str
-
-
-BARENTSWATCH_DEFAULT_GUIDE = CredentialGuideDefault(
-    provider="barentswatch",
-    title="BarentsWatch AIS 凭证获取教程",
-    prompt=(
-        "请生成一份中文教程，指导开发者获取 BarentsWatch Live AIS API 的 "
-        "OAuth client credentials。教程要面向已经有本地开发环境的人，包含注册/登录、"
-        "创建 client、申请或确认 ais scope、复制 client id 和 client secret、"
-        "在系统设置中填写并验证连接、常见失败排查。不要编造具体页面按钮文案，"
-        "必须参考官方 tutorial：https://developer.barentswatch.no/docs/tutorial 。"
-        "必须强调 Live AIS 要选择 AIS-client / AIS - API，而不是普通 API-client。"
-        "如果步骤可能变化，要提醒以 BarentsWatch developer portal 当前页面为准。"
-    ),
-    markdown="""## BarentsWatch AIS 凭证获取
-
-官方教程：https://developer.barentswatch.no/docs/tutorial
-
-1. 先打开上面的 BarentsWatch 官方 tutorial，按官方流程登录或注册开发者账号。
-2. 在 Developer access 页面选择 `AIS - API`，不要选择普通的 `BarentsWatch - API`。
-3. 在 `AIS - API` 下创建用于 Planet 的 AIS client。
-4. 创建时记下你设置的 password / client secret。
-5. 回到 My Page 复制完整 `Client ID`。它通常长得像 `your.email@example.com:client-name`。
-6. 回到 Planet 的 `设置 -> 采集器设置 -> BarentsWatch AIS`，填入 `Client ID` 和 `Client Secret`。
-7. 点击 `连接` 验证 token 和 AIS endpoint 是否可访问。
-8. 连接成功后保存凭证。
-
-### 请求规则
-
-- Token 地址：`https://id.barentswatch.no/connect/token`
-- 请求方式：`POST`
-- Content-Type：`application/x-www-form-urlencoded`
-- Body 必须包含：`grant_type=client_credentials`、`client_id`、`client_secret`、`scope=ais`
-- `client_id`、`client_secret`、`scope`、`grant_type` 都要放在 body，不要放在 header。
-- AIS 数据请求使用 header：`Authorization: Bearer <access_token>`
-
-### 常见排查
-
-- `未找到凭证`：确认 `Client ID` 和 `Client Secret` 已填写，或已经写入 `~/.zshrc`。
-- `HTTP 401/403`：通常是选成了普通 `BarentsWatch - API` client、client secret 错误，或 token 请求没有使用 `scope=ais`。
-- `network` 错误：检查本机是否能访问 `id.barentswatch.no` 和 `live.ais.barentswatch.no`。
-- Endpoint 建议保持默认：`https://live.ais.barentswatch.no/v1/latest/combined`。
-""",
-)
-
-AISSTREAM_DEFAULT_GUIDE = CredentialGuideDefault(
-    provider="aisstream",
-    title="AISStream API Key 获取教程",
-    prompt=(
-        "请生成一份中文教程，指导开发者获取 AISStream 的 API Key 并配置到 Planet。"
-        "教程要面向已经有本地开发环境的人，包含注册/登录 AISStream、获取 API Key、"
-        "理解免费额度和订阅范围、在 Planet 设置中心填写 API Key、配置 bounding boxes "
-        "和 message types、验证连接、常见失败排查。必须提醒用户以 AISStream 当前官网和"
-        "服务条款为准，不要编造具体页面按钮文案。"
-    ),
-    markdown="""## AISStream API Key 获取
-
-官方入口：https://aisstream.io/
-
-1. 打开 AISStream 官网，按当前页面指引注册或登录账号。
-2. 在账号/API 管理页面创建或复制你的 API Key。
-3. 先确认当前账号额度、使用条款和可订阅区域。实时 AIS 流量可能很大，不建议一开始订阅全球范围。
-4. 回到 Planet 的 `设置 -> 采集器设置 -> AISStream 实时船舶`。
-5. 在 `AISStream 凭证` 中填入 API Key。
-6. Endpoint 通常保持默认：`wss://stream.aisstream.io/v0/stream`。
-7. 按需配置 `Bounding Boxes JSON` 和 `消息类型`。
-8. 点击连接测试，确认系统能读取凭证且 WebSocket endpoint 格式有效。
-9. 保存采集器设置后再运行 `aisstream_vessels` collector。
-
-### 推荐配置
-
-默认消息类型：
-
-```json
-["PositionReport", "ShipStaticData"]
-```
-
-默认 Bounding Boxes 示例：
-
-```json
-[[[-90, -180], [90, 180]]]
-```
-
-这个示例表示全球范围。实际使用时建议先改成较小区域，降低消息量和处理压力。
-
-### 请求规则
-
-- Endpoint：`wss://stream.aisstream.io/v0/stream`
-- 传输方式：WebSocket
-- API Key 放在订阅 payload 中，不放在 HTTP header。
-- Planet 会把 AISStream 标记为 `delivery_mode = realtime_stream`、`transport = websocket`。
-- AISStream collector 只写入 AIS raw observations，不直接覆盖最终船只展示表。
-
-### 常见排查
-
-- `未找到凭证`：确认 API Key 已保存到采集器设置，或设置了 `AISSTREAM_API_KEY` 环境变量 / `~/.zshrc`。
-- `endpoint 必须是 ws:// 或 wss://`：AISStream 是 WebSocket 流接口，不要填普通 `https://` API 地址。
-- 采集量过大：缩小 `Bounding Boxes JSON`，减少 `message_types`，或降低单次最大消息数。
-- 没有船只数据：确认订阅区域内确实有 AIS 活动，并检查 API Key 当前额度和权限。
-- 连接中断：实时流可能受网络和上游限流影响，collector 会记录源健康状态供聚合服务回退。
-""",
-)
-
-
-DEFAULT_CREDENTIAL_GUIDES = {
-    BARENTSWATCH_DEFAULT_GUIDE.provider: BARENTSWATCH_DEFAULT_GUIDE,
-    AISSTREAM_DEFAULT_GUIDE.provider: AISSTREAM_DEFAULT_GUIDE,
-}
-
-
-async def _get_guide_store(db) -> tuple[SystemSetting | None, dict[str, Any]]:
-    result = await db.execute(
-        select(SystemSetting).where(SystemSetting.category == CREDENTIAL_GUIDES_CATEGORY)
-    )
-    record = result.scalar_one_or_none()
-    payload = dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
-    return record, payload
-
-
-async def get_credential_guide(db, provider: str) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
-
-    _record, store = await _get_guide_store(db)
-    custom = store.get(provider) if isinstance(store.get(provider), dict) else None
-    return {
-        "provider": provider,
-        "title": custom.get("title") if custom else default.title,
-        "markdown": custom.get("markdown") if custom else default.markdown,
-        "prompt": default.prompt,
-        "source": "ai" if custom else "default",
-    }
-
-
-async def save_credential_guide(db, provider: str, title: str, markdown: str) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
-
-    record, store = await _get_guide_store(db)
-    store[provider] = {
-        "title": title or default.title,
-        "markdown": markdown,
-    }
-    if record is None:
-        db.add(SystemSetting(category=CREDENTIAL_GUIDES_CATEGORY, payload=store))
-    else:
-        record.payload = store
-    await db.commit()
-    return await get_credential_guide(db, provider)
-
-
-async def reset_credential_guide(db, provider: str) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
-
-    record, store = await _get_guide_store(db)
-    if provider in store:
-        store.pop(provider, None)
-        if record is not None:
-            record.payload = store
-            await db.commit()
-    return await get_credential_guide(db, provider)
-
-
-async def generate_credential_guide(
-    db,
-    provider: str,
-    ai_client: AIProviderClient,
-) -> dict[str, Any]:
-    default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
-    if default is None:
-        raise ValueError(f"Unsupported credential guide provider: {provider}")
-
-    response = await ai_client.analyze(
-        SituationalAnalysisRequest(
-            title=f"Generate credential guide for {provider}",
-            objective=default.prompt,
-            context={
-                "provider": provider,
-                "current_default_guide": default.markdown,
-                "product_context": "Planet collector credential settings",
-            },
-            observations=[
-                "Use concise Chinese markdown.",
-                "Prefer stable concepts over brittle UI labels.",
-                "Include verification and troubleshooting steps.",
-            ],
-            constraints=[
-                "Do not ask the user for secrets.",
-                "Do not include fabricated screenshots.",
-                "Return markdown only.",
-            ],
-        )
-    )
-    markdown = response.content.strip()
-    if not markdown:
-        markdown = default.markdown
-    return await save_credential_guide(db, provider, default.title, markdown)

backend/app/services/datasource_connectivity.py

@@ -1,437 +0,0 @@
-"""Connectivity validation helpers for built-in datasource overrides."""
-
-from __future__ import annotations
-
-from datetime import UTC, datetime
-import hashlib
-import json
-import os
-from typing import Any
-
-import httpx
-from sqlalchemy import func, select
-
-from app.core.data_sources import get_data_sources_config
-from app.core.datasource_defaults import DEFAULT_DATASOURCES
-from app.models.collected_data import CollectedData
-from app.models.datasource import DataSource
-from app.models.datasource_config import DataSourceConfig
-from app.models.system_setting import SystemSetting
-from app.services.barentswatch import (
-    _read_zshrc_env,
-    fetch_barentswatch_access_token,
-    resolve_barentswatch_config,
-)
-
-
-CONNECTIVITY_VALIDATION_KEY = "connectivity_validation"
-CONNECTIVITY_STORE_CATEGORY = "datasource_connectivity_validations"
-SUPPORTED_CREDENTIAL_PROVIDERS = {"barentswatch", "spacetrack", "aisstream"}
-
-
-def _sha256_json(payload: Any) -> str:
-    encoded = json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str).encode("utf-8")
-    return hashlib.sha256(encoded).hexdigest()
-
-
-def _resolve_spacetrack_credentials() -> tuple[str, str, str]:
-    zshrc_env = _read_zshrc_env()
-    username = os.getenv("SPACETRACK_USERNAME") or zshrc_env.get("SPACETRACK_USERNAME") or ""
-    password = os.getenv("SPACETRACK_PASSWORD") or zshrc_env.get("SPACETRACK_PASSWORD") or ""
-    source = "environment" if os.getenv("SPACETRACK_USERNAME") or os.getenv("SPACETRACK_PASSWORD") else ""
-    if not source and (username or password):
-        source = "~/.zshrc"
-    return username, password, source or "missing"
-
-
-async def _resolve_aisstream_api_key(
-    db=None,
-    credential_override: dict[str, str] | None = None,
-) -> tuple[str, str]:
-    if credential_override and credential_override.get("api_key"):
-        return str(credential_override["api_key"]), "draft"
-
-    env_key = os.getenv("AISSTREAM_API_KEY")
-    zshrc_key = _read_zshrc_env().get("AISSTREAM_API_KEY")
-    if db is not None:
-        result = await db.execute(
-            select(DataSourceConfig)
-            .where(DataSourceConfig.name == "aisstream_vessels")
-            .where(DataSourceConfig.is_active.is_(True))
-        )
-        record = result.scalar_one_or_none()
-        if record:
-            auth_config = record.auth_config or {}
-            runtime_config = record.config or {}
-            api_key = auth_config.get("api_key") or runtime_config.get("api_key")
-            if api_key:
-                return str(api_key), "datasource_config"
-
-    if env_key:
-        return env_key, "environment"
-    if zshrc_key:
-        return zshrc_key, "~/.zshrc"
-    return "", "missing"
-
-
-def strip_connectivity_validation(config: dict | None) -> dict:
-    cleaned = dict(config or {})
-    cleaned.pop(CONNECTIVITY_VALIDATION_KEY, None)
-    return cleaned
-
-
-def merge_connectivity_validation(existing_config: dict | None, next_config: dict | None) -> dict:
-    merged = strip_connectivity_validation(next_config)
-    validation = (existing_config or {}).get(CONNECTIVITY_VALIDATION_KEY)
-    if validation:
-        merged[CONNECTIVITY_VALIDATION_KEY] = validation
-    return merged
-
-
-def get_connectivity_validation(config: DataSourceConfig | None) -> dict | None:
-    validation = (config.config or {}).get(CONNECTIVITY_VALIDATION_KEY) if config else None
-    return validation if isinstance(validation, dict) else None
-
-
-async def build_builtin_connectivity_checksum(
-    source: str,
-    endpoint: str,
-    auth_type: str,
-    headers: dict | None,
-    config: dict | None,
-    db=None,
-    credential_override: dict[str, str] | None = None,
-) -> tuple[str, dict[str, Any]]:
-    defaults = DEFAULT_DATASOURCES.get(source, {})
-    credential_provider = defaults.get("credential_provider")
-    credential_fingerprint = ""
-    credential_source = "none"
-    has_credentials = not defaults.get("requires_credentials", False)
-
-    if credential_provider == "barentswatch":
-        if credential_override:
-            client_id = credential_override.get("client_id", "")
-            client_secret = credential_override.get("client_secret", "")
-            credential_source = "draft"
-        else:
-            barentswatch_config = await resolve_barentswatch_config(db)
-            client_id = barentswatch_config.client_id
-            client_secret = barentswatch_config.client_secret
-            credential_source = barentswatch_config.credential_source
-        has_credentials = bool(client_id and client_secret)
-        credential_fingerprint = _sha256_json(
-            {
-                "client_id": client_id,
-                "client_secret": client_secret,
-            }
-        )
-    elif credential_provider == "spacetrack":
-        username, password, credential_source = _resolve_spacetrack_credentials()
-        has_credentials = bool(username and password)
-        credential_fingerprint = _sha256_json(
-            {
-                "username": username,
-                "password": password,
-            }
-        )
-    elif credential_provider == "aisstream":
-        api_key, credential_source = await _resolve_aisstream_api_key(db, credential_override)
-        has_credentials = bool(api_key)
-        credential_fingerprint = _sha256_json({"api_key": api_key})
-    elif defaults.get("requires_credentials"):
-        credential_source = str(credential_provider or "unsupported")
-
-    checksum_payload = {
-        "source": source,
-        "endpoint": endpoint,
-        "auth_type": "none",
-        "headers": headers or {},
-        "config": strip_connectivity_validation(config),
-        "credential_provider": credential_provider or "none",
-        "credential_fingerprint": credential_fingerprint,
-    }
-    return _sha256_json(checksum_payload), {
-        "requires_credentials": bool(defaults.get("requires_credentials", False)),
-        "credential_provider": credential_provider,
-        "credential_source": credential_source,
-        "has_credentials": has_credentials,
-    }
-
-
-async def test_builtin_connectivity(
-    source: str,
-    endpoint: str,
-    auth_type: str,
-    headers: dict | None,
-    config: dict | None,
-    db=None,
-    credential_override: dict[str, str] | None = None,
-) -> dict[str, Any]:
-    defaults = DEFAULT_DATASOURCES.get(source)
-    if not defaults:
-        return {
-            "success": False,
-            "message": "未知内置采集器，无法执行连接校验。",
-        }
-
-    checksum, credential_context = await build_builtin_connectivity_checksum(
-        source,
-        endpoint,
-        auth_type,
-        headers,
-        config,
-        db,
-        credential_override,
-    )
-    if credential_context["requires_credentials"] and not credential_context["has_credentials"]:
-        return {
-            "success": False,
-            "checksum": checksum,
-            "stage": "credentials",
-            "message": "该采集器需要凭证，请先到采集器凭证设置中配置。",
-            "settings_tab": "collector_credentials",
-            **credential_context,
-        }
-    if (
-        credential_context["requires_credentials"]
-        and credential_context["credential_provider"] not in SUPPORTED_CREDENTIAL_PROVIDERS
-    ):
-        return {
-            "success": False,
-            "checksum": checksum,
-            "stage": "credentials",
-            "message": "该采集器的凭证链路尚未接入，暂时无法完成连接校验。",
-            "settings_tab": "collector_credentials",
-            **credential_context,
-        }
-
-    request_headers = {str(key): str(value) for key, value in (headers or {}).items()}
-    request_config = strip_connectivity_validation(config)
-    timeout = float(request_config.get("timeout") or 30)
-    request_endpoint = endpoint
-
-    if credential_context["credential_provider"] == "aisstream":
-        if not str(request_endpoint).startswith(("ws://", "wss://")):
-            return {
-                "success": False,
-                "checksum": checksum,
-                "stage": "endpoint",
-                "message": "AISStream endpoint 必须是 ws:// 或 wss:// WebSocket 地址。",
-                **credential_context,
-            }
-        return {
-            "success": True,
-            "checksum": checksum,
-            "stage": "credentials",
-            "message": "AISStream 凭证已配置，WebSocket endpoint 格式有效。",
-            **credential_context,
-        }
-
-    try:
-        async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
-            if credential_context["credential_provider"] == "barentswatch":
-                barentswatch_config = await resolve_barentswatch_config(db)
-                token = await fetch_barentswatch_access_token(client, barentswatch_config)
-                if not token:
-                    return {
-                        "success": False,
-                        "checksum": checksum,
-                        "stage": "token",
-                        "message": "凭证可读取，但 token 响应中没有 access_token。",
-                        "settings_tab": "collector_credentials",
-                        **credential_context,
-                    }
-                request_headers["Authorization"] = f"Bearer {token}"
-            elif credential_context["credential_provider"] == "spacetrack":
-                username, password, _source = _resolve_spacetrack_credentials()
-                login_url = "https://www.space-track.org/ajaxauth/login"
-                login_response = await client.post(
-                    login_url,
-                    data={
-                        "identity": username,
-                        "password": password,
-                    },
-                )
-                login_response.raise_for_status()
-
-            started = datetime.now(UTC)
-            async with client.stream("GET", request_endpoint, headers=request_headers) as response:
-                response.raise_for_status()
-                status_code = response.status_code
-            elapsed_ms = (datetime.now(UTC) - started).total_seconds() * 1000
-
-        return {
-            "success": True,
-            "checksum": checksum,
-            "stage": "endpoint",
-            "message": "连接验证成功。",
-            "status_code": status_code,
-            "response_time_ms": elapsed_ms,
-            **credential_context,
-        }
-    except httpx.HTTPStatusError as exc:
-        return {
-            "success": False,
-            "checksum": checksum,
-            "stage": "endpoint",
-            "message": f"连接验证失败：HTTP {exc.response.status_code}",
-            "error": f"HTTP Error: {exc.response.status_code}",
-            **credential_context,
-        }
-    except httpx.HTTPError as exc:
-        return {
-            "success": False,
-            "checksum": checksum,
-            "stage": "network",
-            "message": f"连接验证失败：{exc.__class__.__name__}",
-            "error": str(exc),
-            **credential_context,
-        }
-
-
-def make_success_validation(checksum: str, result: dict[str, Any]) -> dict[str, Any]:
-    return {
-        "checksum": checksum,
-        "status": "success",
-        "validated_at": datetime.now(UTC).isoformat(),
-        "status_code": result.get("status_code"),
-        "credential_source": result.get("credential_source"),
-    }
-
-
-def is_builtin_validation_current(config: DataSourceConfig | None, checksum: str) -> bool:
-    validation = get_connectivity_validation(config)
-    return bool(
-        validation
-        and validation.get("status") == "success"
-        and validation.get("checksum") == checksum
-    )
-
-
-async def get_connectivity_store(db) -> dict[str, Any]:
-    result = await db.execute(
-        select(SystemSetting).where(SystemSetting.category == CONNECTIVITY_STORE_CATEGORY)
-    )
-    record = result.scalar_one_or_none()
-    return dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
-
-
-async def save_connectivity_success(
-    db,
-    source: str,
-    checksum: str,
-    result: dict[str, Any],
-    *,
-    connected_by: str,
-) -> dict[str, Any]:
-    store = await get_connectivity_store(db)
-    validation = {
-        **make_success_validation(checksum, result),
-        "connected_by": connected_by,
-    }
-    store[source] = validation
-
-    existing = await db.execute(
-        select(SystemSetting).where(SystemSetting.category == CONNECTIVITY_STORE_CATEGORY)
-    )
-    record = existing.scalar_one_or_none()
-    if record is None:
-        db.add(SystemSetting(category=CONNECTIVITY_STORE_CATEGORY, payload=store))
-    else:
-        record.payload = store
-    return validation
-
-
-async def load_builtin_override_config(db, source: str) -> DataSourceConfig | None:
-    result = await db.execute(
-        select(DataSourceConfig)
-        .where(DataSourceConfig.name == source)
-        .where(DataSourceConfig.is_active.is_(True))
-    )
-    return result.scalar_one_or_none()
-
-
-async def get_builtin_effective_candidate(db, source: str) -> dict[str, Any]:
-    override = await load_builtin_override_config(db, source)
-    default_endpoint = get_data_sources_config().get_yaml_url(source)
-    return {
-        "name": source,
-        "endpoint": (override.endpoint if override and override.endpoint else default_endpoint) or "",
-        "auth_type": override.auth_type if override else "none",
-        "headers": override.headers if override else {},
-        "config": strip_connectivity_validation(override.config if override else {}),
-    }
-
-
-async def has_collected_data(db, source: str) -> bool:
-    result = await db.execute(select(func.count(CollectedData.id)).where(CollectedData.source == source))
-    if (result.scalar() or 0) > 0:
-        return True
-
-    datasource_result = await db.execute(select(DataSource).where(DataSource.source == source))
-    datasource = datasource_result.scalar_one_or_none()
-    return bool(datasource and datasource.last_status == "success")
-
-
-async def get_builtin_connection_status(
-    db,
-    source: str,
-    endpoint: str,
-    auth_type: str,
-    headers: dict | None,
-    config: dict | None,
-) -> dict[str, Any]:
-    checksum, credential_context = await build_builtin_connectivity_checksum(
-        source,
-        endpoint,
-        auth_type,
-        headers,
-        config,
-        db,
-    )
-    store = await get_connectivity_store(db)
-    validation = store.get(source)
-    if isinstance(validation, dict) and validation.get("status") == "success":
-        if validation.get("checksum") == checksum:
-            return {
-                "connected": True,
-                "checksum": checksum,
-                "connected_by": validation.get("connected_by") or "connection_button",
-                "message": "当前配置已完成连接验证。",
-                **credential_context,
-            }
-
-    effective = await get_builtin_effective_candidate(db, source)
-    effective_checksum, _ = await build_builtin_connectivity_checksum(
-        source,
-        effective["endpoint"],
-        effective["auth_type"],
-        effective["headers"],
-        effective["config"],
-        db,
-    )
-    if checksum == effective_checksum and await has_collected_data(db, source):
-        return {
-            "connected": True,
-            "checksum": checksum,
-            "connected_by": "collection",
-            "message": "当前配置已有成功采集数据，视为已连接。",
-            **credential_context,
-        }
-
-    if isinstance(validation, dict) and validation.get("status") == "success":
-        return {
-            "connected": False,
-            "checksum": checksum,
-            "connected_by": None,
-            "message": "接口地址或凭证指纹已变化，请重新点击连接验证。",
-            **credential_context,
-        }
-
-    return {
-        "connected": False,
-        "checksum": checksum,
-        "connected_by": None,
-        "message": "当前配置尚未连接，请点击连接验证。",
-        **credential_context,
-    }

backend/app/services/datasource_mapping.py

@@ -1,358 +0,0 @@
-"""Deterministic mapping support for custom data sources."""
-
-from __future__ import annotations
-
-import hashlib
-import json
-import re
-from datetime import UTC, datetime
-from typing import Any
-
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from app.core.target_schema_registry import TargetSchema, get_target_schema
-
-SECRET_KEY_PATTERN = re.compile(
-    r"(token|secret|password|passwd|authorization|api[_-]?key|client[_-]?secret)",
-    re.IGNORECASE,
-)
-
-
-class MappingError(ValueError):
-    """Raised when a mapping definition cannot be executed."""
-
-
-def stable_payload_hash(payload: Any) -> str:
-    encoded = json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str).encode()
-    return hashlib.sha256(encoded).hexdigest()
-
-
-def redact_for_llm(value: Any) -> Any:
-    if isinstance(value, dict):
-        redacted = {}
-        for key, item in value.items():
-            if SECRET_KEY_PATTERN.search(str(key)):
-                redacted[key] = "[REDACTED]"
-            else:
-                redacted[key] = redact_for_llm(item)
-        return redacted
-    if isinstance(value, list):
-        return [redact_for_llm(item) for item in value[:20]]
-    return value
-
-
-def extract_path(payload: Any, path: str | None) -> Any:
-    if not path or path == "$":
-        return payload
-
-    normalized = path.strip()
-    if normalized.startswith("$."):
-        normalized = normalized[2:]
-    elif normalized.startswith("$"):
-        normalized = normalized[1:]
-    normalized = normalized.strip(".")
-    if not normalized:
-        return payload
-
-    current = payload
-    for raw_segment in normalized.split("."):
-        segment = raw_segment.strip()
-        if not segment:
-            continue
-
-        list_all = segment.endswith("[*]")
-        if list_all:
-            segment = segment[:-3]
-
-        index = None
-        match = re.fullmatch(r"(.+)\[(\d+)\]", segment)
-        if match:
-            segment = match.group(1)
-            index = int(match.group(2))
-
-        if segment:
-            if isinstance(current, dict):
-                current = current.get(segment)
-            else:
-                return None
-
-        if list_all:
-            return current if isinstance(current, list) else []
-
-        if index is not None:
-            if not isinstance(current, list) or index >= len(current):
-                return None
-            current = current[index]
-
-    return current
-
-
-def _convert_value(value: Any, target_type: str | None) -> Any:
-    if value is None or target_type in (None, "", "any"):
-        return value
-
-    if target_type == "string":
-        return str(value)
-    if target_type == "integer":
-        return int(value)
-    if target_type == "float":
-        return float(value)
-    if target_type == "boolean":
-        if isinstance(value, bool):
-            return value
-        if isinstance(value, str):
-            return value.strip().lower() in {"1", "true", "yes", "y", "on"}
-        return bool(value)
-    if target_type == "datetime":
-        if isinstance(value, datetime):
-            return value
-        if isinstance(value, (int, float)):
-            return datetime.fromtimestamp(value)
-        if isinstance(value, str):
-            return datetime.fromisoformat(value.replace("Z", "+00:00"))
-        return value
-    if target_type == "object":
-        if isinstance(value, dict):
-            return value
-        raise ValueError("expected object")
-    if target_type == "array":
-        if isinstance(value, list):
-            return value
-        raise ValueError("expected array")
-
-    return value
-
-
-def _apply_enum(value: Any, enum_map: Any) -> Any:
-    if not isinstance(enum_map, dict):
-        return value
-    key = str(value)
-    return enum_map.get(key, enum_map.get(value, value))
-
-
-def _map_one(item: Any, field_mapping: dict[str, Any]) -> tuple[dict[str, Any], list[str]]:
-    output: dict[str, Any] = {}
-    errors: list[str] = []
-
-    for field_name, rule in field_mapping.items():
-        if isinstance(rule, str):
-            rule = {"path": rule}
-        if not isinstance(rule, dict):
-            errors.append(f"{field_name}: mapping rule must be an object or path string")
-            continue
-
-        value = extract_path(item, rule.get("path"))
-        if value is None and "default" in rule:
-            value = rule.get("default")
-        value = _apply_enum(value, rule.get("enum"))
-
-        try:
-            value = _convert_value(value, rule.get("type"))
-        except (TypeError, ValueError) as exc:
-            errors.append(f"{field_name}: failed to convert value {value!r}: {exc}")
-            continue
-
-        if value is not None or rule.get("include_null", False):
-            output[field_name] = value
-
-    return output, errors
-
-
-def execute_mapping(
-    payload: Any,
-    mapping_json: dict[str, Any],
-    target_schema: str | TargetSchema,
-    *,
-    limit: int | None = None,
-) -> dict[str, Any]:
-    schema = get_target_schema(target_schema) if isinstance(target_schema, str) else target_schema
-    source = mapping_json.get("source") or {}
-    fields = mapping_json.get("fields")
-    if not isinstance(fields, dict) or not fields:
-        raise MappingError("mapping_json.fields must be a non-empty object")
-
-    items_path = source.get("items_path") or mapping_json.get("items_path") or "$"
-    items = extract_path(payload, items_path)
-    if isinstance(items, dict):
-        items = [items]
-    elif not isinstance(items, list):
-        items = []
-
-    if limit is not None:
-        items = items[:limit]
-
-    mapped_records: list[dict[str, Any]] = []
-    errors: list[dict[str, Any]] = []
-    for index, item in enumerate(items):
-        mapped, mapping_errors = _map_one(item, fields)
-        validated, validation_errors = schema.validate_record(mapped)
-        all_errors = mapping_errors + validation_errors
-        if all_errors:
-            errors.append({"index": index, "errors": all_errors, "record": mapped})
-            continue
-        if validated is not None:
-            mapped_records.append(validated)
-
-    return {
-        "target_schema": schema.key,
-        "total_items": len(items),
-        "mapped_count": len(mapped_records),
-        "failed_count": len(errors),
-        "records": mapped_records,
-        "errors": errors,
-    }
-
-
-def build_heuristic_mapping(sample_payload: Any, target_schema_key: str) -> dict[str, Any]:
-    schema = get_target_schema(target_schema_key)
-    items_path = "$"
-    sample_item = sample_payload
-    if isinstance(sample_payload, dict):
-        for key in ("data", "items", "results", "features", "vessels"):
-            candidate = sample_payload.get(key)
-            if isinstance(candidate, list) and candidate:
-                items_path = f"$.{key}[*]"
-                sample_item = candidate[0]
-                break
-    elif isinstance(sample_payload, list) and sample_payload:
-        items_path = "$"
-        sample_item = sample_payload[0]
-
-    available = _flatten_keys(sample_item if isinstance(sample_item, dict) else {})
-    fields: dict[str, Any] = {}
-    for field in schema.fields:
-        candidate = _best_field_match(field.name, available)
-        if candidate:
-            fields[field.name] = {"path": f"$.{candidate}", "type": field.type}
-        elif field.name == "data" and target_schema_key == "generic_records":
-            fields[field.name] = {"path": "$", "type": "object"}
-        elif not field.required:
-            fields[field.name] = {"path": f"$.{field.name}", "type": field.type, "default": None}
-
-    return {
-        "source": {"items_path": items_path},
-        "fields": fields,
-        "meta": {
-            "generated_by": "heuristic",
-            "requires_review": True,
-        },
-    }
-
-
-def _flatten_keys(payload: dict[str, Any], prefix: str = "") -> list[str]:
-    keys: list[str] = []
-    for key, value in payload.items():
-        dotted = f"{prefix}.{key}" if prefix else str(key)
-        keys.append(dotted)
-        if isinstance(value, dict):
-            keys.extend(_flatten_keys(value, dotted))
-    return keys
-
-
-def _best_field_match(field_name: str, candidates: list[str]) -> str | None:
-    aliases = {
-        "lat": ("lat", "latitude", "y"),
-        "lon": ("lon", "lng", "longitude", "x"),
-        "mmsi": ("mmsi",),
-        "sog": ("sog", "speed", "speedOverGround"),
-        "cog": ("cog", "course", "courseOverGround"),
-        "received_at": ("received_at", "timestamp", "time", "updated_at"),
-        "observed_at": ("observed_at", "timestamp", "time", "updated_at"),
-        "source_id": ("id", "source_id", "uuid"),
-    }.get(field_name, (field_name,))
-
-    lowered = {candidate.lower(): candidate for candidate in candidates}
-    for alias in aliases:
-        if alias.lower() in lowered:
-            return lowered[alias.lower()]
-    for candidate in candidates:
-        tail = candidate.split(".")[-1].lower()
-        if tail in {alias.lower() for alias in aliases}:
-            return candidate
-    return None
-
-
-def _parse_datetime(value: Any) -> datetime | None:
-    if value is None:
-        return None
-    if isinstance(value, datetime):
-        return value
-    if isinstance(value, str):
-        return datetime.fromisoformat(value.replace("Z", "+00:00"))
-    return None
-
-
-async def persist_mapped_records(
-    db: AsyncSession,
-    *,
-    datasource_name: str,
-    datasource_config_id: int,
-    target_schema: str,
-    records: list[dict[str, Any]],
-    mapping_version: int,
-) -> int:
-    """Persist validated mapped records to the destination for a target schema."""
-    if target_schema == "vessel_ais":
-        from app.models.vessel import VesselPosition
-
-        for record in records:
-            db.add(
-                VesselPosition(
-                    mmsi=record["mmsi"],
-                    lat=record["lat"],
-                    lon=record["lon"],
-                    sog=record.get("sog"),
-                    cog=record.get("cog"),
-                    heading=record.get("heading"),
-                    received_at=_parse_datetime(record.get("received_at")) or datetime.now(UTC),
-                )
-            )
-        await db.commit()
-        return len(records)
-
-    from app.models.collected_data import CollectedData
-
-    collected_at = datetime.now(UTC)
-    for index, record in enumerate(records):
-        if target_schema == "geo_points":
-            source_id = record.get("source_id") or f"{datasource_config_id}:{index}"
-            name = record.get("name")
-            metadata = {
-                "latitude": record.get("lat"),
-                "longitude": record.get("lon"),
-                "type": record.get("type"),
-                "mapping_version": mapping_version,
-                "target_schema": target_schema,
-                **(record.get("metadata") or {}),
-            }
-            reference_date = _parse_datetime(record.get("observed_at"))
-        else:
-            source_id = record.get("source_id") or f"{datasource_config_id}:{index}"
-            name = None
-            metadata = {
-                "data": record.get("data") or {},
-                "mapping_version": mapping_version,
-                "target_schema": target_schema,
-            }
-            reference_date = _parse_datetime(record.get("observed_at"))
-
-        db.add(
-            CollectedData(
-                source=datasource_name,
-                source_id=str(source_id),
-                entity_key=f"{datasource_name}:{source_id}",
-                data_type=target_schema,
-                name=name,
-                title=name,
-                extra_data=metadata,
-                collected_at=collected_at,
-                reference_date=reference_date,
-                is_valid=1,
-                is_current=True,
-                change_type="created",
-                change_summary={},
-            )
-        )
-
-    await db.commit()
-    return len(records)

backend/app/services/earth_news.py

@@ -30,14 +30,6 @@ class RegionProfile:
     accent: str
 
 
-@dataclass(frozen=True)
-class RegionAnchor:
-    region: str
-    label: str
-    latitude: float
-    longitude: float
-
-
 @dataclass(frozen=True)
 class NewsFeedSource:
     id: str
@@ -103,39 +95,6 @@ REGION_PROFILES: dict[str, RegionProfile] = {
     ),
 }
 
-REGION_ANCHORS: dict[str, RegionAnchor] = {
-    "americas": RegionAnchor(
-        region="americas",
-        label="美洲",
-        latitude=37.0902,
-        longitude=-95.7129,
-    ),
-    "europe": RegionAnchor(
-        region="europe",
-        label="欧洲",
-        latitude=50.1109,
-        longitude=8.6821,
-    ),
-    "middle-east-africa": RegionAnchor(
-        region="middle-east-africa",
-        label="中东与非洲",
-        latitude=25.2048,
-        longitude=55.2708,
-    ),
-    "asia-pacific": RegionAnchor(
-        region="asia-pacific",
-        label="亚太",
-        latitude=1.3521,
-        longitude=103.8198,
-    ),
-    "global": RegionAnchor(
-        region="global",
-        label="全球",
-        latitude=20.0,
-        longitude=0.0,
-    ),
-}
-
 
 def _google_news_feed(query: str, *, hl: str, gl: str, ceid: str) -> str:
     return (
@@ -254,10 +213,6 @@ def get_region_profile(region: str) -> RegionProfile:
     return REGION_PROFILES.get(region, REGION_PROFILES["global"])
 
 
-def get_region_anchor(region: str) -> RegionAnchor:
-    return REGION_ANCHORS.get(region, REGION_ANCHORS["global"])
-
-
 def get_sources_for_region(region: str) -> list[NewsFeedSource]:
     return sorted(
         [source for source in NEWS_FEED_SOURCES if source.region in {"global", region}],
@@ -387,7 +342,6 @@ def _serialize_sources(sources: list[NewsFeedSource]) -> list[dict[str, Any]]:
 
 def _serialize_item(item: ParsedNewsItem, *, active_region: str) -> dict[str, Any]:
     published_at = item.published_at
-    anchor = get_region_anchor(item.feed_region)
     return {
         "id": item.id,
         "title": item.title,
@@ -398,10 +352,6 @@ def _serialize_item(item: ParsedNewsItem, *, active_region: str) -> dict[str, An
         "region": item.feed_region,
         "homepage_url": item.homepage_url,
         "published_at": published_at.isoformat().replace("+00:00", "Z") if published_at else None,
-        "latitude": anchor.latitude,
-        "longitude": anchor.longitude,
-        "location_label": anchor.label,
-        "location_inferred": True,
         "is_focus_match": item.feed_region == active_region,
     }
 

backend/app/services/llm_provider_catalog.py

@@ -1,150 +0,0 @@
-"""LLM provider presets used by Settings and the runtime AI provider bridge."""
-
-from __future__ import annotations
-
-from typing import Any
-
-import httpx
-
-MODELS_DEV_URL = "https://models.dev/api.json"
-
-
-FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
-    "minimax": {
-        "provider": "minimax",
-        "label": "MiniMax",
-        "provider_api": "anthropic-messages",
-        "base_url": "https://api.minimaxi.com/anthropic",
-        "model": "MiniMax-M2.7",
-        "models": ["MiniMax-M2.7", "MiniMax-M2.7-highspeed", "MiniMax-M2.5", "MiniMax-M2"],
-        "api_key_env": "MINIMAX_API_KEY",
-        "source": "fallback",
-    },
-    "openai": {
-        "provider": "openai",
-        "label": "OpenAI",
-        "provider_api": "openai-completions",
-        "base_url": "https://api.openai.com/v1",
-        "model": "gpt-5.1",
-        "models": ["gpt-5.1", "gpt-5.1-codex", "gpt-4.1", "gpt-4o"],
-        "api_key_env": "OPENAI_API_KEY",
-        "source": "fallback",
-    },
-    "anthropic": {
-        "provider": "anthropic",
-        "label": "Anthropic",
-        "provider_api": "anthropic-messages",
-        "base_url": "https://api.anthropic.com/v1",
-        "model": "claude-sonnet-4-6",
-        "models": ["claude-sonnet-4-6", "claude-opus-4-5", "claude-3-5-haiku-20241022"],
-        "api_key_env": "ANTHROPIC_API_KEY",
-        "source": "fallback",
-    },
-    "deepseek": {
-        "provider": "deepseek",
-        "label": "DeepSeek",
-        "provider_api": "openai-completions",
-        "base_url": "https://api.deepseek.com/v1",
-        "model": "deepseek-chat",
-        "models": ["deepseek-chat", "deepseek-reasoner"],
-        "api_key_env": "DEEPSEEK_API_KEY",
-        "source": "fallback",
-    },
-    "alibaba": {
-        "provider": "alibaba",
-        "label": "Alibaba Qwen / DashScope",
-        "provider_api": "openai-completions",
-        "base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
-        "model": "qwen3-max",
-        "models": ["qwen3-max", "qwen3.5-plus", "qwen-max", "qwen-plus"],
-        "api_key_env": "DASHSCOPE_API_KEY",
-        "source": "fallback",
-    },
-    "moonshotai": {
-        "provider": "moonshotai",
-        "label": "Moonshot AI / Kimi",
-        "provider_api": "openai-completions",
-        "base_url": "https://api.moonshot.ai/v1",
-        "model": "kimi-k2.5",
-        "models": ["kimi-k2.5", "kimi-k2-thinking", "kimi-k2-turbo-preview"],
-        "api_key_env": "MOONSHOT_API_KEY",
-        "source": "fallback",
-    },
-    "openrouter": {
-        "provider": "openrouter",
-        "label": "OpenRouter",
-        "provider_api": "openai-completions",
-        "base_url": "https://openrouter.ai/api/v1",
-        "model": "openai/gpt-5.1",
-        "models": ["openai/gpt-5.1", "anthropic/claude-sonnet-4.5", "qwen/qwen3-max"],
-        "api_key_env": "OPENROUTER_API_KEY",
-        "source": "fallback",
-    },
-    "ollama": {
-        "provider": "ollama",
-        "label": "Ollama Local",
-        "provider_api": "ollama-generate",
-        "base_url": "http://127.0.0.1:11434",
-        "model": "qwen2.5:7b",
-        "models": ["qwen2.5:7b", "llama3.1:8b", "mistral:7b"],
-        "api_key_env": "",
-        "source": "fallback",
-    },
-}
-
-MODELS_DEV_PROVIDER_KEYS = {
-    "minimax": "minimax",
-    "openai": "openai",
-    "anthropic": "anthropic",
-    "deepseek": "deepseek",
-    "alibaba": "alibaba",
-    "moonshotai": "moonshotai",
-    "openrouter": "openrouter",
-}
-
-
-def list_fallback_llm_provider_presets() -> list[dict[str, Any]]:
-    return [dict(value) for value in FALLBACK_LLM_PROVIDER_PRESETS.values()]
-
-
-def get_fallback_llm_provider_preset(provider: str) -> dict[str, Any]:
-    key = provider.strip().lower()
-    if key not in FALLBACK_LLM_PROVIDER_PRESETS:
-        raise ValueError(f"Unsupported LLM provider preset: {provider}")
-    return dict(FALLBACK_LLM_PROVIDER_PRESETS[key])
-
-
-async def refresh_llm_provider_preset(provider: str) -> dict[str, Any]:
-    fallback = get_fallback_llm_provider_preset(provider)
-    models_dev_key = MODELS_DEV_PROVIDER_KEYS.get(fallback["provider"])
-    if not models_dev_key:
-        return fallback
-
-    async with httpx.AsyncClient(timeout=15.0, follow_redirects=True) as client:
-        response = await client.get(
-            MODELS_DEV_URL,
-            headers={"User-Agent": "Planet/1.0"},
-        )
-        response.raise_for_status()
-        catalog = response.json()
-
-    upstream = catalog.get(models_dev_key)
-    if not isinstance(upstream, dict):
-        return fallback
-
-    upstream_models = upstream.get("models") if isinstance(upstream.get("models"), dict) else {}
-    model_ids = list(upstream_models.keys())[:80]
-    base_url = upstream.get("api") or fallback["base_url"]
-    if fallback["provider"] == "deepseek" and base_url == "https://api.deepseek.com":
-        base_url = "https://api.deepseek.com/v1"
-
-    refreshed = {
-        **fallback,
-        "label": upstream.get("name") or fallback["label"],
-        "base_url": base_url,
-        "model": model_ids[0] if model_ids else fallback["model"],
-        "models": model_ids or fallback["models"],
-        "api_key_env": (upstream.get("env") or [fallback["api_key_env"]])[0],
-        "source": MODELS_DEV_URL,
-    }
-    return refreshed

backend/app/services/persistent_logs.py

@@ -1,86 +0,0 @@
-from __future__ import annotations
-
-from typing import Any
-
-from app.core.logging import get_logger, sanitize_log_value
-from app.core.request_context import get_request_id
-from app.db.session import async_session_factory
-from app.models.system_log import AuditLog, SystemLog
-
-logger = get_logger(__name__)
-
-
-async def record_system_log(
-    *,
-    source: str,
-    level: str,
-    message: str,
-    service: str | None = None,
-    module: str | None = None,
-    event: str | None = None,
-    request_id: str | None = None,
-    trace_id: str | None = None,
-    user_id: int | None = None,
-    category: str | None = None,
-    context: dict[str, Any] | None = None,
-) -> None:
-    try:
-        async with async_session_factory() as session:
-            session.add(
-                SystemLog(
-                    source=source,
-                    service=service,
-                    module=module,
-                    event=event,
-                    level=level.lower(),
-                    message=str(sanitize_log_value(message)),
-                    request_id=request_id or get_request_id(),
-                    trace_id=trace_id,
-                    user_id=user_id,
-                    category=category,
-                    context=sanitize_log_value(context or {}),
-                )
-            )
-            await session.commit()
-    except Exception:
-        logger.exception_event(
-            "Failed to persist system log",
-            event="system_log.persist.failed",
-            context={"event_name": event, "source": source},
-        )
-
-
-async def record_audit_log(
-    *,
-    action: str,
-    actor_id: int | None = None,
-    actor_name: str | None = None,
-    target_type: str | None = None,
-    target_id: str | None = None,
-    result: str | None = None,
-    request_id: str | None = None,
-    ip: str | None = None,
-    details: dict[str, Any] | None = None,
-) -> None:
-    try:
-        async with async_session_factory() as session:
-            session.add(
-                AuditLog(
-                    actor_id=actor_id,
-                    actor_name=actor_name,
-                    action=action,
-                    target_type=target_type,
-                    target_id=target_id,
-                    result=result,
-                    request_id=request_id or get_request_id(),
-                    ip=ip,
-                    details=sanitize_log_value(details or {}),
-                )
-            )
-            await session.commit()
-    except Exception:
-        logger.exception_event(
-            "Failed to persist audit log",
-            event="audit_log.persist.failed",
-            context={"action": action},
-        )

backend/app/services/scheduler.py

@@ -1,6 +1,7 @@
 """Task Scheduler for running collection jobs."""
 
 import asyncio
+import logging
 from datetime import UTC, datetime, timedelta
 from typing import Any, Dict, Optional
 
@@ -8,19 +9,13 @@ from apscheduler.schedulers.asyncio import AsyncIOScheduler
 from apscheduler.triggers.interval import IntervalTrigger
 from sqlalchemy import select
 
-from app.core.logging import get_logger
 from app.db.session import async_session_factory
 from app.core.time import to_iso8601_utc
 from app.models.datasource import DataSource
 from app.models.task import CollectionTask
 from app.services.collectors.registry import collector_registry
-from app.services.datasource_connectivity import (
-    build_builtin_connectivity_checksum,
-    get_builtin_effective_candidate,
-    save_connectivity_success,
-)
 
-logger = get_logger(__name__)
+logger = logging.getLogger(__name__)
 
 scheduler = AsyncIOScheduler()
 RUNNING_TASK_GUARD_TIMEOUT_MINUTES = 90
@@ -59,11 +54,7 @@ async def _update_next_run_at(datasource: DataSource, session) -> None:
 async def _apply_datasource_schedule(datasource: DataSource, session) -> None:
     collector = collector_registry.get(datasource.source)
     if not collector:
-        logger.warning_event(
-            "Collector not found for datasource",
-            event="collector.schedule.collector_missing",
-            context={"collector_name": datasource.source},
-        )
+        logger.warning("Collector not found for datasource %s", datasource.source)
         return
 
     collector_registry.set_active(datasource.source, datasource.is_active)
@@ -81,17 +72,13 @@ async def _apply_datasource_schedule(datasource: DataSource, session) -> None:
             replace_existing=True,
             kwargs={"collector_name": datasource.source},
         )
-        logger.info_event(
-            "Scheduled collector",
-            event="collector.schedule.updated",
-            context={"collector_name": datasource.source, "frequency_minutes": datasource.frequency_minutes},
+        logger.info(
+            "Scheduled collector: %s (every %sm)",
+            datasource.source,
+            datasource.frequency_minutes,
         )
     else:
-        logger.info_event(
-            "Collector disabled",
-            event="collector.schedule.disabled",
-            context={"collector_name": datasource.source},
-        )
+        logger.info("Collector disabled: %s", datasource.source)
 
     await _update_next_run_at(datasource, session)
 
@@ -100,30 +87,18 @@ async def run_collector_task(collector_name: str):
     """Run a single collector task."""
     collector = collector_registry.get(collector_name)
     if not collector:
-        logger.error_event(
-            "Collector not found",
-            event="collector.run.collector_missing",
-            context={"collector_name": collector_name},
-        )
+        logger.error("Collector not found: %s", collector_name)
         return
 
     async with async_session_factory() as db:
         result = await db.execute(select(DataSource).where(DataSource.source == collector_name))
         datasource = result.scalar_one_or_none()
         if not datasource:
-            logger.error_event(
-                "Datasource not found for collector",
-                event="collector.run.datasource_missing",
-                context={"collector_name": collector_name},
-            )
+            logger.error("Datasource not found for collector: %s", collector_name)
             return
 
         if not datasource.is_active:
-            logger.info_event(
-                "Skipping disabled collector",
-                event="collector.run.skipped_disabled",
-                context={"collector_name": collector_name},
-            )
+            logger.info("Skipping disabled collector: %s", collector_name)
             return
 
         running_result = await db.execute(
@@ -147,10 +122,10 @@ async def run_collector_task(collector_name: str):
                 and (now - started_at) > timedelta(minutes=RUNNING_TASK_GUARD_TIMEOUT_MINUTES)
             )
             if not is_stale:
-                logger.warning_event(
-                    "Skipping collector trigger because task is already running",
-                    event="collector.run.skipped_already_running",
-                    context={"collector_name": collector_name, "task_id": existing_running.id},
+                logger.warning(
+                    "Skipping collector %s trigger because task %s is already running",
+                    collector_name,
+                    existing_running.id,
                 )
                 return
 
@@ -168,64 +143,31 @@ async def run_collector_task(collector_name: str):
                 else stale_reason
             )
             await db.commit()
-            logger.warning_event(
-                "Marked stale running task as failed before rerun",
-                event="collector.run.stale_task_failed",
-                context={"collector_name": collector_name, "task_id": existing_running.id},
+            logger.warning(
+                "Marked stale running task %s as failed before rerun of %s",
+                existing_running.id,
+                collector_name,
             )
 
         try:
             collector._datasource_id = datasource.id
-            logger.info_event(
-                "Running collector",
-                event="collector.run.started",
-                context={"collector_name": collector_name, "datasource_id": datasource.id},
-            )
+            logger.info("Running collector: %s (datasource_id=%s)", collector_name, datasource.id)
             task_result = await collector.run(db)
             datasource.last_run_at = datetime.now(UTC)
             datasource.last_status = task_result.get("status")
-            if datasource.last_status == "success":
-                effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
-                checksum, _credential_context = await build_builtin_connectivity_checksum(
-                    datasource.source,
-                    effective_candidate["endpoint"],
-                    effective_candidate["auth_type"],
-                    effective_candidate["headers"],
-                    effective_candidate["config"],
-                    db,
-                )
-                await save_connectivity_success(
-                    db,
-                    datasource.source,
-                    checksum,
-                    {"status_code": None},
-                    connected_by="collection",
-                )
             await _update_next_run_at(datasource, db)
-            logger.info_event(
-                "Collector completed",
-                event="collector.run.completed",
-                context={"collector_name": collector_name, "datasource_id": datasource.id, "result": task_result},
-            )
+            logger.info("Collector %s completed: %s", collector_name, task_result)
         except asyncio.CancelledError:
             datasource.last_run_at = datetime.now(UTC)
             datasource.last_status = "cancelled"
             await db.commit()
-            logger.warning_event(
-                "Collector cancelled by operator",
-                event="collector.run.cancelled",
-                context={"collector_name": collector_name, "datasource_id": datasource.id},
-            )
+            logger.warning("Collector %s cancelled by operator", collector_name)
             raise
         except Exception as exc:
             datasource.last_run_at = datetime.now(UTC)
             datasource.last_status = "failed"
             await db.commit()
-            logger.exception_event(
-                "Collector failed",
-                event="collector.run.failed",
-                context={"collector_name": collector_name, "datasource_id": datasource.id, "error": str(exc)},
-            )
+            logger.exception("Collector %s failed: %s", collector_name, exc)
 
 
 async def cleanup_stale_running_tasks(max_age_hours: int = 2) -> int:
@@ -252,11 +194,7 @@ async def cleanup_stale_running_tasks(max_age_hours: int = 2) -> int:
 
         if stale_tasks:
             await db.commit()
-            logger.warning_event(
-                "Cleaned up stale running collection tasks",
-                event="collector.cleanup.stale_tasks_cleaned",
-                context={"count": len(stale_tasks)},
-            )
+            logger.warning("Cleaned up %s stale running collection task(s)", len(stale_tasks))
 
         return len(stale_tasks)
 
@@ -265,14 +203,14 @@ def start_scheduler() -> None:
     """Start the scheduler."""
     if not scheduler.running:
         scheduler.start()
-        logger.info_event("Scheduler started", event="scheduler.started")
+        logger.info("Scheduler started")
 
 
 def stop_scheduler() -> None:
     """Stop the scheduler."""
     if scheduler.running:
         scheduler.shutdown(wait=False)
-        logger.info_event("Scheduler stopped", event="scheduler.stopped")
+        logger.info("Scheduler stopped")
 
 
 async def sync_scheduler_with_datasources() -> None:
@@ -333,20 +271,12 @@ def run_collector_now(collector_name: str) -> bool:
     """Run a collector immediately (not scheduled)."""
     collector = collector_registry.get(collector_name)
     if not collector:
-        logger.error_event(
-            "Collector not found",
-            event="collector.trigger.collector_missing",
-            context={"collector_name": collector_name},
-        )
+        logger.error("Collector not found: %s", collector_name)
         return False
 
     existing_task = get_running_collector_task(collector_name)
     if existing_task is not None and not existing_task.done():
-        logger.warning_event(
-            "Collector is already running in-memory; skipping duplicate trigger",
-            event="collector.trigger.skipped_already_running",
-            context={"collector_name": collector_name},
-        )
+        logger.warning("Collector %s is already running in-memory; skipping duplicate trigger", collector_name)
         return False
 
     try:
@@ -359,18 +289,10 @@ def run_collector_now(collector_name: str) -> bool:
                 RUNNING_COLLECTOR_TASKS.pop(collector_name, None)
 
         task.add_done_callback(_cleanup_task)
-        logger.info_event(
-            "Triggered collector",
-            event="collector.trigger.started",
-            context={"collector_name": collector_name},
-        )
+        logger.info("Triggered collector: %s", collector_name)
         return True
     except Exception as exc:
-        logger.error_event(
-            "Failed to trigger collector",
-            event="collector.trigger.failed",
-            context={"collector_name": collector_name, "error": str(exc)},
-        )
+        logger.error("Failed to trigger collector %s: %s", collector_name, exc)
         return False
 
 

backend/app/services/system_control.py

@@ -19,10 +19,6 @@ ALLOWED_ACTIONS: dict[str, dict[str, Any]] = {
         "command": ["./planet.sh", "restart", "-b"],
         "recovery_mode": "backend",
     },
-    "restart-frontend": {
-        "command": ["./planet.sh", "restart", "-f"],
-        "recovery_mode": "frontend",
-    },
     "restart-ai-provider": {
         "command": ["./planet.sh", "restart", "-a"],
         "recovery_mode": "ai-provider",

backend/app/services/system_logs.py

@@ -1,532 +0,0 @@
-from __future__ import annotations
-
-import json
-import re
-import shutil
-import subprocess
-
-from collections import Counter, deque
-from dataclasses import dataclass
-from datetime import UTC, datetime
-from pathlib import Path
-from typing import Any
-
-from app.core.security import redis_client
-
-DEFAULT_LOG_LINE_LIMIT = 200
-MAX_LOG_LINE_LIMIT = 1000
-BUFFER_LOG_LIMIT = 1000
-BUFFER_LOG_TTL_SECONDS = 7 * 24 * 60 * 60
-LOG_BUFFER_KEY_PREFIX = "planet:system_logs"
-
-LOG_LEVEL_ERROR = "error"
-LOG_LEVEL_WARNING = "warning"
-LOG_LEVEL_INFO = "info"
-LOG_LEVEL_DEBUG = "debug"
-LOG_LEVEL_ALL = "all"
-
-SUPPORTED_LOG_LEVELS = {
-    LOG_LEVEL_ALL,
-    LOG_LEVEL_ERROR,
-    LOG_LEVEL_WARNING,
-    LOG_LEVEL_INFO,
-    LOG_LEVEL_DEBUG,
-}
-
-LOG_LEVEL_ALIASES = {
-    "warn": LOG_LEVEL_WARNING,
-    "warning": LOG_LEVEL_WARNING,
-    "err": LOG_LEVEL_ERROR,
-    "error": LOG_LEVEL_ERROR,
-    "info": LOG_LEVEL_INFO,
-    "information": LOG_LEVEL_INFO,
-    "debug": LOG_LEVEL_DEBUG,
-    "trace": LOG_LEVEL_DEBUG,
-    "critical": LOG_LEVEL_ERROR,
-    "fatal": LOG_LEVEL_ERROR,
-}
-
-TIMESTAMP_FORMATS = (
-    "%Y-%m-%d %H:%M:%S.%f",
-    "%Y-%m-%d %H:%M:%S",
-    "%Y-%m-%dT%H:%M:%S.%f",
-    "%Y-%m-%dT%H:%M:%S",
-)
-
-LEVEL_PATTERNS = (
-    ("CRITICAL", LOG_LEVEL_ERROR),
-    ("FATAL", LOG_LEVEL_ERROR),
-    ("ERROR", LOG_LEVEL_ERROR),
-    ("WARNING", LOG_LEVEL_WARNING),
-    ("WARN", LOG_LEVEL_WARNING),
-    ("INFO", LOG_LEVEL_INFO),
-    ("DEBUG", LOG_LEVEL_DEBUG),
-    ("TRACE", LOG_LEVEL_DEBUG),
-)
-
-LEADING_LEVEL_PATTERN = re.compile(
-    r"^\s*(?:\[[^\]]+\]\s*)?(CRITICAL|FATAL|ERROR|WARNING|WARN|INFO|DEBUG|TRACE)\b[:\s-]*",
-    re.IGNORECASE,
-)
-EMBEDDED_LEVEL_PATTERN = re.compile(
-    r"\b(CRITICAL|FATAL|ERROR|WARNING|WARN|INFO|DEBUG|TRACE)\b",
-    re.IGNORECASE,
-)
-CONTROL_CHAR_PATTERN = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]")
-
-
-@dataclass(frozen=True)
-class LogSource:
-    source_id: str
-    name: str
-    kind: str
-    location: str
-    description: str
-    category: str
-    status: str = "ok"
-    buffer_key: str | None = None
-    container_name: str | None = None
-
-
-@dataclass
-class StructuredLogEntry:
-    timestamp: datetime | None
-    level: str | None
-    display_line: str
-    raw_line: str
-    search_text: str
-
-
-@dataclass
-class DailyLogMarker:
-    date_token: str
-    total: int
-    dominant_level: str
-
-
-LOG_SOURCES: dict[str, LogSource] = {
-    "backend": LogSource(
-        source_id="backend",
-        name="后端服务",
-        kind="file",
-        location="/tmp/planet_backend.log",
-        description="FastAPI 后端、调度器和采集任务共享日志。",
-        category="service",
-    ),
-    "frontend": LogSource(
-        source_id="frontend",
-        name="前端开发服务",
-        kind="file",
-        location="/tmp/planet_frontend.log",
-        description="控制台与 Earth 前端开发服务输出。",
-        category="service",
-    ),
-    "ai-provider": LogSource(
-        source_id="ai-provider",
-        name="AI Provider",
-        kind="docker",
-        location="docker://planet_aiprovider",
-        description="AI Provider 容器实时输出日志。",
-        category="service",
-        container_name="planet_aiprovider",
-    ),
-    "earth-client": LogSource(
-        source_id="earth-client",
-        name="Earth 浏览器端",
-        kind="buffer",
-        location="redis://planet:system_logs:earth-client",
-        description="Earth 浏览器端上报的运行时错误与关键业务日志。",
-        category="client",
-        buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:earth-client",
-    ),
-}
-
-
-def normalize_log_level(level: str | None) -> str:
-    if level is None:
-        return LOG_LEVEL_ALL
-    normalized = str(level).strip().lower()
-    if normalized in {"", LOG_LEVEL_ALL}:
-        return LOG_LEVEL_ALL
-    return LOG_LEVEL_ALIASES.get(normalized, LOG_LEVEL_ALL)
-
-
-def normalize_log_levels(level: str | None = None, levels: str | None = None) -> tuple[str, ...]:
-    normalized_levels: list[str] = []
-    if levels:
-        for item in str(levels).split(","):
-            normalized = normalize_log_level(item)
-            if normalized != LOG_LEVEL_ALL and normalized not in normalized_levels:
-                normalized_levels.append(normalized)
-    normalized_level = normalize_log_level(level)
-    if normalized_level != LOG_LEVEL_ALL and normalized_level not in normalized_levels:
-        normalized_levels.append(normalized_level)
-    return tuple(normalized_levels)
-
-
-def get_source_status(source: LogSource) -> str:
-    if source.kind == "file":
-        path = Path(source.location)
-        if not path.exists():
-            return "missing"
-        return "ok" if path.stat().st_size > 0 else "empty"
-    if source.kind == "docker":
-        return "ok" if shutil.which("docker") else "docker_unavailable"
-    if source.kind == "buffer":
-        if not source.buffer_key:
-            return "source_unavailable"
-        try:
-            return "ok" if redis_client.llen(source.buffer_key) > 0 else "empty"
-        except Exception:
-            return "source_unavailable"
-    return "source_unavailable"
-
-
-def list_log_sources() -> list[dict[str, str]]:
-    items: list[dict[str, str]] = []
-    for source in LOG_SOURCES.values():
-        items.append(
-            {
-                "source_id": source.source_id,
-                "name": source.name,
-                "kind": source.kind,
-                "location": source.location,
-                "description": source.description,
-                "category": source.category,
-                "status": get_source_status(source),
-            }
-        )
-    return items
-
-
-def get_buffer_log_key(source_id: str) -> str:
-    return f"{LOG_BUFFER_KEY_PREFIX}:{source_id}"
-
-
-def append_buffer_log(
-    source_id: str,
-    *,
-    level: str,
-    message: str,
-    context: dict[str, Any] | None = None,
-) -> None:
-    payload = {
-        "timestamp": datetime.now(tz=UTC).isoformat(),
-        "level": normalize_log_level(level),
-        "message": message,
-        "context": context or {},
-    }
-    buffer_key = get_buffer_log_key(source_id)
-    redis_client.rpush(buffer_key, json.dumps(payload, ensure_ascii=False))
-    redis_client.ltrim(buffer_key, -BUFFER_LOG_LIMIT, -1)
-    redis_client.expire(buffer_key, BUFFER_LOG_TTL_SECONDS)
-
-
-def parse_timestamp(raw_value: str | None) -> datetime | None:
-    if not raw_value:
-        return None
-    candidate = str(raw_value).strip()
-    if not candidate:
-        return None
-    candidate = candidate.replace("Z", "+00:00")
-    try:
-        parsed = datetime.fromisoformat(candidate)
-        return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
-    except ValueError:
-        pass
-    for fmt in TIMESTAMP_FORMATS:
-        try:
-            return datetime.strptime(candidate, fmt).replace(tzinfo=UTC)
-        except ValueError:
-            continue
-    return None
-
-
-def parse_prefixed_timestamp(line: str) -> tuple[datetime | None, str]:
-    stripped = line.strip()
-    if not stripped:
-        return None, ""
-    for prefix_length in (35, 32, 29, 26, 23, 19):
-        if len(stripped) < prefix_length:
-            continue
-        prefix = stripped[:prefix_length]
-        timestamp = parse_timestamp(prefix)
-        if timestamp is not None:
-            return timestamp, stripped[prefix_length:].lstrip()
-    first_token = stripped.split(maxsplit=1)[0]
-    timestamp = parse_timestamp(first_token)
-    if timestamp is not None:
-        remainder = stripped[len(first_token):].lstrip()
-        return timestamp, remainder
-    return None, stripped
-
-
-def infer_log_level_from_text(text: str, *, allow_embedded: bool = True) -> str | None:
-    leading_match = LEADING_LEVEL_PATTERN.match(text)
-    if leading_match:
-        return normalize_log_level(leading_match.group(1))
-
-    if allow_embedded:
-        embedded_match = EMBEDDED_LEVEL_PATTERN.search(text)
-        if embedded_match:
-            return normalize_log_level(embedded_match.group(1))
-        upper_text = text.upper()
-        for pattern, normalized in LEVEL_PATTERNS:
-            if f"{pattern}:" in upper_text or f"{pattern} " in upper_text:
-                return normalized
-    return None
-
-
-def build_display_line(timestamp: datetime | None, level: str | None, message: str) -> str:
-    message_part = message.strip() if message else ""
-    parts = []
-    if timestamp is not None:
-        parts.append(timestamp.astimezone(UTC).strftime("%Y-%m-%d %H:%M:%S"))
-    if level:
-        parts.append(level.upper())
-    if message_part:
-        parts.append(message_part)
-    return " ".join(parts).strip()
-
-
-def sanitize_text_log_line(line: str) -> str:
-    return CONTROL_CHAR_PATTERN.sub("", line)
-
-
-def parse_text_log_entry(line: str) -> StructuredLogEntry:
-    sanitized_line = sanitize_text_log_line(line).rstrip("\n")
-    timestamp, remainder = parse_prefixed_timestamp(sanitized_line)
-    level = infer_log_level_from_text(remainder or sanitized_line, allow_embedded=False)
-    display_line = sanitized_line
-    return StructuredLogEntry(
-        timestamp=timestamp,
-        level=level,
-        display_line=display_line,
-        raw_line=display_line,
-        search_text=display_line.lower(),
-    )
-
-
-def build_buffer_entry(payload: dict[str, Any]) -> StructuredLogEntry:
-    timestamp = parse_timestamp(str(payload.get("timestamp", "")).strip())
-    level = normalize_log_level(payload.get("level"))
-    if level == LOG_LEVEL_ALL:
-        level = None
-    message = str(payload.get("message", "")).strip()
-    context = payload.get("context")
-    context_map = context if isinstance(context, dict) else {}
-    context_fragments = []
-    for key in ("category", "module", "url", "detail"):
-        value = str(context_map.get(key, "")).strip()
-        if value:
-            context_fragments.append(f"{key}={value}")
-    message_with_context = " | ".join([message, *context_fragments]) if context_fragments else message
-    display_line = build_display_line(timestamp, level, message_with_context)
-    search_text = " ".join(
-        [
-            message,
-            json.dumps(context_map, ensure_ascii=False, sort_keys=True),
-            display_line,
-        ]
-    ).lower()
-    return StructuredLogEntry(
-        timestamp=timestamp,
-        level=level,
-        display_line=display_line,
-        raw_line=json.dumps(payload, ensure_ascii=False, sort_keys=True),
-        search_text=search_text,
-    )
-
-
-def read_file_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
-    path = Path(source.location)
-    if not path.exists():
-        return []
-    with path.open("r", encoding="utf-8", errors="replace") as handle:
-        recent_lines = deque(handle, maxlen=scan_limit)
-    return [
-        parse_text_log_entry(line)
-        for line in recent_lines
-        if sanitize_text_log_line(line).strip()
-    ]
-
-
-def read_docker_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
-    if not shutil.which("docker") or not source.container_name:
-        return []
-    try:
-        completed = subprocess.run(
-            [
-                "docker",
-                "logs",
-                "--timestamps",
-                "--tail",
-                str(scan_limit),
-                source.container_name,
-            ],
-            capture_output=True,
-            text=True,
-            check=False,
-        )
-    except OSError:
-        return []
-    if completed.returncode != 0:
-        return []
-    return [
-        parse_text_log_entry(line)
-        for line in completed.stdout.splitlines()
-        if line.strip()
-    ]
-
-
-def read_buffer_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
-    if not source.buffer_key:
-        return []
-    try:
-        raw_items = redis_client.lrange(source.buffer_key, -scan_limit, -1)
-    except Exception:
-        return []
-    entries: list[StructuredLogEntry] = []
-    for raw_item in raw_items:
-        try:
-            payload = json.loads(raw_item)
-        except json.JSONDecodeError:
-            entries.append(parse_text_log_entry(str(raw_item)))
-            continue
-        if isinstance(payload, dict):
-            entries.append(build_buffer_entry(payload))
-        else:
-            entries.append(parse_text_log_entry(str(raw_item)))
-    return entries
-
-
-def read_source_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
-    if source.kind == "file":
-        return read_file_entries(source, scan_limit)
-    if source.kind == "docker":
-        return read_docker_entries(source, scan_limit)
-    if source.kind == "buffer":
-        return read_buffer_entries(source, scan_limit)
-    return []
-
-
-def matches_levels(entry: StructuredLogEntry, selected_levels: tuple[str, ...]) -> bool:
-    if not selected_levels:
-        return True
-    return entry.level in selected_levels
-
-
-def matches_date_range(
-    entry: StructuredLogEntry,
-    start_date: str | None,
-    end_date: str | None,
-) -> bool:
-    if not start_date and not end_date:
-        return True
-    if entry.timestamp is None:
-        return False
-    date_token = entry.timestamp.astimezone(UTC).date().isoformat()
-    if start_date and date_token < start_date:
-        return False
-    if end_date and date_token > end_date:
-        return False
-    return True
-
-
-def matches_search(entry: StructuredLogEntry, search: str | None) -> bool:
-    if search is None:
-        return True
-    query = search.strip().lower()
-    if not query:
-        return True
-    return query in entry.search_text
-
-
-def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str, Any]]:
-    grouped: dict[str, list[StructuredLogEntry]] = {}
-    for entry in entries:
-        if entry.timestamp is None:
-            continue
-        date_token = entry.timestamp.astimezone(UTC).date().isoformat()
-        grouped.setdefault(date_token, []).append(entry)
-
-    markers: list[DailyLogMarker] = []
-    for date_token, group in sorted(grouped.items()):
-        level_counts = Counter(
-            entry.level
-            for entry in group
-            if entry.level in SUPPORTED_LOG_LEVELS and entry.level != LOG_LEVEL_ALL
-        )
-        dominant_level = LOG_LEVEL_INFO
-        if level_counts:
-            dominant_level = sorted(
-                level_counts.items(),
-                key=lambda item: (
-                    -item[1],
-                    ("error", "warning", "info", "debug").index(item[0]),
-                ),
-            )[0][0]
-        markers.append(
-            DailyLogMarker(
-                date_token=date_token,
-                total=len(group),
-                dominant_level=dominant_level,
-            )
-        )
-    return [marker.__dict__ for marker in markers]
-
-
-def read_log_snapshot(
-    source_id: str,
-    limit: int,
-    *,
-    level: str = LOG_LEVEL_ALL,
-    levels: str | None = None,
-    start_date: str | None = None,
-    end_date: str | None = None,
-    search: str | None = None,
-) -> dict[str, Any] | None:
-    source = LOG_SOURCES.get(source_id)
-    if source is None:
-        return None
-
-    selected_levels = normalize_log_levels(level, levels)
-    search_query = (search or "").strip()
-    scan_limit = max(min(MAX_LOG_LINE_LIMIT * 5, 5000), limit * 5, BUFFER_LOG_LIMIT if source.kind == "buffer" else 1000)
-    all_entries = read_source_entries(source, scan_limit)
-    marker_entries = [
-        entry
-        for entry in all_entries
-        if matches_levels(entry, selected_levels) and matches_search(entry, search_query)
-    ]
-    filtered_entries = [
-        entry
-        for entry in marker_entries
-        if matches_date_range(entry, start_date, end_date)
-    ]
-    visible_entries = filtered_entries[-limit:]
-
-    compatibility_level = selected_levels[0] if len(selected_levels) == 1 else LOG_LEVEL_ALL
-    return {
-        "source_id": source.source_id,
-        "name": source.name,
-        "kind": source.kind,
-        "location": source.location,
-        "description": source.description,
-        "category": source.category,
-        "status": get_source_status(source),
-        "level": compatibility_level,
-        "selected_levels": list(selected_levels),
-        "search_query": search_query,
-        "available_levels": [
-            LOG_LEVEL_ALL,
-            LOG_LEVEL_ERROR,
-            LOG_LEVEL_WARNING,
-            LOG_LEVEL_INFO,
-            LOG_LEVEL_DEBUG,
-        ],
-        "daily_markers": build_daily_log_markers(marker_entries),
-        "line_limit": limit,
-        "line_count": len(visible_entries),
-        "lines": [entry.display_line for entry in visible_entries],
-    }

backend/app/services/vessel_ais_aggregation.py

@@ -1,574 +0,0 @@
-"""AIS raw observation and aggregation support for vessel collectors."""
-
-from datetime import UTC, datetime
-from hashlib import sha256
-import json
-from typing import Any, Iterable
-
-from sqlalchemy import select
-from sqlalchemy.ext.asyncio import AsyncSession
-
-from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth
-from app.services.vessel_types import normalize_vessel_type_name
-
-VESSEL_AIS_SCHEMA = "vessel_ais"
-BARENTSWATCH_DELIVERY_MODE = "polling"
-BARENTSWATCH_TRANSPORT = "http"
-AISSTREAM_DELIVERY_MODE = "realtime_stream"
-AISSTREAM_TRANSPORT = "websocket"
-DELIVERY_MODE_PRIORITY = {
-    "realtime_stream": 40,
-    "batch_stream": 30,
-    "polling": 20,
-    "snapshot": 10,
-}
-DYNAMIC_FIELDS = ("lat", "lon", "sog", "cog", "heading", "nav_status")
-CONFLICT_FIELDS = (
-    "name",
-    "callsign",
-    "imo",
-    "flag",
-    "vessel_type",
-    "vessel_type_name",
-    "length",
-    "width",
-    "draught",
-)
-
-
-def _json_default(value: Any) -> Any:
-    if isinstance(value, datetime):
-        return value.astimezone(UTC).isoformat()
-    return str(value)
-
-
-def _stable_payload(value: Any) -> str:
-    return json.dumps(value, sort_keys=True, separators=(",", ":"), default=_json_default)
-
-
-def _jsonable(value: Any) -> Any:
-    if isinstance(value, datetime):
-        return value.astimezone(UTC).isoformat()
-    if isinstance(value, dict):
-        return {str(key): _jsonable(item) for key, item in value.items()}
-    if isinstance(value, list):
-        return [_jsonable(item) for item in value]
-    return value
-
-
-def _coerce_datetime(value: Any) -> datetime | None:
-    if isinstance(value, datetime):
-        return value if value.tzinfo else value.replace(tzinfo=UTC)
-    if isinstance(value, (int, float)):
-        timestamp = float(value)
-        if timestamp > 10_000_000_000:
-            timestamp /= 1000
-        return datetime.fromtimestamp(timestamp, UTC)
-    if isinstance(value, str) and value:
-        try:
-            parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
-            return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
-        except ValueError:
-            return None
-    return None
-
-
-def build_observation_hash(
-    *,
-    source: str,
-    entity_key: str,
-    message_type: str | None,
-    observed_at: datetime,
-    normalized_payload: dict[str, Any],
-    source_message_id: str | None = None,
-) -> str:
-    """Build a deterministic idempotency key for one source-level AIS observation."""
-
-    if source_message_id:
-        basis = {
-            "source": source,
-            "entity_key": entity_key,
-            "source_message_id": source_message_id,
-        }
-    else:
-        basis = {
-            "source": source,
-            "entity_key": entity_key,
-            "message_type": message_type,
-            "observed_at": observed_at.astimezone(UTC).isoformat(),
-            "payload": normalized_payload,
-        }
-    return sha256(_stable_payload(basis).encode("utf-8")).hexdigest()
-
-
-def build_field_conflict_candidates(
-    observations: Iterable[AISRawObservation],
-    fields: Iterable[str] = CONFLICT_FIELDS,
-) -> list[dict[str, Any]]:
-    """Return current field disagreements from raw observations without mutating state."""
-
-    candidates_by_field: dict[str, dict[str, Any]] = {}
-    for observation in observations:
-        payload = observation.normalized_payload or {}
-        for field in fields:
-            value = payload.get(field)
-            if value in (None, ""):
-                continue
-            field_candidates = candidates_by_field.setdefault(field, {})
-            field_candidates[observation.source] = value
-
-    conflicts = []
-    for field, candidates in sorted(candidates_by_field.items()):
-        unique_values = {_stable_payload(value) for value in candidates.values()}
-        if len(unique_values) <= 1:
-            continue
-        conflicts.append(
-            {
-                "field": field,
-                "candidates": candidates,
-                "status": "candidate",
-            }
-        )
-    return conflicts
-
-
-def _payload_value(payload: dict[str, Any], field: str) -> Any:
-    value = payload.get(field)
-    return None if value in (None, "") else value
-
-
-def _clean_text(value: Any) -> str | None:
-    if value in (None, ""):
-        return None
-    text = str(value).strip()
-    return text or None
-
-
-def _raw_metadata_value(observation: AISRawObservation, field: str) -> Any:
-    raw_payload = observation.raw_payload or {}
-    metadata = raw_payload.get("MetaData") if isinstance(raw_payload, dict) else None
-    if not isinstance(metadata, dict):
-        return None
-    if field == "name":
-        return _clean_text(metadata.get("ShipName") or metadata.get("ship_name") or metadata.get("name"))
-    return None
-
-
-def _delivery_priority(observation: AISRawObservation) -> int:
-    return DELIVERY_MODE_PRIORITY.get(str(observation.delivery_mode or ""), 0)
-
-
-def _has_valid_position(payload: dict[str, Any]) -> bool:
-    try:
-        lat = float(payload.get("lat"))
-        lon = float(payload.get("lon"))
-    except (TypeError, ValueError):
-        return False
-    return -90 <= lat <= 90 and -180 <= lon <= 180
-
-
-def _is_future_observation(observation: AISRawObservation, now: datetime) -> bool:
-    return observation.observed_at > now
-
-
-def _select_position_observation(
-    observations: list[AISRawObservation],
-    *,
-    now: datetime,
-) -> tuple[AISRawObservation | None, list[str]]:
-    rejected_flags: list[str] = []
-    candidates = []
-    for observation in observations:
-        payload = observation.normalized_payload or {}
-        if not _has_valid_position(payload):
-            rejected_flags.append("invalid_position")
-            continue
-        if _is_future_observation(observation, now):
-            rejected_flags.append("future_timestamp")
-            continue
-        candidates.append(observation)
-
-    if not candidates:
-        return None, sorted(set(rejected_flags))
-
-    candidates.sort(
-        key=lambda item: (
-            item.observed_at,
-            _delivery_priority(item),
-            item.collected_at,
-            item.id or 0,
-        ),
-        reverse=True,
-    )
-    return candidates[0], sorted(set(rejected_flags))
-
-
-def _select_static_field(
-    observations: list[AISRawObservation],
-    field: str,
-) -> tuple[Any, str | None, str | None]:
-    candidates = []
-    for observation in observations:
-        value = _payload_value(observation.normalized_payload or {}, field)
-        if value is None:
-            value = _raw_metadata_value(observation, field)
-        if value is None:
-            continue
-        candidates.append((observation, value))
-
-    if not candidates:
-        return None, None, None
-
-    candidates.sort(
-        key=lambda item: (
-            _delivery_priority(item[0]),
-            item[0].observed_at,
-            item[0].collected_at,
-            item[0].id or 0,
-        ),
-        reverse=True,
-    )
-    selected_observation, selected_value = candidates[0]
-    unique_values = {_stable_payload(value) for _, value in candidates}
-    reason = "delivery_mode_priority" if len(unique_values) > 1 else "non_empty_priority"
-    return selected_value, selected_observation.source, reason
-
-
-def _build_source_summary(observations: list[AISRawObservation]) -> dict[str, dict[str, Any]]:
-    summary: dict[str, dict[str, Any]] = {}
-    for observation in observations:
-        source_summary = summary.setdefault(
-            observation.source,
-            {
-                "observation_count": 0,
-                "latest_observed_at": None,
-                "delivery_mode": observation.delivery_mode,
-                "transport": observation.transport,
-                "message_types": [],
-            },
-        )
-        source_summary["observation_count"] += 1
-        latest_observed_at = source_summary["latest_observed_at"]
-        if latest_observed_at is None or observation.observed_at > latest_observed_at:
-            source_summary["latest_observed_at"] = observation.observed_at
-        if observation.message_type and observation.message_type not in source_summary["message_types"]:
-            source_summary["message_types"].append(observation.message_type)
-    return summary
-
-
-def _build_aggregated_vessel(
-    entity_key: str,
-    observations: list[AISRawObservation],
-    *,
-    now: datetime,
-) -> dict[str, Any] | None:
-    position_observation, rejected_flags = _select_position_observation(observations, now=now)
-    if position_observation is None:
-        return None
-
-    payload = position_observation.normalized_payload or {}
-    mmsi = int(entity_key)
-    result: dict[str, Any] = {
-        "mmsi": mmsi,
-        "lat": float(payload["lat"]),
-        "lon": float(payload["lon"]),
-        "received_at": position_observation.observed_at,
-        "field_sources": {},
-        "selected_reasons": {},
-        "source_summary": _build_source_summary(observations),
-        "quality_flags": sorted(
-            set((position_observation.quality_flags or []) + rejected_flags)
-        ),
-    }
-
-    for field in DYNAMIC_FIELDS:
-        value = _payload_value(payload, field)
-        if field in ("lat", "lon") or value is not None:
-            result[field] = value
-            result["field_sources"][field] = position_observation.source
-            result["selected_reasons"][field] = "newest_observation"
-
-    for field in CONFLICT_FIELDS:
-        selected_value, selected_source, reason = _select_static_field(observations, field)
-        if selected_value is None:
-            continue
-        result[field] = selected_value
-        result["field_sources"][field] = selected_source
-        result["selected_reasons"][field] = reason
-
-    result["name"] = result.get("name") or f"MMSI {mmsi}"
-    result["vessel_type_name"] = result.get("vessel_type_name") or normalize_vessel_type_name(
-        result.get("vessel_type")
-    )
-    return result
-
-
-async def _upsert_conflict_records(
-    db: AsyncSession,
-    entity_key: str,
-    observations: list[AISRawObservation],
-    aggregated: dict[str, Any],
-) -> int:
-    conflicts = build_field_conflict_candidates(observations)
-    now = datetime.now(UTC)
-    for conflict in conflicts:
-        field = conflict["field"]
-        result = await db.execute(
-            select(AISConflictRecord)
-            .where(AISConflictRecord.target_schema == VESSEL_AIS_SCHEMA)
-            .where(AISConflictRecord.entity_key == entity_key)
-            .where(AISConflictRecord.field == field)
-            .limit(1)
-        )
-        record = result.scalar_one_or_none()
-        if record is None:
-            record = AISConflictRecord(
-                target_schema=VESSEL_AIS_SCHEMA,
-                entity_key=entity_key,
-                field=field,
-            )
-            db.add(record)
-        record.candidates = conflict["candidates"]
-        record.selected_source = (aggregated.get("field_sources") or {}).get(field)
-        record.selected_value = aggregated.get(field)
-        record.selected_reason = (aggregated.get("selected_reasons") or {}).get(field)
-        record.resolved_by = "system"
-        record.status = "open"
-        record.updated_at = now
-    return len(conflicts)
-
-
-def _group_observations(observations: Iterable[AISRawObservation]) -> dict[str, list[AISRawObservation]]:
-    grouped: dict[str, list[AISRawObservation]] = {}
-    for observation in observations:
-        grouped.setdefault(str(observation.entity_key), []).append(observation)
-    return grouped
-
-
-async def record_vessel_ais_observation(
-    db: AsyncSession,
-    *,
-    source: str,
-    normalized_payload: dict[str, Any],
-    raw_payload: dict[str, Any] | None = None,
-    delivery_mode: str,
-    transport: str,
-    message_type: str | None = "PositionReport",
-    source_message_id: str | None = None,
-    observed_at: datetime | None = None,
-    collected_at: datetime | None = None,
-    quality_flags: list[str] | None = None,
-) -> AISRawObservation | None:
-    """Insert one raw observation if the source-level fact has not already been stored."""
-
-    entity_key = str(normalized_payload["mmsi"])
-    collected_at = collected_at or datetime.now(UTC)
-    observed_at = (
-        _coerce_datetime(observed_at)
-        or _coerce_datetime(normalized_payload.get("received_at"))
-        or collected_at
-    )
-    normalized_json = _jsonable(normalized_payload)
-    raw_json = _jsonable(raw_payload or {})
-
-    observation_hash = build_observation_hash(
-        source=source,
-        entity_key=entity_key,
-        message_type=message_type,
-        observed_at=observed_at,
-        normalized_payload=normalized_json,
-        source_message_id=source_message_id,
-    )
-    existing_result = await db.execute(
-        select(AISRawObservation.id).where(AISRawObservation.observation_hash == observation_hash)
-    )
-    if existing_result.scalar_one_or_none() is not None:
-        return None
-
-    observation = AISRawObservation(
-        target_schema=VESSEL_AIS_SCHEMA,
-        source=source,
-        entity_key=entity_key,
-        delivery_mode=delivery_mode,
-        transport=transport,
-        message_type=message_type,
-        source_message_id=source_message_id,
-        observation_hash=observation_hash,
-        observed_at=observed_at,
-        collected_at=collected_at,
-        normalized_payload=normalized_json,
-        raw_payload=raw_json,
-        quality_flags=quality_flags or [],
-    )
-    db.add(observation)
-    return observation
-
-
-async def aggregate_vessel_observations(
-    db: AsyncSession,
-    observations: Iterable[AISRawObservation],
-    *,
-    write_conflicts: bool = True,
-) -> list[dict[str, Any]]:
-    now = datetime.now(UTC)
-    vessels = []
-    for entity_key, entity_observations in _group_observations(observations).items():
-        aggregated = _build_aggregated_vessel(entity_key, entity_observations, now=now)
-        if aggregated is None:
-            continue
-        if write_conflicts:
-            aggregated["conflict_count"] = await _upsert_conflict_records(
-                db,
-                entity_key,
-                entity_observations,
-                aggregated,
-            )
-        else:
-            aggregated["conflict_count"] = len(build_field_conflict_candidates(entity_observations))
-        vessels.append(aggregated)
-
-    vessels.sort(key=lambda item: item.get("received_at") or datetime.min.replace(tzinfo=UTC), reverse=True)
-    return vessels
-
-
-async def get_aggregated_vessels(
-    db: AsyncSession,
-    *,
-    bbox: tuple[float, float, float, float] | None = None,
-    limit: int | None = None,
-) -> list[dict[str, Any]]:
-    stmt = (
-        select(AISRawObservation)
-        .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-        .order_by(AISRawObservation.observed_at.desc(), AISRawObservation.id.desc())
-    )
-    if limit and limit > 0:
-        stmt = stmt.limit(max(limit * 20, limit))
-
-    result = await db.execute(stmt)
-    if not hasattr(result, "scalars"):
-        return []
-    vessels = await aggregate_vessel_observations(db, result.scalars().all())
-
-    if bbox is not None:
-        lon_min, lat_min, lon_max, lat_max = bbox
-        vessels = [
-            vessel
-            for vessel in vessels
-            if lon_min <= float(vessel["lon"]) <= lon_max
-            and lat_min <= float(vessel["lat"]) <= lat_max
-        ]
-
-    if limit and limit > 0:
-        return vessels[:limit]
-    return vessels
-
-
-async def get_aggregated_vessel(db: AsyncSession, mmsi: int) -> dict[str, Any] | None:
-    observations = await get_vessel_raw_observations(db, mmsi, limit=1000)
-    vessels = await aggregate_vessel_observations(db, observations)
-    return vessels[0] if vessels else None
-
-
-async def get_aggregated_vessel_track(
-    db: AsyncSession,
-    mmsi: int,
-    *,
-    cutoff: datetime,
-) -> list[dict[str, Any]]:
-    result = await db.execute(
-        select(AISRawObservation)
-        .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-        .where(AISRawObservation.entity_key == str(mmsi))
-        .where(AISRawObservation.observed_at >= cutoff)
-        .order_by(AISRawObservation.observed_at.asc(), AISRawObservation.id.asc())
-    )
-    if not hasattr(result, "scalars"):
-        return []
-
-    points: list[dict[str, Any]] = []
-    seen: set[tuple[str, float, float, str]] = set()
-    for observation in result.scalars().all():
-        payload = observation.normalized_payload or {}
-        if not _has_valid_position(payload):
-            continue
-        lat = float(payload["lat"])
-        lon = float(payload["lon"])
-        key = (
-            observation.observed_at.isoformat(),
-            round(lat, 5),
-            round(lon, 5),
-            observation.source,
-        )
-        if key in seen:
-            continue
-        seen.add(key)
-        points.append(
-            {
-                "lat": lat,
-                "lon": lon,
-                "observed_at": observation.observed_at,
-                "source": observation.source,
-                "selected_reason": "track_timeline",
-                "quality_flags": observation.quality_flags or [],
-            }
-        )
-    return points
-
-
-async def update_ais_source_health(
-    db: AsyncSession,
-    *,
-    source: str,
-    connection_state: str,
-    observed_count: int = 0,
-    last_seen_at: datetime | None = None,
-    last_success_at: datetime | None = None,
-    last_error: str | None = None,
-    lag_seconds: float | None = None,
-) -> AISSourceHealth:
-    """Upsert the health row for an AIS source."""
-
-    now = datetime.now(UTC)
-    health = await db.get(AISSourceHealth, source)
-    if health is None:
-        health = AISSourceHealth(source=source)
-        db.add(health)
-
-    health.connection_state = connection_state
-    health.last_seen_at = last_seen_at or health.last_seen_at
-    health.last_success_at = last_success_at or health.last_success_at
-    health.last_error = last_error
-    health.message_rate = float(observed_count)
-    health.lag_seconds = lag_seconds
-    health.updated_at = now
-    return health
-
-
-async def get_vessel_raw_observations(
-    db: AsyncSession,
-    mmsi: int,
-    *,
-    limit: int = 100,
-) -> list[AISRawObservation]:
-    result = await db.execute(
-        select(AISRawObservation)
-        .where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
-        .where(AISRawObservation.entity_key == str(mmsi))
-        .order_by(AISRawObservation.observed_at.desc(), AISRawObservation.id.desc())
-        .limit(limit)
-    )
-    return list(result.scalars().all())
-
-
-async def get_vessel_conflict_records(
-    db: AsyncSession,
-    mmsi: int,
-) -> list[AISConflictRecord]:
-    result = await db.execute(
-        select(AISConflictRecord)
-        .where(AISConflictRecord.target_schema == VESSEL_AIS_SCHEMA)
-        .where(AISConflictRecord.entity_key == str(mmsi))
-        .order_by(AISConflictRecord.updated_at.desc(), AISConflictRecord.id.desc())
-    )
-    return list(result.scalars().all())

backend/app/services/vessel_types.py

@@ -1,31 +0,0 @@
-"""Shared AIS vessel type helpers."""
-
-from typing import Any
-
-VESSEL_TYPE_NAMES = {
-    30: "Fishing",
-    35: "Military",
-    60: "Passenger",
-    70: "Cargo",
-    80: "Tanker",
-}
-
-
-def normalize_vessel_type_name(vessel_type: Any) -> str:
-    """Map AIS numeric vessel type codes to display buckets."""
-
-    try:
-        type_code = int(float(vessel_type))
-    except (TypeError, ValueError):
-        return "Other"
-    if 70 <= type_code <= 79:
-        return "Cargo"
-    if 80 <= type_code <= 89:
-        return "Tanker"
-    if 60 <= type_code <= 69:
-        return "Passenger"
-    if type_code == 30:
-        return "Fishing"
-    if type_code == 35:
-        return "Military"
-    return VESSEL_TYPE_NAMES.get(type_code, "Other")

backend/scripts/system_restart_runner.py

@@ -2,7 +2,6 @@ from __future__ import annotations
 
 import argparse
 import os
-import shlex
 import subprocess
 import sys
 import time
@@ -60,8 +59,6 @@ def wait_for_recovery(action: str) -> tuple[bool, str]:
     recovery_mode = get_action_recovery_mode(action)
     if recovery_mode == "backend":
         return wait_for_http("http://localhost:8000/health"), "backend health recovery"
-    if recovery_mode == "frontend":
-        return wait_for_http("http://localhost:3000"), "frontend entrypoint recovery"
     if recovery_mode == "ai-provider":
         return wait_for_http("http://localhost:8010/health"), "ai provider health recovery"
     if recovery_mode == "database":
@@ -111,9 +108,8 @@ def main() -> int:
         )
         append_task_log(args.task_id, "restart command started")
 
-        shell_command = shlex.join(command)
         completed = subprocess.run(
-            ["zsh", "-ic", shell_command],
+            command,
             cwd=str(ROOT_DIR),
             env=env,
             capture_output=True,

backend/tests/test_api.py

@@ -35,7 +35,6 @@ async def test_health_check():
     data = response.json()
     assert data["status"] == "healthy"
     assert "version" in data
-    assert response.headers["x-request-id"]
 
 
 @pytest.mark.asyncio
@@ -162,345 +161,6 @@ async def test_alerts_endpoint_with_auth(auth_headers):
         app.dependency_overrides.clear()
 
 
-@pytest.mark.asyncio
-async def test_system_log_sources_requires_super_admin(auth_headers):
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="testuser",
-            email="test@example.com",
-            password_hash="hashed",
-            role="admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {
-        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        async with AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.get("/api/v1/system/logs/sources", headers=auth_headers)
-        assert response.status_code == 403
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_system_log_sources_with_super_admin(auth_headers):
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="root",
-            email="root@example.com",
-            password_hash="hashed",
-            role="super_admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {
-        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        with patch(
-            "app.api.v1.system_control.list_log_sources",
-            return_value=[
-                {
-                    "source_id": "backend",
-                    "name": "后端服务",
-                    "kind": "file",
-                    "location": "/tmp/planet_backend.log",
-                    "description": "FastAPI 后端、调度器和采集任务共享日志。",
-                    "category": "service",
-                    "status": "ok",
-                }
-            ],
-        ):
-            async with AsyncClient(transport=transport, base_url="http://test") as client:
-                response = await client.get("/api/v1/system/logs/sources", headers=auth_headers)
-        assert response.status_code == 200
-        data = response.json()
-        assert data["items"][0]["source_id"] == "backend"
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_system_log_snapshot_with_super_admin(auth_headers):
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="root",
-            email="root@example.com",
-            password_hash="hashed",
-            role="super_admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {
-        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        with patch(
-            "app.api.v1.system_control.read_log_snapshot",
-            return_value={
-                "source_id": "backend",
-                "name": "后端服务",
-                "kind": "file",
-                "location": "/tmp/planet_backend.log",
-                "description": "FastAPI 后端、调度器和采集任务共享日志。",
-                "category": "service",
-                "status": "ok",
-                "level": "all",
-                "selected_levels": [],
-                "search_query": "",
-                "available_levels": ["all", "error", "warning", "info", "debug"],
-                "daily_markers": [],
-                "line_limit": 50,
-                "line_count": 2,
-                "lines": ["line 1", "line 2"],
-            },
-        ):
-            async with AsyncClient(transport=transport, base_url="http://test") as client:
-                response = await client.get("/api/v1/system/logs/backend?limit=50", headers=auth_headers)
-        assert response.status_code == 200
-        data = response.json()
-        assert data["source_id"] == "backend"
-        assert data["line_count"] == 2
-        assert data["lines"] == ["line 1", "line 2"]
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_system_log_snapshot_supports_level_filter(auth_headers):
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="root",
-            email="root@example.com",
-            password_hash="hashed",
-            role="super_admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {
-        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        with patch(
-            "app.api.v1.system_control.read_log_snapshot",
-            return_value={
-                "source_id": "backend",
-                "name": "后端服务",
-                "kind": "file",
-                "location": "/tmp/planet_backend.log",
-                "description": "FastAPI 后端、调度器和采集任务共享日志。",
-                "category": "service",
-                "status": "ok",
-                "level": "error",
-                "selected_levels": ["error"],
-                "search_query": "",
-                "available_levels": ["all", "error", "warning", "info", "debug"],
-                "daily_markers": [],
-                "line_limit": 50,
-                "line_count": 1,
-                "lines": ["ERROR: failed"],
-            },
-        ):
-            async with AsyncClient(transport=transport, base_url="http://test") as client:
-                response = await client.get("/api/v1/system/logs/backend?limit=50&level=error", headers=auth_headers)
-        assert response.status_code == 200
-        data = response.json()
-        assert data["level"] == "error"
-        assert data["lines"] == ["ERROR: failed"]
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_system_log_snapshot_supports_date_range_filter(auth_headers):
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="root",
-            email="root@example.com",
-            password_hash="hashed",
-            role="super_admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {
-        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        with patch(
-            "app.api.v1.system_control.read_log_snapshot",
-            return_value={
-                "source_id": "backend",
-                "name": "后端服务",
-                "kind": "file",
-                "location": "/tmp/planet_backend.log",
-                "description": "FastAPI 后端、调度器和采集任务共享日志。",
-                "category": "service",
-                "status": "ok",
-                "level": "all",
-                "selected_levels": [],
-                "search_query": "",
-                "available_levels": ["all", "error", "warning", "info", "debug"],
-                "daily_markers": [],
-                "line_limit": 50,
-                "line_count": 1,
-                "lines": ["2026-04-23 INFO: service started"],
-            },
-        ) as mock_read_log_snapshot:
-            async with AsyncClient(transport=transport, base_url="http://test") as client:
-                response = await client.get(
-                    "/api/v1/system/logs/backend?limit=50&start_date=2026-04-20&end_date=2026-04-23",
-                    headers=auth_headers,
-                )
-        assert response.status_code == 200
-        mock_read_log_snapshot.assert_called_once_with(
-            "backend",
-            50,
-            level="all",
-            levels=None,
-            start_date="2026-04-20",
-            end_date="2026-04-23",
-            search=None,
-        )
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_system_log_snapshot_supports_levels_and_search_filter(auth_headers):
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="root",
-            email="root@example.com",
-            password_hash="hashed",
-            role="super_admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {
-        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        with patch(
-            "app.api.v1.system_control.read_log_snapshot",
-            return_value={
-                "source_id": "backend",
-                "name": "后端服务",
-                "kind": "file",
-                "location": "/tmp/planet_backend.log",
-                "description": "FastAPI 后端、调度器和采集任务共享日志。",
-                "category": "service",
-                "status": "ok",
-                "level": "all",
-                "selected_levels": ["error", "warning"],
-                "search_query": "timeout",
-                "available_levels": ["all", "error", "warning", "info", "debug"],
-                "daily_markers": [],
-                "line_limit": 50,
-                "line_count": 1,
-                "lines": ["2026-04-23 10:00:00 ERROR timeout"],
-            },
-        ) as mock_read_log_snapshot:
-            async with AsyncClient(transport=transport, base_url="http://test") as client:
-                response = await client.get(
-                    "/api/v1/system/logs/backend?limit=50&levels=error,warning&search=timeout",
-                    headers=auth_headers,
-                )
-        assert response.status_code == 200
-        mock_read_log_snapshot.assert_called_once_with(
-            "backend",
-            50,
-            level="all",
-            levels="error,warning",
-            start_date=None,
-            end_date=None,
-            search="timeout",
-        )
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_system_log_snapshot_rejects_invalid_date_range(auth_headers):
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="root",
-            email="root@example.com",
-            password_hash="hashed",
-            role="super_admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {
-        __import__("app.core.security", fromlist=["get_current_user"]).get_current_user: override_get_current_user,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        async with AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.get(
-                "/api/v1/system/logs/backend?start_date=2026-04-31",
-                headers=auth_headers,
-            )
-        assert response.status_code == 400
-        assert "start_date must be in YYYY-MM-DD format" in response.json()["detail"]
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_ingest_earth_client_log_accepts_public_events():
-    transport = ASGITransport(app=app)
-    try:
-        with patch("app.api.v1.system_control.append_buffer_log") as mock_append_buffer_log:
-            with patch("app.api.v1.system_control.record_system_log", new_callable=AsyncMock) as mock_record_system_log:
-                async with AsyncClient(transport=transport, base_url="http://test") as client:
-                    response = await client.post(
-                        "/api/v1/system/logs/earth-client",
-                        json={
-                            "level": "error",
-                            "message": "登陆点加载失败: 登陆点接口返回 HTTP 500",
-                            "category": "startup-load",
-                            "module": "layer-startup",
-                        },
-                    )
-        assert response.status_code == 200
-        data = response.json()
-        assert data["accepted"] is True
-        assert data["source_id"] == "earth-client"
-        mock_append_buffer_log.assert_called_once()
-        mock_record_system_log.assert_awaited_once()
-        persisted_kwargs = mock_record_system_log.await_args.kwargs
-        assert persisted_kwargs["source"] == "earth-client"
-        assert persisted_kwargs["event"] == "earth.client.runtime_log"
-        assert persisted_kwargs["category"] == "startup-load"
-        assert persisted_kwargs["level"] == "error"
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_request_id_header_is_echoed_when_provided():
-    transport = ASGITransport(app=app)
-    async with AsyncClient(transport=transport, base_url="http://test") as client:
-        response = await client.get("/health", headers={"X-Request-ID": "planet-test-request"})
-    assert response.status_code == 200
-    assert response.headers["x-request-id"] == "planet-test-request"
-
-
 @pytest.mark.asyncio
 async def test_invalid_token():
     """Test that invalid token is rejected"""
@@ -603,8 +263,6 @@ async def test_ai_situational_analysis_returns_503_when_disabled(auth_headers):
         assert "content_blocks" in data
         assert "text_blocks" in data
         assert "thinking_blocks" in data
-    finally:
-        app.dependency_overrides.clear()
 
 
 @pytest.mark.asyncio
@@ -724,6 +382,8 @@ async def test_save_playground_session_with_auth(auth_headers):
         assert data["state"]["objective"] == "测试目标"
     finally:
         app.dependency_overrides.clear()
+    finally:
+        app.dependency_overrides.clear()
 
 
 @pytest.mark.asyncio

backend/tests/test_collectors.py

@@ -1,14 +1,11 @@
 """Unit tests for data collectors"""
 
 import pytest
-from unittest.mock import AsyncMock, patch
+from datetime import datetime
+from unittest.mock import AsyncMock, MagicMock, patch
 
-from app.core.datasource_defaults import DEFAULT_DATASOURCES
-from app.services.credential_guides import DEFAULT_CREDENTIAL_GUIDES
 from app.services.collectors.top500 import TOP500Collector
-from app.services.collectors.registry import collector_registry
-from app.services.datasource_connectivity import SUPPORTED_CREDENTIAL_PROVIDERS
-from app.models.task import CollectionTask
+from app.services.collectors.base import BaseCollector, HTTPCollector
 
 
 class TestBaseCollector:
@@ -22,31 +19,6 @@ class TestBaseCollector:
         assert collector.module == "L1"
         assert collector.frequency_hours == 4
 
-    @pytest.mark.asyncio
-    async def test_update_phase_progress_tracks_phase_fields(self, mock_db_session):
-        """Test phase-level progress updates independently from record totals"""
-        collector = TOP500Collector()
-        task = CollectionTask(datasource_id=1, status="running", phase="fetching")
-        collector._current_task = task
-        collector._db_session = mock_db_session
-
-        with patch.object(collector, "_publish_task_update", new=AsyncMock()) as publish:
-            await collector.update_phase_progress(
-                current=512,
-                total=1024,
-                unit="bytes",
-                message="Downloading dataset",
-                commit=True,
-            )
-
-        assert task.phase_progress == 50.0
-        assert task.phase_current == 512
-        assert task.phase_total == 1024
-        assert task.phase_unit == "bytes"
-        assert task.phase_message == "Downloading dataset"
-        mock_db_session.commit.assert_awaited_once()
-        publish.assert_awaited_once()
-
 
 class TestTOP500Collector:
     """Tests for TOP500Collector"""
@@ -147,30 +119,3 @@ class TestHTTPCollector:
         assert hasattr(collector, "parse_response")
         assert callable(collector.fetch)
         assert callable(collector.parse_response)
-
-
-def test_aisstream_collector_is_registered():
-    collector = collector_registry.get("aisstream_vessels")
-
-    assert collector is not None
-    assert collector.data_type == "vessel_ais"
-
-
-def test_supported_credential_collectors_have_guides_and_connectivity_provider():
-    missing: list[str] = []
-    for source, info in DEFAULT_DATASOURCES.items():
-        if not info.get("requires_credentials"):
-            continue
-        if info.get("credential_status") != "supported":
-            continue
-
-        provider = info.get("credential_provider")
-        if not provider:
-            missing.append(f"{source}: missing credential_provider")
-            continue
-        if provider not in DEFAULT_CREDENTIAL_GUIDES:
-            missing.append(f"{source}: missing credential guide for {provider}")
-        if provider not in SUPPORTED_CREDENTIAL_PROVIDERS:
-            missing.append(f"{source}: missing connectivity provider for {provider}")
-
-    assert missing == []

backend/tests/test_datasource_mapping.py

@@ -1,199 +0,0 @@
-from types import SimpleNamespace
-
-import pytest
-from httpx import ASGITransport, AsyncClient
-
-from app.api.v1.datasource_config import get_ai_provider_client
-from app.core.security import get_current_user
-from app.core.target_schema_registry import get_target_schema, list_target_schemas
-from app.main import app
-from app.models.user import User
-from app.services.datasource_mapping import execute_mapping, persist_mapped_records, redact_for_llm
-
-
-SAMPLE_AIS = {
-    "data": [
-        {
-            "mmsi": "257123000",
-            "latitude": "59.91",
-            "longitude": "10.75",
-            "speedOverGround": "12.4",
-            "timestamp": "2026-04-28T00:00:00Z",
-            "api_token": "secret-value",
-        }
-    ]
-}
-
-
-def test_registry_exposes_v1_target_schemas():
-    keys = {schema["key"] for schema in list_target_schemas()}
-
-    assert {"vessel_ais", "geo_points", "generic_records"}.issubset(keys)
-    assert get_target_schema("vessel_ais").destination == "vessel_position"
-
-
-def test_mapping_engine_maps_and_validates_vessel_ais():
-    mapping = {
-        "source": {"items_path": "$.data[*]"},
-        "fields": {
-            "mmsi": {"path": "$.mmsi", "type": "integer"},
-            "lat": {"path": "$.latitude", "type": "float"},
-            "lon": {"path": "$.longitude", "type": "float"},
-            "sog": {"path": "$.speedOverGround", "type": "float"},
-            "received_at": {"path": "$.timestamp", "type": "datetime"},
-        },
-    }
-
-    result = execute_mapping(SAMPLE_AIS, mapping, "vessel_ais")
-
-    assert result["mapped_count"] == 1
-    assert result["failed_count"] == 0
-    assert result["records"][0]["mmsi"] == 257123000
-    assert result["records"][0]["lat"] == 59.91
-
-
-def test_mapping_engine_reports_schema_errors():
-    mapping = {
-        "source": {"items_path": "$.data[*]"},
-        "fields": {
-            "mmsi": {"path": "$.mmsi", "type": "integer"},
-            "lat": {"path": "$.missing_lat", "type": "float"},
-            "lon": {"path": "$.longitude", "type": "float"},
-        },
-    }
-
-    result = execute_mapping(SAMPLE_AIS, mapping, "vessel_ais")
-
-    assert result["mapped_count"] == 0
-    assert result["failed_count"] == 1
-    assert any("lat" in error for error in result["errors"][0]["errors"])
-
-
-def test_redact_for_llm_masks_secret_like_fields():
-    redacted = redact_for_llm(SAMPLE_AIS)
-
-    assert redacted["data"][0]["api_token"] == "[REDACTED]"
-
-
-@pytest.mark.asyncio
-async def test_persist_mapped_records_writes_generic_records():
-    class FakeDB:
-        def __init__(self):
-            self.added = []
-            self.committed = False
-
-        def add(self, value):
-            self.added.append(value)
-
-        async def commit(self):
-            self.committed = True
-
-    db = FakeDB()
-
-    count = await persist_mapped_records(
-        db,
-        datasource_name="custom_weather",
-        datasource_config_id=42,
-        target_schema="generic_records",
-        records=[{"source_id": "row-1", "data": {"temp": 25}}],
-        mapping_version=3,
-    )
-
-    assert count == 1
-    assert db.committed is True
-    assert db.added[0].source == "custom_weather"
-    assert db.added[0].data_type == "generic_records"
-    assert db.added[0].extra_data["mapping_version"] == 3
-
-
-@pytest.mark.asyncio
-async def test_mapping_preview_api_uses_deterministic_engine():
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="testuser",
-            email="test@example.com",
-            password_hash="hashed",
-            role="admin",
-            is_active=True,
-        )
-
-    app.dependency_overrides = {get_current_user: override_get_current_user}
-    transport = ASGITransport(app=app)
-    try:
-        async with AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.post(
-                "/api/v1/datasources/mappings/preview",
-                json={
-                    "sample_payload": SAMPLE_AIS,
-                    "target_schema": "vessel_ais",
-                    "mapping_json": {
-                        "source": {"items_path": "$.data[*]"},
-                        "fields": {
-                            "mmsi": {"path": "$.mmsi", "type": "integer"},
-                            "lat": {"path": "$.latitude", "type": "float"},
-                            "lon": {"path": "$.longitude", "type": "float"},
-                        },
-                    },
-                },
-            )
-    finally:
-        app.dependency_overrides.clear()
-
-    assert response.status_code == 200
-    payload = response.json()
-    assert payload["success"] is True
-    assert payload["preview"]["records"][0]["mmsi"] == 257123000
-
-
-@pytest.mark.asyncio
-async def test_mapping_propose_api_redacts_sample_before_ai():
-    seen_context = {}
-
-    class FakeAIClient:
-        async def analyze(self, request, request_id=None):
-            seen_context.update(request.context)
-            return SimpleNamespace(
-                content=(
-                    '{"source":{"items_path":"$.data[*]"},"fields":{'
-                    '"mmsi":{"path":"$.mmsi","type":"integer"},'
-                    '"lat":{"path":"$.latitude","type":"float"},'
-                    '"lon":{"path":"$.longitude","type":"float"}}}'
-                )
-            )
-
-    def override_get_current_user():
-        return User(
-            id=1,
-            username="testuser",
-            email="test@example.com",
-            password_hash="hashed",
-            role="admin",
-            is_active=True,
-        )
-
-    def override_ai_client():
-        return FakeAIClient()
-
-    app.dependency_overrides = {
-        get_current_user: override_get_current_user,
-        get_ai_provider_client: override_ai_client,
-    }
-    transport = ASGITransport(app=app)
-    try:
-        async with AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.post(
-                "/api/v1/datasources/mappings/propose",
-                json={
-                    "sample_payload": SAMPLE_AIS,
-                    "target_schema": "vessel_ais",
-                    "use_ai": True,
-                },
-            )
-    finally:
-        app.dependency_overrides.clear()
-
-    assert response.status_code == 200
-    payload = response.json()
-    assert payload["mapping_json"]["meta"]["generated_by"] == "ai_provider"
-    assert seen_context["sample_payload"]["data"][0]["api_token"] == "[REDACTED]"

backend/tests/test_earth_news.py

@@ -1,49 +0,0 @@
-from datetime import UTC, datetime
-
-from app.services.earth_news import ParsedNewsItem, _serialize_item
-
-
-def test_serialize_item_includes_region_anchor_for_cruise():
-    item = ParsedNewsItem(
-        id="google-apac:test",
-        title="Example APAC story",
-        summary="Example summary",
-        url="https://example.com/story",
-        source="Example Source",
-        feed_name="Global Monitor / APAC",
-        feed_region="asia-pacific",
-        homepage_url="https://example.com",
-        published_at=datetime(2026, 4, 23, 2, 30, tzinfo=UTC),
-    )
-
-    payload = _serialize_item(item, active_region="asia-pacific")
-
-    assert payload["latitude"] == 1.3521
-    assert payload["longitude"] == 103.8198
-    assert payload["location_label"] == "亚太"
-    assert payload["location_inferred"] is True
-    assert payload["is_focus_match"] is True
-    assert payload["published_at"] == "2026-04-23T02:30:00Z"
-
-
-def test_serialize_item_falls_back_to_global_anchor():
-    item = ParsedNewsItem(
-        id="custom:test",
-        title="Fallback story",
-        summary="Fallback summary",
-        url="https://example.com/fallback",
-        source="Fallback Source",
-        feed_name="Fallback Feed",
-        feed_region="unknown-region",
-        homepage_url="https://example.com",
-        published_at=None,
-    )
-
-    payload = _serialize_item(item, active_region="americas")
-
-    assert payload["latitude"] == 20.0
-    assert payload["longitude"] == 0.0
-    assert payload["location_label"] == "全球"
-    assert payload["location_inferred"] is True
-    assert payload["is_focus_match"] is False
-    assert payload["published_at"] is None

backend/tests/test_logging.py

@@ -1,78 +0,0 @@
-from __future__ import annotations
-
-import logging
-
-from io import StringIO
-
-from app.core.logging import PlanetContextFilter, PlanetFormatter, get_logger
-from app.core.request_context import set_request_id
-
-
-def _capture_output(callback):
-    stream = StringIO()
-    handler = logging.StreamHandler(stream)
-    handler.setFormatter(PlanetFormatter(datefmt="%Y-%m-%d %H:%M:%S"))
-    handler.addFilter(PlanetContextFilter())
-
-    adapter = get_logger("tests.logging")
-    target_logger = adapter.logger
-    original_handlers = list(target_logger.handlers)
-    original_level = target_logger.level
-    original_propagate = target_logger.propagate
-
-    target_logger.handlers = [handler]
-    target_logger.setLevel(logging.INFO)
-    target_logger.propagate = False
-
-    try:
-        callback(adapter)
-    finally:
-        handler.flush()
-        target_logger.handlers = original_handlers
-        target_logger.setLevel(original_level)
-        target_logger.propagate = original_propagate
-
-    return stream.getvalue()
-
-
-def test_structured_logger_injects_request_id_and_event():
-    set_request_id("req-test-123")
-    try:
-        output = _capture_output(
-            lambda logger: logger.info_event(
-                "collector started",
-                event="collector.run.started",
-                context={"collector_name": "bgp_news"},
-            )
-        )
-    finally:
-        set_request_id(None)
-
-    assert "request_id=req-test-123" in output
-    assert "event=collector.run.started" in output
-    assert "service=backend" in output
-    assert '"collector_name": "bgp_news"' in output
-
-
-def test_structured_logger_redacts_sensitive_text_and_context():
-    set_request_id("req-test-redact")
-    try:
-        output = _capture_output(
-            lambda logger: logger.error_event(
-                "Authorization: Bearer super-secret-token",
-                event="auth.token.failed",
-                context={
-                    "token": "plain-secret",
-                    "nested": {"password": "hunter2"},
-                    "safe": "visible",
-                },
-            )
-        )
-    finally:
-        set_request_id(None)
-
-    assert "super-secret-token" not in output
-    assert "plain-secret" not in output
-    assert "hunter2" not in output
-    assert "[REDACTED]" in output
-    assert '"safe": "visible"' in output

backend/tests/test_models.py

@@ -121,25 +121,6 @@ class TestCollectionTaskModel:
         )
         assert task.records_processed == 100
 
-    def test_task_with_phase_progress(self):
-        """Test collection task phase-level progress fields"""
-        task = CollectionTask(
-            datasource_id=1,
-            status="running",
-            phase="fetching",
-            phase_progress=42.5,
-            phase_message="Downloading dataset",
-            phase_current=1024,
-            phase_total=4096,
-            phase_unit="bytes",
-        )
-        assert task.phase == "fetching"
-        assert task.phase_progress == 42.5
-        assert task.phase_message == "Downloading dataset"
-        assert task.phase_current == 1024
-        assert task.phase_total == 4096
-        assert task.phase_unit == "bytes"
-
     def test_task_error_message(self):
         """Test collection task with error message"""
         task = CollectionTask(

backend/tests/test_system_logs.py

@@ -1,217 +0,0 @@
-from __future__ import annotations
-
-import json
-
-from pathlib import Path
-
-from app.services import system_logs
-
-
-class FakeRedis:
-    def __init__(self) -> None:
-        self.store: dict[str, list[str]] = {}
-
-    def rpush(self, key: str, value: str) -> None:
-        self.store.setdefault(key, []).append(value)
-
-    def ltrim(self, key: str, start: int, end: int) -> None:
-        items = self.store.get(key, [])
-        normalized_end = None if end == -1 else end + 1
-        self.store[key] = items[start:normalized_end]
-
-    def expire(self, key: str, seconds: int) -> None:
-        return None
-
-    def lrange(self, key: str, start: int, end: int) -> list[str]:
-        items = self.store.get(key, [])
-        normalized_end = None if end == -1 else end + 1
-        return items[start:normalized_end]
-
-    def llen(self, key: str) -> int:
-        return len(self.store.get(key, []))
-
-
-def test_read_log_snapshot_uses_structured_buffer_timestamp_level_and_search(monkeypatch):
-    fake_redis = FakeRedis()
-    monkeypatch.setattr(system_logs, "redis_client", fake_redis)
-    monkeypatch.setattr(
-        system_logs,
-        "LOG_SOURCES",
-        {
-            "earth-client": system_logs.LogSource(
-                source_id="earth-client",
-                name="Earth 浏览器端",
-                kind="buffer",
-                location="redis://planet:system_logs:earth-client",
-                description="Earth 浏览器端上报日志",
-                category="client",
-                buffer_key=system_logs.get_buffer_log_key("earth-client"),
-            )
-        },
-    )
-
-    fake_redis.rpush(
-        system_logs.get_buffer_log_key("earth-client"),
-        json.dumps(
-            {
-                "timestamp": "2026-04-22T10:15:30Z",
-                "level": "warning",
-                "message": "news feed degraded",
-                "context": {"module": "news", "detail": "timeout"},
-            },
-            ensure_ascii=False,
-        ),
-    )
-    fake_redis.rpush(
-        system_logs.get_buffer_log_key("earth-client"),
-        json.dumps(
-            {
-                "timestamp": "2026-04-23T06:01:00Z",
-                "level": "error",
-                "message": "landing points failed",
-                "context": {"module": "layer-startup", "detail": "http 500"},
-            },
-            ensure_ascii=False,
-        ),
-    )
-
-    snapshot = system_logs.read_log_snapshot(
-        "earth-client",
-        50,
-        levels="error,warning",
-        start_date="2026-04-23",
-        end_date="2026-04-23",
-        search="landing",
-    )
-
-    assert snapshot is not None
-    assert snapshot["selected_levels"] == ["error", "warning"]
-    assert snapshot["search_query"] == "landing"
-    assert snapshot["line_count"] == 1
-    assert snapshot["lines"][0].startswith("2026-04-23 06:01:00 ERROR landing points failed")
-    assert snapshot["daily_markers"] == [
-        {"date_token": "2026-04-23", "total": 1, "dominant_level": "error"}
-    ]
-
-
-def test_read_log_snapshot_parses_file_timestamp_and_builds_markers(tmp_path: Path, monkeypatch):
-    log_path = tmp_path / "backend.log"
-    log_path.write_text(
-        "\n".join(
-            [
-                "2026-04-22 08:00:00 INFO service booted",
-                "2026-04-23 09:15:00 WARNING disk pressure detected",
-                "2026-04-23 09:16:00 ERROR sync failed",
-                "2026-04-24 10:00:00 DEBUG collector trace",
-            ]
-        ),
-        encoding="utf-8",
-    )
-
-    monkeypatch.setattr(
-        system_logs,
-        "LOG_SOURCES",
-        {
-            "backend": system_logs.LogSource(
-                source_id="backend",
-                name="后端服务",
-                kind="file",
-                location=str(log_path),
-                description="测试文件日志",
-                category="service",
-            )
-        },
-    )
-
-    snapshot = system_logs.read_log_snapshot(
-        "backend",
-        50,
-        levels="warning,error",
-        search="failed",
-    )
-
-    assert snapshot is not None
-    assert snapshot["line_count"] == 1
-    assert snapshot["lines"] == ["2026-04-23 09:16:00 ERROR sync failed"]
-    assert snapshot["daily_markers"] == [
-        {"date_token": "2026-04-23", "total": 1, "dominant_level": "error"}
-    ]
-    assert snapshot["status"] == "ok"
-
-
-def test_append_buffer_log_persists_normalized_level(monkeypatch):
-    fake_redis = FakeRedis()
-    monkeypatch.setattr(system_logs, "redis_client", fake_redis)
-
-    system_logs.append_buffer_log(
-        "earth-client",
-        level="warn",
-        message="feed delayed",
-        context={"module": "news"},
-    )
-
-    stored_items = fake_redis.lrange(system_logs.get_buffer_log_key("earth-client"), 0, -1)
-    payload = json.loads(stored_items[0])
-    assert payload["level"] == "warning"
-    assert payload["message"] == "feed delayed"
-
-
-def test_infer_log_level_prefers_leading_prefix_over_query_string():
-    line = 'INFO: 127.0.0.1 - "GET /api/v1/system/logs/backend?limit=200&level=error&levels=error HTTP/1.1" 200 OK'
-
-    entry = system_logs.parse_text_log_entry(line)
-
-    assert entry.level == "info"
-
-
-def test_parse_text_log_entry_does_not_promote_exception_context_to_error():
-    line = "websockets.exceptions.ConnectionClosedError: sent 1011 (internal error) keepalive ping timeout"
-
-    entry = system_logs.parse_text_log_entry(line)
-
-    assert entry.level is None
-
-
-def test_parse_text_log_entry_still_detects_explicit_error_prefix():
-    line = "ERROR: [Errno 98] Address already in use"
-
-    entry = system_logs.parse_text_log_entry(line)
-
-    assert entry.level == "error"
-
-
-def test_read_log_snapshot_strips_nul_bytes_from_file_lines(tmp_path: Path, monkeypatch):
-    log_path = tmp_path / "backend.log"
-    log_path.write_bytes(
-        (
-            b"INFO: service booted\n"
-            b"ERROR: bind failed\n"
-            + b"\x00" * 32
-            + b"2026-04-23 23:41:32 INFO service=backend message=request served\n"
-        )
-    )
-
-    monkeypatch.setattr(
-        system_logs,
-        "LOG_SOURCES",
-        {
-            "backend": system_logs.LogSource(
-                source_id="backend",
-                name="后端服务",
-                kind="file",
-                location=str(log_path),
-                description="测试文件日志",
-                category="service",
-            )
-        },
-    )
-
-    snapshot = system_logs.read_log_snapshot("backend", 50)
-
-    assert snapshot is not None
-    assert snapshot["line_count"] == 3
-    assert snapshot["lines"] == [
-        "INFO: service booted",
-        "ERROR: bind failed",
-        "2026-04-23 23:41:32 INFO service=backend message=request served",
-    ]

backend/tests/test_vessels.py

@@ -1,479 +0,0 @@
-from datetime import datetime, timedelta, timezone
-from unittest.mock import AsyncMock
-
-import pytest
-from httpx import ASGITransport, AsyncClient
-
-from app.api.v1.visualization import convert_vessels_to_geojson
-from app.db.session import get_db
-from app.main import app
-from app.models.vessel import AISRawObservation, VesselPosition, VesselStatic
-from app.services import barentswatch
-from app.services.collectors.aisstream import AISStreamCollector
-from app.services.collectors.vessel_ais import VesselAISCollector
-from app.services.vessel_ais_aggregation import (
-    aggregate_vessel_observations,
-    build_field_conflict_candidates,
-    build_observation_hash,
-    record_vessel_ais_observation,
-)
-
-
-def test_vessel_collector_transforms_barentswatch_like_records():
-    collector = VesselAISCollector()
-    records = collector.transform(
-        [
-            {
-                "mmsi": "257123000",
-                "lat": "59.91",
-                "lon": "10.73",
-                "sog": 12.4,
-                "cog": 214,
-                "nav_status": 0,
-                "shipType": 70,
-                "name": "OSLO TRADER",
-            },
-            {"mmsi": "bad", "lat": 120, "lon": 10},
-        ]
-    )
-
-    assert len(records) == 1
-    assert records[0]["mmsi"] == 257123000
-    assert records[0]["vessel_type_name"] == "Cargo"
-    assert records[0]["lat"] == pytest.approx(59.91)
-
-
-def test_vessel_observation_hash_is_stable_for_same_payload():
-    observed_at = datetime(2026, 4, 30, 12, 0, tzinfo=timezone.utc)
-    payload = {
-        "mmsi": 257123000,
-        "lat": 59.91,
-        "lon": 10.73,
-        "received_at": observed_at,
-    }
-
-    first = build_observation_hash(
-        source="barentswatch_vessels",
-        entity_key="257123000",
-        message_type="PositionReport",
-        observed_at=observed_at,
-        normalized_payload=payload,
-    )
-    second = build_observation_hash(
-        source="barentswatch_vessels",
-        entity_key="257123000",
-        message_type="PositionReport",
-        observed_at=observed_at,
-        normalized_payload=dict(reversed(payload.items())),
-    )
-
-    assert first == second
-    assert len(first) == 64
-
-
-@pytest.mark.asyncio
-async def test_record_vessel_ais_observation_skips_existing_hash():
-    observed_at = datetime(2026, 4, 30, 12, 0, tzinfo=timezone.utc)
-
-    class _Result:
-        def scalar_one_or_none(self):
-            return 123
-
-    class _Session:
-        def __init__(self):
-            self.added = []
-
-        async def execute(self, _stmt):
-            return _Result()
-
-        def add(self, item):
-            self.added.append(item)
-
-    db = _Session()
-    observation = await record_vessel_ais_observation(
-        db,
-        source="barentswatch_vessels",
-        normalized_payload={
-            "mmsi": 257123000,
-            "lat": 59.91,
-            "lon": 10.73,
-            "received_at": observed_at,
-        },
-        delivery_mode="polling",
-        transport="http",
-        observed_at=observed_at.isoformat(),
-    )
-
-    assert observation is None
-    assert db.added == []
-
-
-def test_build_field_conflict_candidates_from_raw_observations():
-    observations = [
-        AISRawObservation(
-            source="barentswatch_vessels",
-            normalized_payload={"name": "OSLO TRADER", "flag": "NO"},
-        ),
-        AISRawObservation(
-            source="aisstream_vessels",
-            normalized_payload={"name": "OSLO TRADER II", "flag": "NO"},
-        ),
-    ]
-
-    conflicts = build_field_conflict_candidates(observations)
-
-    assert conflicts == [
-        {
-            "field": "name",
-            "candidates": {
-                "aisstream_vessels": "OSLO TRADER II",
-                "barentswatch_vessels": "OSLO TRADER",
-            },
-            "status": "candidate",
-        }
-    ]
-
-
-@pytest.mark.asyncio
-async def test_aggregate_vessel_observations_prefers_realtime_and_records_conflict():
-    observed_at = datetime.now(timezone.utc) - timedelta(minutes=5)
-
-    class _Result:
-        def scalar_one_or_none(self):
-            return None
-
-    class _Session:
-        def __init__(self):
-            self.added = []
-
-        async def execute(self, _stmt):
-            return _Result()
-
-        def add(self, item):
-            self.added.append(item)
-
-    db = _Session()
-    observations = [
-        AISRawObservation(
-            id=1,
-            source="barentswatch_vessels",
-            entity_key="257123000",
-            delivery_mode="polling",
-            transport="http",
-            observed_at=observed_at,
-            collected_at=observed_at,
-            normalized_payload={
-                "mmsi": 257123000,
-                "name": "OSLO TRADER",
-                "lat": 59.91,
-                "lon": 10.73,
-            },
-        ),
-        AISRawObservation(
-            id=2,
-            source="aisstream_vessels",
-            entity_key="257123000",
-            delivery_mode="realtime_stream",
-            transport="websocket",
-            observed_at=observed_at + timedelta(seconds=10),
-            collected_at=observed_at + timedelta(seconds=10),
-            normalized_payload={
-                "mmsi": 257123000,
-                "vessel_type": 79,
-                "lat": 59.92,
-                "lon": 10.74,
-            },
-            raw_payload={"MetaData": {"ShipName": "OSLO TRADER II         "}},
-        ),
-    ]
-
-    vessels = await aggregate_vessel_observations(db, observations)
-
-    assert vessels[0]["lat"] == pytest.approx(59.92)
-    assert vessels[0]["field_sources"]["lat"] == "aisstream_vessels"
-    assert vessels[0]["name"] == "OSLO TRADER II"
-    assert vessels[0]["vessel_type_name"] == "Cargo"
-    assert vessels[0]["source_summary"]["aisstream_vessels"]["observation_count"] == 1
-    assert vessels[0]["source_summary"]["barentswatch_vessels"]["delivery_mode"] == "polling"
-    assert vessels[0]["conflict_count"] == 0
-    assert db.added == []
-
-
-@pytest.mark.asyncio
-async def test_vessel_collector_writes_raw_observations_without_changing_position_save(monkeypatch):
-    collector = VesselAISCollector()
-    collector.update_progress = AsyncMock()
-    record_observation = AsyncMock()
-    update_health = AsyncMock()
-    monkeypatch.setattr(
-        "app.services.collectors.vessel_ais.record_vessel_ais_observation",
-        record_observation,
-    )
-    monkeypatch.setattr(
-        "app.services.collectors.vessel_ais.update_ais_source_health",
-        update_health,
-    )
-
-    class _Session:
-        def __init__(self):
-            self.added = []
-            self.committed = False
-
-        async def get(self, *_args):
-            return None
-
-        def add(self, item):
-            self.added.append(item)
-
-        async def execute(self, _stmt):
-            return None
-
-        async def commit(self):
-            self.committed = True
-
-    db = _Session()
-    observed_at = datetime(2026, 4, 30, 12, 0, tzinfo=timezone.utc)
-
-    saved = await collector._save_data(
-        db,
-        [
-            {
-                "mmsi": 257123000,
-                "name": "OSLO TRADER",
-                "lat": 59.91,
-                "lon": 10.73,
-                "received_at": observed_at,
-            }
-        ],
-    )
-
-    assert saved == 1
-    assert db.committed is True
-    assert any(isinstance(item, VesselStatic) for item in db.added)
-    assert any(isinstance(item, VesselPosition) for item in db.added)
-    record_observation.assert_awaited_once()
-    assert record_observation.await_args.kwargs["source"] == "barentswatch_vessels"
-    assert record_observation.await_args.kwargs["normalized_payload"]["mmsi"] == 257123000
-    update_health.assert_awaited_once()
-
-
-def test_aisstream_collector_normalizes_position_report():
-    collector = AISStreamCollector()
-
-    records = collector.transform(
-        [
-            {
-                "MessageType": "PositionReport",
-                "MetaData": {
-                    "MMSI": 257123000,
-                    "ShipName": "OSLO TRADER         ",
-                    "time_utc": "2026-04-30T12:00:00Z",
-                },
-                "Message": {
-                    "PositionReport": {
-                        "Latitude": 59.91,
-                        "Longitude": 10.73,
-                        "Sog": 12.4,
-                        "Cog": 214,
-                        "TrueHeading": 215,
-                        "NavigationalStatus": 0,
-                    }
-                },
-            }
-        ]
-    )
-
-    assert len(records) == 1
-    assert records[0]["mmsi"] == 257123000
-    assert records[0]["lat"] == pytest.approx(59.91)
-    assert records[0]["name"] == "OSLO TRADER"
-    assert records[0]["_message_type"] == "PositionReport"
-
-
-def test_aisstream_collector_maps_ship_static_type_name():
-    collector = AISStreamCollector()
-
-    records = collector.transform(
-        [
-            {
-                "MessageType": "ShipStaticData",
-                "MetaData": {
-                    "MMSI": 257123000,
-                    "time_utc": "2026-04-30T12:00:00Z",
-                },
-                "Message": {
-                    "ShipStaticData": {
-                        "Name": "OSLO TRADER",
-                        "Type": 79,
-                        "CallSign": "LAAB",
-                    }
-                },
-            }
-        ]
-    )
-
-    assert len(records) == 1
-    assert records[0]["vessel_type"] == 79
-    assert records[0]["vessel_type_name"] == "Cargo"
-
-
-@pytest.mark.asyncio
-async def test_aisstream_collector_writes_only_raw_observations(monkeypatch):
-    collector = AISStreamCollector()
-    collector.update_progress = AsyncMock()
-    record_observation = AsyncMock(return_value=object())
-    update_health = AsyncMock()
-    monkeypatch.setattr(
-        "app.services.collectors.aisstream.record_vessel_ais_observation",
-        record_observation,
-    )
-    monkeypatch.setattr(
-        "app.services.collectors.aisstream.update_ais_source_health",
-        update_health,
-    )
-
-    class _Session:
-        def __init__(self):
-            self.added = []
-            self.committed = False
-
-        def add(self, item):
-            self.added.append(item)
-
-        async def commit(self):
-            self.committed = True
-
-    db = _Session()
-    saved = await collector._save_data(
-        db,
-        [
-            {
-                "mmsi": 257123000,
-                "lat": 59.91,
-                "lon": 10.73,
-                "received_at": datetime(2026, 4, 30, 12, 0, tzinfo=timezone.utc),
-                "_message_type": "PositionReport",
-            }
-        ],
-    )
-
-    assert saved == 1
-    assert db.added == []
-    assert db.committed is True
-    record_observation.assert_awaited_once()
-    assert record_observation.await_args.kwargs["source"] == "aisstream_vessels"
-    update_health.assert_awaited_once()
-
-
-def test_barentswatch_reads_credentials_from_zshrc(tmp_path):
-    zshrc = tmp_path / ".zshrc"
-    zshrc.write_text(
-        "\n".join(
-            [
-                "export BARENTSWATCH_CLIENT_ID='client-from-zshrc'",
-                'export BARENTSWATCH_CLIENT_SECRET="secret-from-zshrc" # local dev credential',
-            ]
-        ),
-        encoding="utf-8",
-    )
-
-    values = barentswatch._read_zshrc_env(zshrc)
-
-    assert values["BARENTSWATCH_CLIENT_ID"] == "client-from-zshrc"
-    assert values["BARENTSWATCH_CLIENT_SECRET"] == "secret-from-zshrc"
-
-
-@pytest.mark.asyncio
-async def test_barentswatch_resolves_config_from_zshrc(tmp_path, monkeypatch):
-    zshrc = tmp_path / ".zshrc"
-    zshrc.write_text(
-        "\n".join(
-            [
-                "export BARENTSWATCH_CLIENT_ID=client-from-zshrc",
-                "export BARENTSWATCH_CLIENT_SECRET=secret-from-zshrc",
-            ]
-        ),
-        encoding="utf-8",
-    )
-    monkeypatch.delenv("BARENTSWATCH_CLIENT_ID", raising=False)
-    monkeypatch.delenv("BARENTSWATCH_CLIENT_SECRET", raising=False)
-    monkeypatch.delenv("BARRENTSWATCH_CLIENT_ID", raising=False)
-    monkeypatch.delenv("BARRENTSWATCH_CLIENT_SECRET", raising=False)
-    monkeypatch.setattr(barentswatch.Path, "home", lambda: tmp_path)
-
-    config = await barentswatch.resolve_barentswatch_config(None)
-
-    assert config.client_id == "client-from-zshrc"
-    assert config.client_secret == "secret-from-zshrc"
-    assert config.credential_source == "~/.zshrc"
-
-
-def test_convert_vessels_to_geojson():
-    position = VesselPosition(
-        mmsi=257123000,
-        lat=59.91,
-        lon=10.73,
-        sog=12.4,
-        cog=214,
-        heading=215,
-        nav_status=0,
-        received_at=datetime(2026, 4, 28, 1, 0, tzinfo=timezone.utc),
-    )
-    static = VesselStatic(
-        mmsi=257123000,
-        name="OSLO TRADER",
-        vessel_type=70,
-        vessel_type_name="Cargo",
-        flag="NO",
-        length=185,
-    )
-
-    payload = convert_vessels_to_geojson([(position, static)])
-
-    assert payload["type"] == "FeatureCollection"
-    assert payload["features"][0]["geometry"]["coordinates"] == [10.73, 59.91]
-    assert payload["features"][0]["properties"]["mmsi"] == 257123000
-    assert payload["features"][0]["properties"]["vessel_type_name"] == "Cargo"
-
-
-@pytest.mark.asyncio
-async def test_vessels_geojson_endpoint_filters_type_and_bbox():
-    now = datetime(2026, 4, 28, 1, 0, tzinfo=timezone.utc)
-    rows = [
-        (
-            VesselPosition(mmsi=1, lat=59.9, lon=10.7, received_at=now),
-            VesselStatic(mmsi=1, name="Cargo Ship", vessel_type=70, vessel_type_name="Cargo"),
-        ),
-        (
-            VesselPosition(mmsi=2, lat=60.3, lon=5.3, received_at=now - timedelta(minutes=1)),
-            VesselStatic(mmsi=2, name="Passenger Ship", vessel_type=60, vessel_type_name="Passenger"),
-        ),
-    ]
-
-    class _Result:
-        def all(self):
-            return rows
-
-    class _FakeSession:
-        async def execute(self, _query):
-            return _Result()
-
-    async def override_get_db():
-        yield _FakeSession()
-
-    app.dependency_overrides[get_db] = override_get_db
-    transport = ASGITransport(app=app)
-    try:
-        async with AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.get(
-                "/api/v1/visualization/geo/vessels",
-                params={"bbox": "0,50,20,70", "type": "cargo", "limit": 0},
-            )
-
-        assert response.status_code == 200
-        data = response.json()
-        assert data["count"] == 1
-        assert data["features"][0]["properties"]["name"] == "Cargo Ship"
-        assert data["stats"]["by_type"]["Cargo"] == 1
-    finally:
-        app.dependency_overrides.clear()

backend/tests/test_visualization_compute_centers.py

@@ -1,347 +0,0 @@
-from datetime import datetime, timezone
-
-import pytest
-from httpx import ASGITransport, AsyncClient
-
-from app.api.v1.visualization import convert_compute_centers_to_geojson
-from app.db.session import get_db
-from app.main import app
-from app.models.collected_data import CollectedData
-
-
-def _build_record(
-    *,
-    record_id: int,
-    source: str,
-    data_type: str,
-    name: str,
-    country: str,
-    city: str,
-    latitude: float,
-    longitude: float,
-    metadata: dict,
-):
-    return CollectedData(
-        id=record_id,
-        source=source,
-        data_type=data_type,
-        source_id=f"{source}-{record_id}",
-        name=name,
-        extra_data={
-            "country": country,
-            "city": city,
-            "latitude": latitude,
-            "longitude": longitude,
-            **metadata,
-        },
-        collected_at=datetime(2026, 4, 22, tzinfo=timezone.utc),
-        reference_date=datetime(2026, 4, 21, tzinfo=timezone.utc),
-        is_current=True,
-    )
-
-
-def test_convert_compute_centers_to_geojson_unifies_sources():
-    top500_record = _build_record(
-        record_id=1,
-        source="top500",
-        data_type="supercomputer",
-        name="Frontier",
-        country="United States",
-        city="Oak Ridge",
-        latitude=35.93,
-        longitude=-84.31,
-        metadata={
-            "rank": 1,
-            "manufacturer": "HPE",
-            "organization": "ORNL",
-            "rmax": 1102000.0,
-            "cores": 8730112,
-            "power": 21510.0,
-        },
-    )
-    gpu_record = _build_record(
-        record_id=2,
-        source="epoch_ai_gpu",
-        data_type="gpu_cluster",
-        name="Colossus",
-        country="United States",
-        city="Memphis",
-        latitude=35.15,
-        longitude=-90.05,
-        metadata={
-            "organization": "xAI",
-            "gpu_type": "H100",
-            "gpu_count": 100000,
-            "value": "20000",
-            "unit": "TFlop/s",
-        },
-    )
-
-    payload = convert_compute_centers_to_geojson([top500_record, gpu_record])
-
-    assert payload["type"] == "FeatureCollection"
-    assert len(payload["features"]) == 2
-
-    supercomputer_feature = payload["features"][0]
-    assert supercomputer_feature["properties"]["site_type"] == "supercomputer"
-    assert supercomputer_feature["properties"]["capacity_unit"] == "GFlops"
-    assert supercomputer_feature["properties"]["capacity_band"] == "exascale"
-    assert supercomputer_feature["properties"]["operator"] == "ORNL"
-    assert supercomputer_feature["properties"]["location_precision"] == "precise"
-    assert supercomputer_feature["properties"]["is_estimated"] is False
-
-    gpu_feature = payload["features"][1]
-    assert gpu_feature["properties"]["site_type"] == "gpu_cluster"
-    assert gpu_feature["properties"]["vendor"] == "H100"
-    assert gpu_feature["properties"]["gpu_count"] == 100000
-    assert gpu_feature["properties"]["capacity_band"] == "large"
-    assert gpu_feature["properties"]["location_precision"] == "precise"
-
-
-def test_convert_compute_centers_to_geojson_uses_coordinate_hints():
-    hinted_record = _build_record(
-        record_id=3,
-        source="top500",
-        data_type="supercomputer",
-        name="Frontier",
-        country="United States",
-        city="",
-        latitude=0.0,
-        longitude=0.0,
-        metadata={
-            "organization": "Oak Ridge National Laboratory",
-            "rmax": 1102000.0,
-        },
-    )
-
-    payload = convert_compute_centers_to_geojson([hinted_record])
-
-    assert len(payload["features"]) == 1
-    coords = payload["features"][0]["geometry"]["coordinates"]
-    assert coords[0] == pytest.approx(-84.3107)
-    assert coords[1] == pytest.approx(35.9319)
-    assert payload["features"][0]["properties"]["is_estimated"] is True
-    assert payload["features"][0]["properties"]["location_precision"] == "estimated_site"
-
-
-def test_convert_compute_centers_to_geojson_falls_back_to_country_centroid():
-    centroid_record = _build_record(
-        record_id=4,
-        source="epoch_ai_gpu",
-        data_type="gpu_cluster",
-        name="Unknown Cluster",
-        country="United States",
-        city="",
-        latitude=0.0,
-        longitude=0.0,
-        metadata={
-            "organization": "Unknown Operator",
-            "value": "10000",
-            "unit": "TFlop/s",
-        },
-    )
-
-    payload = convert_compute_centers_to_geojson([centroid_record])
-
-    assert len(payload["features"]) == 1
-    props = payload["features"][0]["properties"]
-    coords = payload["features"][0]["geometry"]["coordinates"]
-    assert coords[0] == pytest.approx(-98.5795)
-    assert coords[1] == pytest.approx(39.8283)
-    assert props["is_estimated"] is True
-    assert props["location_precision"] == "estimated_country"
-    assert props["geography_mode"] == "country_centroid"
-
-
-@pytest.mark.asyncio
-async def test_compute_centers_geojson_endpoint_returns_stats():
-    records = [
-        _build_record(
-            record_id=1,
-            source="top500",
-            data_type="supercomputer",
-            name="Frontier",
-            country="United States",
-            city="Oak Ridge",
-            latitude=35.93,
-            longitude=-84.31,
-            metadata={"rank": 1, "rmax": 1102000.0},
-        ),
-        _build_record(
-            record_id=2,
-            source="epoch_ai_gpu",
-            data_type="gpu_cluster",
-            name="Colossus",
-            country="United States",
-            city="Memphis",
-            latitude=35.15,
-            longitude=-90.05,
-            metadata={"value": "20000", "unit": "TFlop/s"},
-        ),
-    ]
-
-    class _ScalarResult:
-        def __init__(self, rows):
-            self._rows = rows
-
-        def scalars(self):
-            class _Scalars:
-                def __init__(self, rows):
-                    self._rows = rows
-
-                def all(self):
-                    return self._rows
-
-            return _Scalars(self._rows)
-
-    class _FakeSession:
-        async def execute(self, _query):
-            return _ScalarResult(records)
-
-    async def override_get_db():
-        yield _FakeSession()
-
-    app.dependency_overrides[get_db] = override_get_db
-    transport = ASGITransport(app=app)
-    try:
-        async with AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.get("/api/v1/visualization/geo/compute-centers")
-
-        assert response.status_code == 200
-        data = response.json()
-        assert data["count"] == 2
-        assert data["stats"]["supercomputers"] == 1
-        assert data["stats"]["gpu_clusters"] == 1
-        assert data["features"][0]["properties"]["data_type"] == "compute_center"
-    finally:
-        app.dependency_overrides.clear()
-
-
-@pytest.mark.asyncio
-async def test_visualization_geo_summary_returns_counts(monkeypatch):
-    records = [
-        _build_record(
-            record_id=1,
-            source="arcgis_cables",
-            data_type="submarine_cable",
-            name="Test Cable",
-            country="",
-            city="",
-            latitude=0,
-            longitude=0,
-            metadata={
-                "route_coordinates": [[[0, 0], [1, 1]]],
-                "status": "active",
-            },
-        ),
-        _build_record(
-            record_id=2,
-            source="arcgis_landing_points",
-            data_type="landing_point",
-            name="Test Landing",
-            country="United States",
-            city="New York",
-            latitude=40.7,
-            longitude=-74.0,
-            metadata={"city_id": 10},
-        ),
-        _build_record(
-            record_id=3,
-            source="celestrak_tle",
-            data_type="satellite_tle",
-            name="TESTSAT",
-            country="",
-            city="",
-            latitude=0,
-            longitude=0,
-            metadata={
-                "norad_cat_id": 12345,
-                "tle_line1": "1 12345U 98067A   24001.00000000  .00000000  00000-0  00000-0 0  9991",
-                "tle_line2": "2 12345  51.6000 100.0000 0001000  10.0000  20.0000 15.50000000    01",
-            },
-        ),
-        _build_record(
-            record_id=4,
-            source="top500",
-            data_type="supercomputer",
-            name="Frontier",
-            country="United States",
-            city="Oak Ridge",
-            latitude=35.93,
-            longitude=-84.31,
-            metadata={"rank": 1, "rmax": 1102000.0},
-        ),
-        _build_record(
-            record_id=5,
-            source="epoch_ai_gpu",
-            data_type="gpu_cluster",
-            name="Colossus",
-            country="United States",
-            city="Memphis",
-            latitude=35.15,
-            longitude=-90.05,
-            metadata={"value": "20000", "unit": "TFlop/s"},
-        ),
-    ]
-
-    class _ScalarResult:
-        def __init__(self, rows=None, scalar_value=None):
-            self._rows = rows or []
-            self._scalar_value = scalar_value
-
-        def scalar(self):
-            return self._scalar_value
-
-        def scalars(self):
-            class _Scalars:
-                def __init__(self, rows):
-                    self._rows = rows
-
-                def all(self):
-                    return self._rows
-
-            return _Scalars(self._rows)
-
-    class _FakeSession:
-        async def execute(self, query):
-            query_text = str(query)
-            if "bgp_incidents" in query_text:
-                return _ScalarResult(scalar_value=2)
-            if "bgp_anomalies" in query_text:
-                return _ScalarResult(scalar_value=3)
-            return _ScalarResult(rows=records)
-
-    async def override_get_db():
-        yield _FakeSession()
-
-    async def _fake_build_bgp_collector_coverage(*_args, **_kwargs):
-        return [
-            {"collector": "rrc00"},
-            {"collector": "rrc01"},
-        ]
-
-    monkeypatch.setattr(
-        "app.api.v1.visualization.build_bgp_collector_coverage",
-        _fake_build_bgp_collector_coverage,
-    )
-
-    app.dependency_overrides[get_db] = override_get_db
-    transport = ASGITransport(app=app)
-    try:
-        async with AsyncClient(transport=transport, base_url="http://test") as client:
-            response = await client.get("/api/v1/visualization/geo/summary")
-
-        assert response.status_code == 200
-        stats = response.json()["stats"]
-        assert stats["cable_count"] == 1
-        assert stats["landing_point_count"] == 1
-        assert stats["satellite_count"] == 1
-        assert stats["compute_center_count"] == 2
-        assert stats["supercomputer_count"] == 1
-        assert stats["gpu_cluster_count"] == 1
-        assert stats["bgp_event_count"] == 2
-        assert stats["bgp_incident_count"] == 2
-        assert stats["bgp_anomaly_count"] == 3
-        assert stats["bgp_collector_count"] == 2
-    finally:
-        app.dependency_overrides.clear()

backend/tests/test_websocket_scene.py

@@ -0,0 +1,150 @@
+import pytest
+
+from app.core.websocket.manager import ConnectionManager
+from app.core.websocket.ue_scene import (
+    UeSceneStateStore,
+    _item_revision,
+    build_incremental_changes,
+)
+
+
+class _FakeSession:
+    pass
+
+
+class _FakeWebSocket:
+    def __init__(self):
+        self.accepted = False
+        self.messages = []
+
+    async def accept(self):
+        self.accepted = True
+
+    async def send_json(self, message):
+        self.messages.append(message)
+
+    async def close(self):
+        return None
+
+
+def _scene_item(item_id: str, title: str, *, lat: float = 0.0, lng: float = 0.0):
+    item = {
+        "id": item_id,
+        "entity_type": "gpu_cluster",
+        "geo": {"lat": lat, "lng": lng, "alt": 0.0},
+        "visual": {"style": "pulse_marker", "size": 1.0, "color": "#FF8C42"},
+        "metrics": {"name": title},
+        "labels": {"title": title, "subtitle": ""},
+        "status": {"health": "normal", "alert_level": "none"},
+    }
+    item["revision"] = _item_revision(item)
+    return item
+
+
+def _scene_state(state_hash: int, items_by_layer: dict[str, dict[str, dict]]):
+    base_layers = {
+        "satellites": {"revision": 0, "items": {}},
+        "supercomputers": {"revision": 0, "items": {}},
+        "gpu_clusters": {"revision": 0, "items": {}},
+        "submarine_cables": {"revision": 0, "items": {}},
+        "landing_points": {"revision": 0, "items": {}},
+        "alerts": {"revision": 0, "items": {}},
+    }
+    for layer_name, items in items_by_layer.items():
+        base_layers[layer_name]["items"] = items
+        base_layers[layer_name]["revision"] = len(items)
+    return {
+        "generated_at": "2026-04-17T00:00:00Z",
+        "state_hash": state_hash,
+        "total_records": sum(len(items) for items in items_by_layer.values()),
+        "layers": base_layers,
+    }
+
+
+def test_build_incremental_changes_detects_add_update_remove():
+    previous_state = _scene_state(
+        1,
+        {
+            "gpu_clusters": {
+                "gpu:a": _scene_item("gpu:a", "A"),
+                "gpu:b": _scene_item("gpu:b", "B"),
+            }
+        },
+    )
+    current_state = _scene_state(
+        2,
+        {
+            "gpu_clusters": {
+                "gpu:b": _scene_item("gpu:b", "B Updated"),
+                "gpu:c": _scene_item("gpu:c", "C"),
+            }
+        },
+    )
+
+    changes = build_incremental_changes(previous_state, current_state)
+
+    assert changes["gpu_clusters"]["added"][0]["id"] == "gpu:c"
+    assert changes["gpu_clusters"]["updated"][0]["id"] == "gpu:b"
+    assert changes["gpu_clusters"]["removed"] == ["gpu:a"]
+
+
+@pytest.mark.asyncio
+async def test_state_store_replays_incremental_history(monkeypatch):
+    state_list = [
+        _scene_state(1, {"gpu_clusters": {"gpu:a": _scene_item("gpu:a", "A")}}),
+        _scene_state(
+            2,
+            {"gpu_clusters": {"gpu:a": _scene_item("gpu:a", "A"), "gpu:b": _scene_item("gpu:b", "B")}},
+        ),
+        _scene_state(
+            3,
+            {"gpu_clusters": {"gpu:a": _scene_item("gpu:a", "A Updated"), "gpu:b": _scene_item("gpu:b", "B")}},
+        ),
+    ]
+    state_index = {"value": 0}
+
+    async def _fake_build_visualization_scene_state(_db):
+        current_index = state_index["value"]
+        if current_index < len(state_list) - 1:
+            state_index["value"] += 1
+        return state_list[current_index]
+
+    monkeypatch.setattr(
+        "app.core.websocket.ue_scene.build_visualization_scene_state",
+        _fake_build_visualization_scene_state,
+    )
+
+    store = UeSceneStateStore(history_limit=5)
+    session = _FakeSession()
+
+    first_payloads = await store.get_broadcast_payloads(session)
+    second_payloads = await store.get_broadcast_payloads(session)
+    third_payloads = await store.get_broadcast_payloads(session)
+    replay_payloads = await store.get_sync_payloads(
+        session,
+        last_sequence=1,
+        reason="sequence_gap",
+    )
+
+    assert first_payloads[0]["update_type"] == "full"
+    assert second_payloads[0]["update_type"] == "incremental"
+    assert third_payloads[0]["sequence"] == 3
+    assert [payload["sequence"] for payload in replay_payloads] == [2, 3]
+
+
+@pytest.mark.asyncio
+async def test_connection_manager_broadcasts_to_subscribed_channel_only():
+    manager = ConnectionManager()
+    ws_dashboard = _FakeWebSocket()
+    ws_scene = _FakeWebSocket()
+
+    await manager.connect(ws_dashboard, "user-dashboard")
+    await manager.connect(ws_scene, "user-scene")
+
+    manager.subscribe(ws_dashboard, ["dashboard"])
+    manager.subscribe(ws_scene, ["ue_scene"])
+
+    await manager.broadcast({"type": "data_frame", "channel": "ue_scene"}, channel="ue_scene")
+
+    assert ws_dashboard.messages == []
+    assert ws_scene.messages == [{"type": "data_frame", "channel": "ue_scene"}]

database_schema.sql

@@ -86,12 +86,6 @@ CREATE TABLE collection_tasks (
     id BIGSERIAL PRIMARY KEY,
     datasource_id INTEGER NOT NULL REFERENCES data_sources(id) ON DELETE CASCADE,
     status task_status NOT NULL DEFAULT 'pending',
-    phase VARCHAR(30) DEFAULT 'queued',
-    phase_progress FLOAT,
-    phase_message VARCHAR(255),
-    phase_current BIGINT,
-    phase_total BIGINT,
-    phase_unit VARCHAR(30),
     started_at TIMESTAMP WITH TIME ZONE,
     completed_at TIMESTAMP WITH TIME ZONE,
     records_processed INTEGER DEFAULT 0,

docker-compose.local-model.yml

@@ -18,9 +18,6 @@ services:
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
-      args:
-        PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
-        UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
     container_name: planet_aiprovider
     ports:
       - "8010:8010"

docker-compose.simple.yml

@@ -5,12 +5,6 @@ services:
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
-      args:
-        PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
-        UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
-    env_file:
-      - ./aiprovider/.env
-      - ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
     container_name: planet_aiprovider
     ports:
       - "8010:8010"

docker-compose.yml

@@ -5,12 +5,8 @@ services:
     build:
       context: .
       dockerfile: aiprovider/Dockerfile
-      args:
-        PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
-        UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
     env_file:
       - ./aiprovider/.env
-      - ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
     container_name: planet_aiprovider
     ports:
       - "8010:8010"

docs/CHANGELOG.md

@@ -8,436 +8,6 @@ This project follows the repository versioning rule:
 - `improvement` -> `+0.0.1`（bugfix + 小功能混合）
 - `bugfix` -> `+0.0.1`
 
-## [0.47.0] — 2026-04-30
-
-Released: 2026-04-30
-
-### ✨ Highlights
-- 新增 AISStream WebSocket 船只采集器，并将 AIS 多源数据写入原始观测层，由聚合接口统一去重、合并和解释字段来源。
-- 设置页新增 AISStream API Key、采集范围 preset、运行状态、连接验证和凭证教程入口，让全球 AIS 采集链路可配置、可观察。
-- Earth 船只图层默认不再限制 5000 艘，并统一 marker 颜色、详情卡、hover 和搜索结果的船型归一化显示。
-
-### 🔧 Improvements
-- 聚合接口新增 `field_sources`、`selected_reasons`、`source_summary`、`quality_flags` 和冲突记录调试接口，动态字段默认优先采用更新的实时流观测。
-- AISStream 标准化支持 `MetaData.ShipName` 船名兜底，并将 AIS 数字船型映射为 Cargo / Tanker / Passenger / Fishing / Military。
-- 将仓库 docs 技能改为通用文档工作流，Planet 专属白名单、双语、裸文件标题和凭证教程规则迁移到 `docs/documentation-coverage-rules.md`。
-- 更新 AIS v4/v5 TODO 与计划文档，明确后续聚合策略配置、船舶资料 enrichment 和媒体缓存边界。
-
----
-
-## [0.46.3] — 2026-04-30
-
-Released: 2026-04-30
-
-### 🐛 Fixes
-- 优化 Starlink footprint 显示后的地球拖拽性能，避免旋转地球时每帧重建 footprint 大网格，同时保持现有视觉效果不变。
-- 恢复点击线缆后的呼吸透明度动画，让 locked / hover 线缆重新使用既有 pulse 配置。
-
----
-
-## [0.46.2] — 2026-04-30
-
-Released: 2026-04-30
-
-### 🐛 Fixes
-- 修复 Earth 启动时高清材质、云图和图层可见性绕过 `startupPriority` 的问题，统一由启动队列按文档顺序加载。
-- 修复保存为关闭的高清材质/图层仍会先加载再关闭的问题，并保持海陆基座作为国界线图层的常驻底图。
-- 修复搜索跳转会误关媒体面板、船只轨迹末端不贴合当前船只、Iridium footprint 被地表层遮挡等 Earth 交互问题。
-
-### 📝 Documentation
-- 更新 Earth 图层顺序、样式参考、使用手册和 AIS 聚合计划，补齐中英文说明与后续接入策略。
-
----
-
-## [0.46.1] — 2026-04-30
-
-Released: 2026-04-30
-
-### 🐛 Fixes
-- 修复新增 technical docs 文件存在但未进入 Docs 前端白名单时，侧栏不显示且 Markdown 链接无法解析到 `/docs/<slug>` 的问题。
-- 补齐数据源/采集器连接验证与 Earth Interactable 使用说明的英文文档，保证公开 Docs 切换 EN 时同名页面可访问。
-- 清理中英文 technical docs 中裸 `.md` 文件名链接标题，改为面向读者的语义标题。
-
-### 📝 Documentation
-- 将 Docs 前端白名单、公开文档双语配对、裸文件名链接标题三项检查写入 Claude 与 Codex 的 docs 技能流程。
-
----
-
-## [0.46.0] — 2026-04-30
-
-Released: 2026-04-30
-
-### ✨ Highlights
-- Earth 新增通用 Interactable 图标层，船只、算力中心、BGP 事件与观测站统一使用批量 Points、屏幕拾取、状态 glow 和状态缩放。
-- BGP 事件保留向外扩散圈，观测站保留雷达扫描层，并与 Interactable 主图标解耦到稳定的地表渲染层级。
-- 登陆点回归黄色球形 Sprite，贴近海缆层级并保持更稳定的地表显示和遮挡表现。
-
-### 🔧 Improvements
-- 新增 SVG asset 到 canvas texture 的 Interactable 资产加载路径，支持统一图标资源、缓存和可选染色。
-- 同坐标 Interactable 自动做地表切向避让，降低重叠物件无法选择的问题。
-- 优化 Earth toolbar 初始尺寸注入，避免首次显示原始尺寸后再跳到缩放尺寸。
-- 补充 Interactable 计划、使用说明、图层顺序和 Earth 前端上下文文档。
-- 修复船只 hover/locked 状态仅发光但放大反馈不明显的问题，将已有状态缩放接入通用图标层。
-
----
-
-## [0.45.0] — 2026-04-29
-
-### ✨ Highlights
-- 采集任务新增阶段级量化进度，`fetching` 可展示百分比、阶段说明和字节下载量。
-- AI Provider 启动链路支持从 `aiprovider/.env` 与 `~/.zshrc` 注入运行期配置，并避免密钥/模型变化触发镜像重建。
-- AI Provider Docker build context 收敛到服务必需文件，`uv sync` 接入 BuildKit 缓存以减少重复下载。
-
-### 🔧 Improvements
-- IPtoASN、OpenGeoFeed、NRO delegated 下载型采集器接入真实字节进度上报。
-- 数据源页、采集中任务弹窗和任务历史页展示阶段摘要，并在 tooltip 中保留完整进度细节。
-- 调整 Earth 船只默认高度偏移，进一步贴近地表展示。
-
----
-
-## [0.44.2] — 2026-04-29
-
-### 📝 Documentation
-- 补充 Earth 船只图层技术文档，记录分桶 `THREE.Points` 批量渲染、同尺寸交互 overlay 和屏幕空间 picking 的设计约束。
-- 同步 Earth 渲染图层顺序和样式参考，明确 AIS 船只 renderOrder、depthTest、图标尺寸、航向分桶与 hover 命中半径。
-- 更新船只渲染性能计划状态，标注 `0.44.1` 已落地的实现与后续全球 AIS / LOD 演进方向。
-
----
-
-## [0.44.1] — 2026-04-29
-
-### 🐛 Fixes
-- 修复 Earth 船只图层拖动不跟手的问题，将普通船只从独立 Sprite 切换为按航向分桶的批量 Points 渲染。
-- 修正船只 hover/click 拾取错位，改为屏幕空间命中检测并在拖拽/惯性期间跳过 hover 拾取。
-- 统一 AIS 船只普通态与交互态方向，并让 hover/locked glow 与普通图标保持同尺寸覆盖。
-- 恢复船只深度测试并收敛默认图标尺寸，避免北部岛屿/冰面附近出现明显压盖陆地的视觉问题。
-
----
-
-## [0.44.0] — 2026-04-29
-
-### ✨ Highlights
-- 重构数据源与采集器设置边界：数据源页回归目录和采集触发，采集器 endpoint、请求头、凭证与连接验证统一进入设置页
-- BarentsWatch AIS 完整接入凭证解析、连接检查、默认教程、AI 生成教程和船只采集/可视化链路
-- Earth 新增船只图例、缩放反馈胶囊、缩放感知拖拽灵敏度，并将船只渲染性能优化方案沉淀到 plans
-- 仪表盘重启服务新增前端重启 action，并让 runner 通过 `~/.zshrc` 继承本地环境变量
-
-### 🔧 Improvements
-- 采集器连接状态改为基于成功采集或手动连接校验 checksum 判断，避免只依赖前端样式状态
-- 数据源页新增采集中任务标签和任务进度弹窗，内置与自定义数据源统一展示
-- Earth 国界线进一步贴近地表，并补充船只图层渲染顺序、样式和用户手册说明
-- docs skill 与 Claude/Codex 文档流程补齐技术文档和 plans 的职责边界
-
----
-
-## [0.43.1] — 2026-04-28
-
-### 🐛 Fixes
-- 修正 `planet.sh` 在全量 restart 后启动 AI Provider 时的提示语义，避免把预期内未就绪描述成异常不健康
-
----
-
-## [0.43.0] — 2026-04-28
-
-### ✨ Highlights
-- 新增 Earth 船舶追踪链路，接入 BarentsWatch AIS 凭证配置、采集器、后端 vessel 模型/API 与前端 Earth 船舶图层
-- 新增自定义数据源映射流程，支持样本抓取、目标 schema、AI 辅助生成映射、预览校验和映射执行
-- Settings 拆分 AI Provider 与采集器凭证配置，DataSources 只保留采集状态、运行参数和必要引导
-
-### 🔧 Improvements
-- AI Provider 支持运行时 LLM 配置、provider preset 下拉与刷新，并在 Playground 中引导到 AI 配置页
-- Markdown 渲染器补齐代码块复制按钮、语言标签、任务列表、图片、自动链接、删除线和文档主题样式
-- Docs 公开导航改为显式元数据白名单，避免开发任务文档自动出现在“其他”分组
-- 将 Codex/Claude cleanup、docs、goal-driven、release 流程补充 CLI-first 约束，并把 `rules.md` 整理成可按模块加载的工程规则
-
----
-
-## [0.42.2] — 2026-04-28
-
-### 🐛 Fixes
-- Docs 中文模式下补齐左侧分组、文档标题、页头分类与搜索结果分类翻译，并更新文档站品牌标题/副标题文案
-
----
-
-## [0.42.1] — 2026-04-28
-
-### 🐛 Fixes
-- 修正 release skill 的 feature 版本计算规则：minor 进位时 patch 必须重置为 `0`，例如 `0.41.2` 应发布为 `0.42.0`
-
----
-
-## [0.42.0] — 2026-04-28
-
-### ✨ Highlights
-- 新增公开 `/docs` 文档站，支持中英文技术文档、使用手册、Quickstart、搜索、目录锚点与浅色/深色/跟随系统主题
-- Earth 在无高清材质时新增轻量 Fresnel 边缘提示，并调整卫星覆盖默认显示与地表材质可读性
-
-### 🔧 Improvements
-- 将技术文档整理为 `docs/technical/zh` 与 `docs/technical/en`，并补充控制台、`planet.sh`、Earth 与公共组件使用说明
-- 新增 `SegmentedControl` 公共滑块组件，支持缩放参数，复用到 docs 语言与主题切换
-- Markdown 渲染器接入自定义滚动条，表格与代码块在深色模式和 overflow 场景下保持可读
-- Docs 搜索结果支持内部滚动、点击外部关闭、重新聚焦恢复上次搜索结果
-- Earth 工具栏展开状态与设置持久化版本迁移继续收口，改善默认面板和快捷关闭行为
-
----
-
-## [0.41.2] — 2026-04-27
-
-### 🔧 Improvements
-- `planet.sh` 启动链路新增 verbose 滚动输出窗口，并在后端端口占用时打印目标地址和监听进程诊断
-- Docker 构建支持通过 build args 覆盖 Python 与 uv 镜像，方便 Docker Hub 不稳定时切换镜像源
-
-### 🐛 Fixes
-- Earth 海缆登陆点改为基于相机射线与地球遮挡判断可见性，修复旋转后 pin 可见性滞后一帧的问题
-
-### 🔧 Improvements
-- `docker-compose*.yml` 为 AI Provider 构建传入 `PYTHON_IMAGE` / `UV_IMAGE` 参数，默认仍使用官方镜像
-- 后端启动失败遇到 `Address already in use` 时输出 `lsof`、`ss` 与 PID 命令行信息
-- verbose 模式下 AI Provider build、后端与前端启动日志会在 spinner 下方保留最新 5 行滚动展示
-
----
-
-## [0.41.1] — 2026-04-27
-
-### 🐛 Fixes
-- 修复新闻直播面板设置项持久化失效：`closeTransientMobileOverlays` 通过旁路路径隐藏面板导致下次 persist 快照到错误状态，改为不重新从 DOM 读取面板可见性
-- 修复登陆点 pin 在地球侧面被半截遮挡：改为在接近地平线前（dot < 0.05）主动隐藏，避免深度测试切片
-
-### 🔧 Improvements
-- 将所有画布绘制的图标抽取为 SVG，存入 `frontend/public/earth/assets/icons/`，新增图标规范到 `rules.md`
-
----
-
-## [0.41.0] — 2026-04-27
-
-### ✨ Highlights
-- Earth 图层系统完成地表到天空的注册顺序与关注优先的面板顺序拆分，支持基座海陆色块、国界、高清材质、云图、地形、算力、BGP、卫星、轨迹与海缆的稳定层级
-- 国界层新增真实行政区轮廓交互与中国/台湾联动高亮，修复高清材质、地形、footprint、卫星与经纬线之间的遮挡和 hover 竞争
-
-### 🔧 Improvements
-- 新增无轮廓基座地图，所有图层关闭时仍保留 `#010609` 海洋与 `#080f1b` 陆地色块
-- 将大气云图抽象为独立图层并接入桌面/移动端图层开关、持久化状态与启动同步
-- 高清材质改为独立纹理覆盖层，地形显示在高清材质上方，并在高清材质关闭/恢复时保持原地形开关意图
-- 补充 Earth 渲染层级与图层样式文档，记录正式图层名、变量名、材质颜色、线宽与 renderOrder
-
----
-
-## [0.40.5] — 2026-04-26
-
-### 🔧 Improvements
-- 卫星拖尾改用 Instanced screen-space ribbon，单 draw call 渲染所有轨迹段，支持像素级宽度控制
-- Iridium 地面覆盖重写为球面投影径向网格，修复填充光晕不可见问题；新增外圈 LineLoop
-- 搜索面板打开时改用双 rAF 延迟聚焦输入框，确保 CSS 过渡完成后焦点可靠触发
-- 代码清理：提取 `IRIDIUM_OVERLAY_COLOR`、`IRIDIUM_REFERENCE_ALTITUDE_KM` 常量，消除重复三角函数调用
-
----
-
-## [0.39.0] — 2026-04-24
-
-## [0.40.4] — 2026-04-26
-
-### 🔧 Improvements
-- 新增页面可见性恢复处理，页面从后台切回前台时主动刷新卫星位置，避免累积后台时间在下一帧一次性回放
-- 抽出卫星轨迹状态与轨迹几何清理 helper，统一后台恢复与清空数据时的轨迹重置路径
-
-### 🐛 Fixes
-- 修复页面在后台停留较久后恢复前台时，卫星轨迹因超大 `deltaTime` 突然跳变、拖尾异常拉长的问题
-- 修复后台恢复后首帧仍沿用旧轨迹缓存，导致轨迹与当前卫星位置短时错位的问题
-
----
-
-## [0.40.3] — 2026-04-25
-
-### 🔧 Improvements
-- 卫星点云升级为自定义 ShaderMaterial，支持 per-point alpha 控制，锁定/悬停卫星从点云中精确隐藏
-- 修复锁定环与自发光选中标记的 depthTest 错误（false → true），消除远端渲染穿透 artifact
-- 新增锁定环悬停态缩放与线宽（LOCKED_RING_HOVER_SCALE / LOCKED_RING_HOVER_LINE_WIDTH）
-- 修复 updateLockedDotWorldTransform / updateLockedHaloWorldTransform 未强制刷新 matrixWorld 导致的位置漂移
-
----
-
-## [0.40.2] — 2026-04-24
-
-### 🔧 Improvements
-- 卫星点大小随镜头缩放动态调整，拉近变大、拉远变小，响应与相机距离线性对应
-- 调小卫星点默认基础尺寸（dotSize 2.8），缩放范围更合理
-
----
-
-## [0.40.1] — 2026-04-24
-
-### 🔧 Improvements
-- 卫星选中标记（lockedring / lockeddot / 光晕）颜色统一跟随图例轨道倾角分类配色
-- 修复 Starlink footprint 在特定视角下遮蔽卫星点的渲染顺序问题（Group renderOrder 影响子 Mesh 排序）
-- footprint 材质改为 `depthTest: false` + 相机朝向 limbFade，替代 polygonOffset 深度竞争方案
-- 修复选中海缆时误触发附近卫星高亮（该行为属于 BGP 事件点逻辑，不应用于海缆）
-
----
-
-## [0.40.0] — 2026-04-24
-
-### ✨ Highlights
-- Earth 卫星 footprint 正式按星座能力分层：Starlink 保留专用地表覆盖，Iridium 改为独立外圈覆盖表达，其它非 Starlink 星座不再误用同一套 footprint
-- Earth 卫星详情卡补齐覆盖能力与当前显示说明，用户现在可以直接看见每颗卫星为什么显示 footprint、为何回退为自身发光
-
-### 🔧 Improvements
-- 后端可视化接口新增并透传 `constellation_group` 与 `footprint_policy`，前端据此执行 capability-gated footprint renderer
-- 新增 Iridium 独立 coverage ring adapter，并继续保留 Starlink 专用 footprint 调校与昼夜可读性增强
-- 新增 Earth 卫星 footprint 策略技术文档，明确 GNSS、generic LEO、GEO 与 Iridium 的显示边界
-
-### 🐛 Fixes
-- 修复前后端对 Iridium footprint policy 命名不一致，导致策略分发语义含混的问题
-- 清理 Starlink footprint 渲染中的未使用常量与过时命名，减少后续继续调校时的歧义
-
----
-
-## [0.39.0] — 2026-04-24
-
-### ✨ Highlights
-- 后端正式落下统一结构化日志地基：请求上下文、事件名、脱敏与持久化链路开始收口为可扩展的企业级日志体系
-- 系统日志页重构为真正的日志工作台：顶部筛选更紧凑，终端日志区成为主视觉，移动端 Earth 新闻/态势细节交互继续补稳
-
-### 🔧 Improvements
-- 新增 `backend/app/core/logging.py`，统一 `request_id`、`service`、`event` 注入与敏感字段脱敏，并接入后端主入口、调度器、缓存、数据库和可视化链路
-- 系统日志页筛选区重排为更紧凑的两层结构，信息摘要并入终端工具栏 tooltip，日志终端区留出更稳定的按钮避让空间
-- Earth 移动端态势抽屉补齐宽度约束与图例换行规则，新闻详情抽屉在巡航切换时可同步更新标题和摘要
-
-### 🐛 Fixes
-- 修复 `/tmp/planet_backend.log` 中混入空字节时，日志摘要条行数与实际可见日志不一致的问题
-- 修复移动端“态势”tab 在内容渲染后被图例文本撑宽、超出一屏的问题
-- 修复移动端新闻详情抽屉在巡航切换下一条新闻时标题更新但 summary 不同步的问题
-
----
-
-## [0.38.0] — 2026-04-23
-
-### ✨ Highlights
-- Earth 新闻正式接入通用巡航层：新闻和 BGP 统一进入可配置巡航模块，桌面端与移动端都能在巡航聚焦时展示对应新闻卡片
-- 系统日志页升级为结构化过滤链路：按真实时间戳、结构化级别和字符串检索统一筛选，不再依赖前端或后端从日志文本里猜结果
-
-### 🔧 Improvements
-- 新闻巡航补齐业务适配层：按发生地与时间生成巡航目标，桌面端与移动端统一标题 + summary 卡片风格，并增加连线与打字机摘要展示
-- 日志页筛选体验重排，统一服务源、级别、行数、时间和检索布局，日历标记改为由后端返回的结构化每日聚合结果驱动
-- 后端补充 `system_logs` 结构化解析与多级别精确过滤能力，Earth 浏览器端日志缓冲与系统日志 API 现在走同一套筛选语义
-
-### 🐛 Fixes
-- 修复新闻巡航模块开启后难以关闭、桌面/移动端设置状态互相污染的问题
-- 修复新闻巡航卡片缺少摘要、移动端详情样式不统一、新闻巡航缺少连线的问题
-- 修复日志级别筛选会被访问日志 query string 中的 `level=error` 等参数污染，从而把 `INFO` 行误判为 `ERROR` 的问题
-
----
-
-## [0.37.2] — 2026-04-23
-
-### ✨ Highlights
-- Earth 图层系统新增经纬线开关，桌面图层面板与移动端抽屉都可直接控制
-
-### 🔧 Improvements
-- 经纬线正式接入 Earth layer registry，复用现有图层切换、移动端图层卡片与设置持久化流
-
-### 🐛 Fixes
-- 修复经纬线只能默认常驻、无法作为独立图层开关控制的问题
-
----
-
-## [0.37.1] — 2026-04-23
-
-### ✨ Highlights
-- `planet.sh` 后端重启链路修复 `uvicorn --reload` 残留 worker 场景，`restart` 现在能真正替换旧实例
-
-### 🔧 Improvements
-- 收口后端清理逻辑，统一按 `uvicorn` 进程、端口占用进程和进程组执行清理，减少 reload 场景漏杀分支
-
-### 🐛 Fixes
-- 修复部分机器执行 `./planet.sh restart --allow-lan` 后后端仍停留旧实例，导致 `/api/v1/visualization/geo/compute-centers` 返回 `404` 的问题
-
----
-
-## [0.35.1] — 2026-04-22
-## [0.37.0] — 2026-04-23
-
-### ✨ Highlights
-- Earth 连线系统正式从巡航里解耦成通用 callout connector：桌面端和移动端统一支持对象级锚点、四边切换与临界区边缘滑动
-- BGP 巡航展示继续收口为稳定的“先定位卡片、再连真实锚点、再展示卡片”链路，移动端 popup 与桌面 info panel 的路线规则统一
-
-### 🔧 Improvements
-- connector 配置从 `CRUISE_CONFIG` 拆到独立 `CONNECTOR_CONFIG`，默认类名、动画名和实例命名也全部去 cruise 语义
-- 移动端 popup 增加更稳定的 dock/obstacle 处理，拖动卡片时连线起终点会持续按几何关系自适应刷新
-- Earth 多个图层与控制逻辑继续收口，补充算力中心/BGP 风格对齐、layer panel 与相关交互细节调整
-
-### 🐛 Fixes
-- 修复巡航模式下终点只像“视觉锚点”而不是真实绑定对象的问题，卡片拖动后终点现在会跟随
-- 修复移动端与桌面端多类连线路线异常：压线、反向、临界区折返、起点遮挡事件点等问题
-- 修复对象矩形临界区内连线仍强制中点到中点导致路线像“先钻进 source 内部”再出去的问题
-
----
-
-## [0.35.1] — 2026-04-22
-## [0.36.0] — 2026-04-22
-
-### ✨ Highlights
-- Earth 新增统一“算力中心”图层：接入超算与 GPU 集群，支持搜索、统计、图例、详情卡与独立图层开关
-- 算力中心支持精确位置与估算位置两种状态，估算点会以问号角标区分，避免数据不全时整批节点在地图上消失
-
-### 🔧 Improvements
-- Earth 详情卡拖拽与地球拖拽交互继续收口，减少拖动卡片和旋转地球时的选中文本与 pointer 竞争
-- `planet.sh` 改为通过独立脚本计算 AI Provider 依赖指纹，降低与根仓库依赖版本文件的无关耦合
-- README 补充 WSL / Windows 局域网访问排查与转发配置说明，便于开发环境联调
-
-### 🐛 Fixes
-- 修复 Earth 算力中心图层在无原始坐标时无法显示的问题，支持站点提示和国家级估算回退
-- 修复信息卡拖拽事件可能被卡片级 stopPropagation 吞掉，导致拖拽流中断的问题
-
----
-
-## [0.35.1] — 2026-04-22
-
-### ✨ Highlights
-- Earth 统计展示改为统一 `data-earth-stat` 绑定机制，桌面 HUD 和移动端抽屉复用同一套状态更新入口
-
-### 🔧 Improvements
-- 收口海缆、登陆点、卫星、BGP 事件与 BGP 状态的统计写入逻辑，减少后续继续补桌面/移动双写分支的成本
-
-### 🐛 Fixes
-- 修复移动端态势抽屉中的海缆、登陆点与 BGP 统计在图层切换后可能停留旧值的问题
-
----
-
-## [0.35.0] — 2026-04-22
-
-### ✨ Highlights
-- Earth 移动端底部抽屉系统全面上线：响应式布局自动切换、Tab 导航、手势上拉/下滑开合、惯性速度判定
-- 移动端点击可交互物件（海缆、登陆点、卫星、BGP）后弹出智能定位悬浮卡片，可拖动，点击跳转详情
-
-### 🔧 Improvements
-- 抽屉把手区域缩小至 36px（collapsed 时仅露出把手，不遮挡地球操作区）
-- 抽屉定期弹跳动画提示用户可上拉，5 秒间隔，打开后自动停止
-- 通知胶囊位置调整，不再覆盖品牌 logo
-- 移动端单指旋转、双指捏合缩放地球，触控事件冲突修复（pointer-events 级联）
-
-### 🐛 Fixes
-- 修复移动端抽屉 shell 因 layout 高度（240px+）遮挡地球触控区域，pointer-events 改为按层级精确控制
-- 修复悬浮卡片因 setPointerCapture 在 iOS Safari 抑制合成 click 事件导致无法点击的问题
-
----
-
-## [0.34.0] — 2026-04-22
-
-### ✨ Highlights
-- Earth 搜索面板正式接入，支持搜索海缆、登陆点、卫星、BGP 事件与观测站，并可直接聚焦到对应对象
-- `planet.sh --allow-lan` 打通 Bun + Vite 的局域网开放链路，启动成功后自动打印推荐访问地址与后端健康检查地址
-
-### 🔧 Improvements
-- 前端开发启动链统一改成 Bun 直接执行 Vite 入口，不再依赖 shell 中额外暴露的 Node 路径
-- Earth 搜索结果接入登陆点详情卡片与对象聚焦，搜索后可直接进入对应详情流
-- `planet.sh` 补充局域网 IPv4 自动识别与推荐地址输出，减少 WSL 局域网调试成本
-
-### 🐛 Fixes
-- 修复 `./planet.sh restart --allow-lan` 全量重启时未把 `--allow-lan` 继续传给 `start()`，导致前端退回本机监听的问题
-- 修复 WSL + Bun 环境下前端偶发因 Vite 启动链不稳定而无法正确监听 `0.0.0.0:3000` 的问题
-
----
-
 ## [0.33.0] — 2026-04-22
 
 ### ✨ Highlights
@@ -495,6 +65,8 @@ Released: 2026-04-30
 
 ---
 
+## [0.31.0] — 2026-04-21
+
 ## [0.31.2] — 2026-04-21
 
 ### ✨ Highlights
@@ -531,8 +103,6 @@ Released: 2026-04-30
 
 ---
 
-## [0.31.0] — 2026-04-21
-
 ### ✨ Features
 - Earth 新增"巡航展示"模式：自动轮播 BGP 异常事件，逐帧追踪连接线位置，支持外部交互立即中断序列（cancel notifier 模式）
 - 巡航目标事件点高亮显示：hover 外观 + 锁定脉冲动画，并与点击行为统一展示周边受影响卫星与海缆
@@ -546,6 +116,8 @@ Released: 2026-04-30
 
 ---
 
+## [0.29.1] — 2026-04-20
+
 ## [0.30.0] — 2026-04-21
 
 ### ✨ Features

docs/documentation-coverage-rules.md

@@ -1,117 +0,0 @@
-# Documentation Coverage Rules
-
-This file contains Planet-specific documentation coverage rules. Documentation skills and agents should read this file before deciding which docs to update. Keep tool-specific workflow in skills; keep product and repository rules here.
-
-## Scope Rules
-
-- User-visible workflow changes must update `docs/technical/zh/manual.md` and usually `docs/technical/zh/quickstart.md`.
-- If an English counterpart exists for user-facing docs such as `manual.md` or `quickstart.md`, update `docs/technical/en/...` enough that it does not contradict the Chinese source.
-- Control console page responsibility changes must update `docs/technical/zh/frontend-admin-frontend-context.md`.
-- Earth frontend behavior changes must update `docs/technical/zh/earth-frontend-context.md`.
-- Earth layer additions, `renderOrder`, altitude/radius offsets, depth strategy, pointer picking, legend modes, or layer panel/startup ordering must update `docs/technical/zh/earth-render-layer-order.md`.
-- Earth layer visual style or legend symbol/color semantics should also update `docs/technical/zh/earth-layer-style-reference.md` when that reference is affected.
-- Collector, datasource, credential, settings, connectivity, scheduler, or API changes must update the relevant backend docs, especially `docs/technical/zh/backend-collectors.md` and any datasource/settings-specific doc.
-- When a change turns an old plan assumption into current behavior, update the relevant `docs/plans/*.md` with a status note instead of leaving contradictory instructions.
-- Search docs for stale terms introduced by the change, for example old tab names, old route responsibilities, obsolete auth assumptions, or renamed UI labels.
-
-## Public Docs Rules
-
-- If adding a new technical document, add it to `docs/technical/zh/README.md` when it should be discoverable from the technical docs index.
-- If a technical document should be visible in the public Docs page or linked from a technical README, register it in `frontend/src/pages/Docs/docs-content.ts` under `DOCS_METADATA`. Files under `docs/technical/{zh,en}/` are not automatically routable.
-- For every public technical doc, keep the bilingual file pair in sync by filename: `docs/technical/zh/<name>.md` and `docs/technical/en/<name>.md`. If content is intentionally Chinese-only or English-only, state that intentionally in the final note.
-- Public docs should use readable link text, not raw filenames such as `manual.md`.
-
-## Credential Collector Rules
-
-- Any built-in collector marked `requires_credentials: true` and `credential_status: supported` must have:
-  - a `credential_provider` in `backend/app/core/datasource_defaults.py`;
-  - a default credential guide in `backend/app/services/credential_guides.py`;
-  - a supported connectivity provider in `backend/app/services/datasource_connectivity.py`;
-  - settings UI guidance or a credential form in `frontend/src/pages/Settings/Settings.tsx`;
-  - a regression test that fails if the guide/provider is missing.
-
-## Recommended Checks
-
-Run the checks that match the affected docs.
-
-### Duplicate Bilingual Docs
-
-```bash
-python - <<'PY'
-from pathlib import Path
-same = []
-for en in sorted(Path("docs/technical/en").glob("*.md")):
-    zh = Path("docs/technical/zh") / en.name
-    if zh.exists() and en.read_text() == zh.read_text():
-        same.append(en.name)
-if same:
-    raise SystemExit("identical en/zh docs: " + ", ".join(same))
-print("no identical en/zh docs")
-PY
-```
-
-### Language-Less Technical Links
-
-```bash
-rg -n "/home/ray/dev/linkong/planet/docs/technical/(?!zh|en)" docs/technical/zh --pcre2
-```
-
-This should return no matches.
-
-### Public Docs Registry
-
-```bash
-python - <<'PY'
-import re
-from pathlib import Path
-
-metadata = Path("frontend/src/pages/Docs/docs-content.ts").read_text()
-known = set(re.findall(r"'([^']+\.md)':\s*\{", metadata))
-known.add("README.md")
-
-missing = []
-for readme in [Path("docs/technical/zh/README.md"), Path("docs/technical/en/README.md")]:
-    if not readme.exists():
-        continue
-    for href in re.findall(r"\]\(([^)]+\.md)\)", readme.read_text()):
-        path = Path(href)
-        if "docs/technical/" not in href:
-            continue
-        filename = path.name
-        if filename not in known:
-            missing.append(f"{readme}: {filename}")
-
-if missing:
-    raise SystemExit("docs README links missing DOCS_METADATA: " + ", ".join(missing))
-print("docs README links are whitelisted")
-PY
-```
-
-### Public Bilingual Pairs
-
-```bash
-python - <<'PY'
-import re
-from pathlib import Path
-
-metadata = Path("frontend/src/pages/Docs/docs-content.ts").read_text()
-filenames = sorted(set(re.findall(r"'([^']+\.md)':\s*\{", metadata)) - {"README.md"})
-missing = []
-for filename in filenames:
-    for lang in ("zh", "en"):
-        path = Path("docs/technical") / lang / filename
-        if not path.exists():
-            missing.append(str(path))
-if missing:
-    raise SystemExit("missing bilingual docs: " + ", ".join(missing))
-print("public docs have zh/en file pairs")
-PY
-```
-
-### Raw Filename Link Titles
-
-```bash
-rg -n "\[[^]]+\.md\]\(" docs/technical/zh docs/technical/en
-```
-
-This should return no matches for polished public docs.

docs/plans/README.md

@@ -16,19 +16,11 @@
 
 当前重点入口：
 
-- [earth-mobile-drawer-ui-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
-- [earth-compute-center-bgp-style-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)
 - [earth-renderer-architecture-separation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-renderer-architecture-separation-plan.md)
-- [earth-country-boundary-overlay-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-country-boundary-overlay-plan.md)
 - [earth-predicted-orbit-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
 - [earth-webgl-instancing-satellites-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
 - [earth-real-terrain-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
 - [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
-- [earth-news-cruise-summary-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
-- [earth-vessel-rendering-performance-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-rendering-performance-plan.md)
-- [AIS 多源采集、冲突记录与聚合接口计划](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-ais-aggregation-plan.md)
-- [earth-interactable-layer-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-interactable-layer-plan.md)
-- [frontend-public-docs-site-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
 - [frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
 - [ue5-mvp-fused-plan.md](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)
 

docs/plans/datasource-custom-api-mapping-plan.md

@@ -1,426 +0,0 @@
-# 自定义 API 数据源与 LLM 映射系统 — 实施计划
-
-**状态**：规划中
-**创建日期**：2026-04-28
-**核心原则**：LLM 辅助生成映射配置；生产采集使用确定性转换引擎
-
-## 已确认决策
-
-| 项目 | 决策 |
-|-----|------|
-| 自定义 API 的定位 | 作为内置数据源的补充入口，不直接等同于 Earth 新功能 |
-| LLM 的职责 | 探索未知 API、分析样本 JSON、生成 mapping 草案 |
-| 采集时是否调用 LLM | 不调用；采集链路必须确定性、可审计、可复现 |
-| 自定义数据如何进入 Earth | 必须映射到已支持的目标 schema，或先进入通用数据沉淀 |
-| 外部凭证放置位置 | Settings / 外部集成统一管理 provider token；DataSources 引用 provider profile |
-| TimescaleDB | 放入 TODO；高频时序数据稳定后再评估迁移 |
-
----
-
-## 一、背景与问题
-
-当前系统已经有 `datasource_configs`，可以配置自定义数据源的 endpoint、auth、headers、config，也已经有部分 collector 会读取这些配置。但这只能解决“怎么请求数据”，还没有解决以下问题：
-
-- API 返回 JSON 后，如何转换成系统已有领域模型。
-- 自定义数据源是补充已有能力，还是全新数据沉淀。
-- 转换规则由谁生成、谁校验、谁执行。
-- 未知数据是否能自动在 Earth 上展示。
-- 外部 token 是放在全局配置中心，还是放在每个 datasource 下。
-
-专业做法是把“请求配置”“外部凭证”“目标 schema”“字段映射”“采集执行”拆开：
-
-- Settings 管外部集成凭证，例如 AI Provider、BarentsWatch、未来付费 AIS API。
-- DataSources 管具体数据源实例，例如 endpoint、调度频率、目标 schema、mapping 版本。
-- LLM 只在配置阶段辅助生成 mapping，不进入生产采集链路。
-- Earth 只消费明确 schema 的数据，不消费任意未知 JSON。
-
----
-
-## 二、目标架构
-
-### 2.1 自定义 API 数据源生命周期
-
-```mermaid
-flowchart LR
-    A[配置 endpoint/auth/request] --> B[抓取 sample JSON]
-    B --> C[选择目标 schema]
-    C --> D[LLM 生成 mapping 草案]
-    D --> E[确定性 mapping engine 预览]
-    E --> F[schema validation]
-    F --> G[保存 mapping version]
-    G --> H[scheduler 执行 mapped collector]
-    H --> I[写入目标表或 generic_records]
-```
-
-### 2.2 目标 schema 分层
-
-| schema | 用途 | Earth 可视化 |
-|-------|------|-------------|
-| `vessel_ais` | 船只 AIS 位置、航速、航向、MMSI 等 | 进入船舶图层 |
-| `geo_points` | 通用点位数据，包含经纬度、名称、类型、时间 | 进入通用 geo layer（TODO） |
-| `news_events` | 新闻/事件类数据，带时间、地点、摘要、来源 | 复用新闻/事件链路 |
-| `compute_centers` | 算力中心、机房、数据中心数据 | 复用算力中心图层 |
-| `generic_records` | 未知结构化数据沉淀 | 不直接展示 |
-
-v1 建议优先实现：
-
-- `vessel_ais`
-- `geo_points`
-- `generic_records`
-
-其他 schema 可先在 registry 中预留名称，但不承诺完整落库与可视化。
-
-### 2.3 LLM 的边界
-
-LLM 可以做：
-
-- 根据 API 文档或 sample JSON 解释字段含义。
-- 推荐目标 schema。
-- 生成 mapping JSON 草案。
-- 给出字段置信度和需要人工确认的字段。
-- 帮用户发现分页、数组路径、时间字段、坐标字段。
-
-LLM 不应该做：
-
-- 在正式采集时参与每批数据转换。
-- 生成并执行 Python/JavaScript 代码。
-- 接触 API key、bearer token、basic auth password。
-- 自动创建新的 Earth 图层或数据库表。
-
----
-
-## 三、后端实施计划
-
-### Phase 1 — Target Schema Registry
-
-新增代码级 registry，统一描述系统支持的目标数据类型。
-
-每个 target schema 至少包含：
-
-- `key`：例如 `vessel_ais`。
-- `label`：前端展示名称。
-- `description`：适用场景。
-- `fields`：字段名、类型、是否必填、说明、示例。
-- `validator`：Pydantic 或等价校验器。
-- `destination`：写入目标，例如 vessel 表、generic_records、future geo layer。
-
-示例概念：
-
-```json
-{
-  "key": "vessel_ais",
-  "fields": [
-    {"name": "mmsi", "type": "integer", "required": true},
-    {"name": "lat", "type": "float", "required": true},
-    {"name": "lon", "type": "float", "required": true},
-    {"name": "sog", "type": "float", "required": false},
-    {"name": "cog", "type": "float", "required": false},
-    {"name": "received_at", "type": "datetime", "required": false}
-  ]
-}
-```
-
-### Phase 2 — Mapping Template Model
-
-新增 mapping 配置持久化表，建议命名为 `datasource_mapping_templates`。
-
-关键字段：
-
-- `id`
-- `datasource_config_id`
-- `target_schema`
-- `mapping_json`
-- `sample_payload_hash`
-- `validation_status`
-- `version`
-- `is_active`
-- `created_at`
-- `updated_at`
-
-`mapping_json` 是声明式 DSL，不允许任意代码执行。
-
-示例：
-
-```json
-{
-  "source": {
-    "items_path": "$.data.vessels[*]"
-  },
-  "fields": {
-    "mmsi": {"path": "$.mmsi", "type": "integer"},
-    "lat": {"path": "$.latitude", "type": "float"},
-    "lon": {"path": "$.longitude", "type": "float"},
-    "sog": {"path": "$.speedOverGround", "type": "float", "default": null},
-    "received_at": {"path": "$.timestamp", "type": "datetime"}
-  }
-}
-```
-
-### Phase 3 — Deterministic Mapping Engine
-
-实现独立 mapping engine，输入 sample/raw payload 和 mapping JSON，输出目标 schema 记录。
-
-v1 支持能力：
-
-- JSONPath/JMESPath 风格路径提取。
-- 数组展开。
-- 默认值。
-- 基础类型转换：string、integer、float、boolean、datetime。
-- 坐标范围校验。
-- 简单枚举映射。
-- 错误收集：缺字段、类型转换失败、路径不存在。
-
-明确不支持：
-
-- 任意表达式执行。
-- 用户提交脚本。
-- LLM runtime 修复。
-
-### Phase 4 — LLM Mapping Assistant API
-
-新增配置阶段 API：
-
-- `POST /api/v1/datasources/custom/sample`
-  - 按 datasource 请求配置抓取 sample JSON。
-- `GET /api/v1/datasources/target-schemas`
-  - 返回可选目标 schema 和字段说明。
-- `POST /api/v1/datasources/mappings/propose`
-  - 输入 sample JSON + target schema，调用 AI provider 生成 mapping 草案。
-- `POST /api/v1/datasources/mappings/preview`
-  - 使用确定性 mapping engine 预览转换结果。
-- `POST /api/v1/datasources/mappings`
-  - 保存 mapping 版本。
-- `PUT /api/v1/datasources/mappings/{id}`
-  - 更新 mapping，生成新版本或覆盖草稿。
-- `POST /api/v1/datasources/{id}/run-mapped`
-  - 手动触发一次 mapped collector。
-
-安全要求：
-
-- `propose` 请求发送给 LLM 前必须脱敏 sample。
-- auth headers、token、password 不进入 prompt。
-- LLM 返回结果必须再经过 mapping schema 校验。
-
-### Phase 5 — Generic Mapped HTTP Collector
-
-新增通用 collector：
-
-- 读取 `DataSourceConfig` 请求配置。
-- 读取 active mapping template。
-- 拉取 API 数据。
-- 使用 mapping engine 转换。
-- 使用 target schema validator 校验。
-- 调用 destination handler 写入目标表或 generic storage。
-- 将失败记录写入错误日志或 dead-letter 结构。
-
-对于 `generic_records`：
-
-- 保存 datasource id。
-- 保存 target schema。
-- 保存 normalized JSON。
-- 保存 raw payload 摘要或 raw reference。
-- 保存采集时间、source timestamp、mapping version。
-
----
-
-## 四、前端实施计划
-
-### Phase 1 — Settings 外部集成
-
-Settings 中保留统一外部集成配置：
-
-- AI Provider：base URL、model、API key。
-- BarentsWatch：client id/client secret 或 bearer token。
-- 未来付费接口：AISHub、MarineTraffic、VesselFinder 等 provider profile。
-
-DataSources 不直接管理全局 secret，只引用 provider profile。
-
-### Phase 2 — Settings 自定义源向导
-
-自定义数据源配置入口应放在 `/settings` 的“采集器设置”或后续专门的自定义采集器设置区。`/datasources` 保持数据源目录和采集触发职责，不再承载编辑入口。
-
-自定义数据源配置改成向导或右侧 drawer：
-
-1. Request
-   - endpoint
-   - method
-   - auth profile
-   - headers
-   - query/body config
-   - schedule
-2. Sample
-   - 点击抓取 sample
-   - 展示 JSON tree
-   - 支持选择数组根路径
-3. Target Schema
-   - 选择 `vessel_ais`、`geo_points`、`generic_records`
-   - 展示该 schema 必填字段
-4. Mapping Proposal
-   - 调用 LLM 生成 mapping 草案
-   - 显示字段匹配置信度
-   - 标出需要人工确认的字段
-5. Preview
-   - 用确定性 engine 预览前 N 条转换结果
-   - 展示校验错误
-6. Save & Enable
-   - 保存 mapping version
-   - 启用调度或仅保存草稿
-
-### Phase 3 — 运维视图
-
-为 mapped datasource 展示：
-
-- 上次运行时间。
-- 成功记录数。
-- 失败记录数。
-- 当前 mapping version。
-- 目标 schema。
-- 最近错误。
-- 手动运行按钮。
-
----
-
-## 五、数据库与存储策略
-
-### v1：继续使用 PostgreSQL
-
-PostgreSQL 可以承载当前规模的采集、关系查询、JSONB 沉淀和基础时序查询。v1 不必因为“时序数据”立刻引入 TimescaleDB。
-
-适合继续用 PostgreSQL 的场景：
-
-- 数据量可控。
-- 最近状态查询为主。
-- 历史保留窗口较短。
-- 查询模式还没稳定。
-- 需要快速迭代 schema 与 mapping。
-
-### TODO：TimescaleDB
-
-以下条件满足后，再评估 TimescaleDB：
-
-- AIS、遥测、轨迹类数据达到高频持续写入。
-- 需要按时间窗口做聚合、降采样、retention policy。
-- 单表时间序列查询明显成为瓶颈。
-- 历史轨迹保留从 24h 扩展到数周或数月。
-
-候选迁移对象：
-
-- `vessel_position`
-- future telemetry tables
-- future generic time-series records
-
-备选方案：
-
-- PostgreSQL 原生按天/月分区。
-- TimescaleDB hypertable。
-- 热数据 PostgreSQL，冷数据对象存储。
-
----
-
-## 六、安全与治理
-
-### Secret 管理
-
-- Settings 中保存 provider credentials。
-- API 返回配置时必须 mask secret。
-- LLM prompt 只能包含脱敏 sample 和 schema 说明。
-- 后续 TODO：引入字段级加密或 KMS。
-
-### Mapping 治理
-
-- 每次 mapping 变更保留版本。
-- active mapping 只能有一个。
-- 允许保存 draft mapping。
-- 运行记录关联 mapping version。
-- 校验失败不能自动启用。
-
-### 错误处理
-
-常见错误类型：
-
-- API 401/403：凭证错误或过期。
-- API 429：限流，需要调整 schedule。
-- JSON path 不存在：上游结构变化。
-- 类型转换失败：mapping 规则错误。
-- schema validation failed：转换结果不满足目标模型。
-
-每次运行需要记录：
-
-- datasource id。
-- mapping version。
-- started_at / finished_at。
-- fetched count。
-- mapped count。
-- written count。
-- failed count。
-- error summary。
-
----
-
-## 七、测试计划
-
-### Backend Unit Tests
-
-- mapping engine：
-  - path 提取。
-  - 数组展开。
-  - 默认值。
-  - 类型转换。
-  - datetime parse。
-  - 枚举映射。
-  - 缺字段错误。
-- target schema registry：
-  - `vessel_ais` 必填字段校验。
-  - `geo_points` 经纬度范围校验。
-  - `generic_records` 接受未知结构。
-- LLM assistant：
-  - mock provider 返回 mapping。
-  - 验证 secret 不进入 prompt。
-  - 验证非法 mapping 被拒绝。
-
-### Backend Integration Tests
-
-- sample JSON -> propose mapping -> preview -> save mapping。
-- mapped collector 使用保存的 mapping 写入 `generic_records`。
-- `vessel_ais` sample 写入船舶相关目标结构。
-- 上游 JSON 结构变化时，运行失败并记录错误。
-
-### Frontend Tests
-
-- 自定义数据源向导完整流程。
-- 未配置 AI Provider 时，提示去 Settings 配置，但允许手写 mapping。
-- LLM 返回不完整 mapping 时，Preview 阶段显示校验错误。
-- 保存 mapping 后展示 active version 和运行状态。
-
----
-
-## 八、分期工作量
-
-| 阶段 | 内容 | 估算 |
-|-----|------|------|
-| Phase 0 | 完成本规划、确认 schema registry 设计 | 0.5 天 |
-| Phase 1 | target schema registry + mapping template model | 1–2 天 |
-| Phase 2 | deterministic mapping engine | 2–3 天 |
-| Phase 3 | sample/propose/preview/save API | 2–3 天 |
-| Phase 4 | DataSources 自定义源向导 | 3–5 天 |
-| Phase 5 | generic mapped collector + run history | 2–4 天 |
-| Phase 6 | vessel_ais / geo_points destination handler | 2–4 天 |
-
----
-
-## 九、当前差距与下一步
-
-当前差距：
-
-- `datasource_configs` 只描述请求配置，不描述目标 schema 和 mapping。
-- 自定义源没有 sample -> schema -> mapping -> preview -> save 的闭环。
-- 生产采集还没有通用 mapped collector。
-- Settings 与 DataSources 的职责边界需要在 UI 上进一步明确。
-- Earth 还没有通用 `geo_points` 图层。
-
-下一步建议：
-
-1. 先实现 target schema registry 和 mapping engine，不急着接 LLM。
-2. 用固定 sample JSON 做 `vessel_ais` 和 `generic_records` 的单元测试。
-3. 再接 LLM propose API，让 LLM 产出的只是 mapping 草案。
-4. 最后做前端向导，把人工确认和 preview 放到启用之前。

docs/plans/earth-compute-center-bgp-style-plan.md

@@ -1,372 +0,0 @@
-# Earth Compute Center BGP-Style Plan
-
-## Goal
-
-这份文档定义如何按照 BGP 模块的产品方式，把“算力中心”提升为 Earth 上的一级能力。
-
-这里的“按 BGP 方式”指的是：
-
-- 有独立的数据语义和接口入口
-- 有独立的 Earth 图层与图例
-- 有独立的 hover / click / 选中态 / 详情卡
-- 有独立的统计口径与后续专题页扩展空间
-
-这里的“按 BGP 方式”不指：
-
-- 机械复制 BGP 的 anomaly / incident / collector 三层事件模型
-- 为静态算力设施强行引入不必要的复杂告警语义
-
-算力中心本质上更接近“长期基础设施分布层”，不是“高频动态异常层”。
-因此应该复用 BGP 的模块化方法，而不是照搬 BGP 的事件结构。
-
-## Why
-
-当前仓库里已经有算力相关基础：
-
-- 后端已有 `top500` 和 `epoch_ai_gpu` 数据采集
-- 可视化接口已有 `/api/v1/visualization/geo/supercomputers` 和 `/api/v1/visualization/geo/gpu-clusters`
-- Earth 信息卡已对 `supercomputer` 和 `gpu_cluster` 做了基础类型兼容
-
-但当前能力还停留在“数据可取到”的阶段，没有形成像 BGP 那样完整的可视化模块：
-
-- Earth 缺少独立的算力图层加载模块
-- 缺少算力 marker 体系和视觉层级
-- 缺少算力图例、统计、开关和搜索接入
-- 缺少与海缆、BGP、卫星的关系表达
-- 缺少算力专题页和后续告警/研判扩展入口
-
-所以当前真正的缺口不是“有没有数据”，而是“有没有产品级模块”。
-
-## Core Principle
-
-算力中心应当采用和 BGP 一致的模块化分层：
-
-1. 数据层：稳定的数据契约和 GeoJSON 输出
-2. 渲染层：独立的 Earth 图层、marker 和视觉状态管理
-3. 交互层：hover、click、锁定态、详情卡、图例和统计
-4. 扩展层：后续专题页、关系分析、告警和 AI 研判
-
-但语义上必须保持算力中心自身的特点：
-
-- `site / center` 是主对象，不是事件
-- `capacity / rank / vendor / operator / status` 是主信息，不是异常严重度
-- `distribution / concentration / dependency` 是后续分析方向，不是第一阶段必须项
-
-## Recommended Scope
-
-第一版“算力中心”建议统一承载两类对象：
-
-- `supercomputer`
-- `gpu_cluster`
-
-并在 Earth 上收口为一个主题层：`compute_centers`
-
-这样做有几个好处：
-
-- 用户看到的是统一的“算力基础设施”语义，而不是零散数据源
-- 后端仍可保留 `top500` 和 `epoch_ai_gpu` 的来源差异
-- 前端可以在一个图层里再细分两种 marker 语言
-
-## Current Gap
-
-和 BGP 对比，当前差距主要在下面几层。
-
-### 1. Data Contract Gap
-
-现在的算力 GeoJSON 还是通用 `collected_data` 输出思路，字段较轻：
-
-- `gpu_cluster` 只有基础名称和地点
-- `supercomputer` 只暴露一部分性能字段
-- 缺少统一的 `site_type / operator / capacity_band / source / updated_at / confidence`
-- 缺少统一的算力层聚合出口
-
-### 2. Earth Rendering Gap
-
-当前 Earth 里没有类似 `bgp.js` 的算力模块：
-
-- `constants.js` 没有算力 API 路径和视觉配置
-- `main.js` 没有算力加载、拾取、状态同步和 HUD 更新
-- `controls.js` 没有算力图层开关和启动加载优先级
-- `layer-startup-tasks.js` 没有算力启动任务
-- `legend.js` / `ui.js` 没有算力统计与图例模式
-
-### 3. Interaction Gap
-
-虽然 `info-card.js` 支持基础字段，但还没有形成 BGP 那种完整交互链路：
-
-- 没有 hover / selected / dimmed 的视觉状态
-- 没有算力对象专属 tooltip 与摘要文案
-- 没有锁定后与其他基础设施的联动高亮
-- 没有搜索、统计卡和详情组织方式
-
-### 4. Product Expansion Gap
-
-当前还没有“算力中心”专题页与分析语义：
-
-- 没有全球分布/国家聚合/厂商聚合视图
-- 没有算力与海缆/BGP/区域的关系表达
-- 没有 AI brief / assessment 的后续落点
-
-## Architecture Direction
-
-推荐把算力中心做成“BGP 同级能力”，但采用更适合静态基础设施的结构。
-
-### Backend
-
-建议新增统一聚合接口，例如：
-
-- `/api/v1/visualization/geo/compute-centers`
-
-它的职责是把：
-
-- `top500`
-- `epoch_ai_gpu`
-
-统一转换成一个主题层输出，同时保留对象细分类型：
-
-- `site_type: supercomputer | gpu_cluster`
-
-建议统一字段至少包括：
-
-- `id`
-- `name`
-- `site_type`
-- `country`
-- `city`
-- `latitude`
-- `longitude`
-- `operator`
-- `vendor`
-- `capacity_value`
-- `capacity_unit`
-- `capacity_band`
-- `rank`
-- `source`
-- `updated_at`
-- `location_precision`
-- `geography_mode`
-- `is_estimated`
-- `estimated_reason`
-- `metadata`
-
-这里建议优先做“统一聚合出口”，而不是一开始就新增独立数据库表。
-
-原因：
-
-- 当前源数据更新频率低，先复用 `collected_data` 成本更低
-- 可以先把 Earth 产品体验做完整
-- 如果后续要做历史趋势、关系推断、告警，再评估是否拆成独立模型
-
-### Frontend Earth
-
-建议新增独立模块，例如：
-
-- `frontend/public/earth/js/compute-centers.js`
-
-职责参照 `bgp.js`：
-
-- 拉取算力中心 GeoJSON
-- 创建 marker
-- 管理 hover / selected / dimmed 状态
-- 输出图例项
-- 输出统计摘要
-- 提供 overlay 和详情格式化辅助函数
-
-推荐视觉分层：
-
-1. `supercomputer` 用更稳定、更规整的设施型符号
-2. `gpu_cluster` 用更活跃、更现代的密度型符号
-3. 选中态通过 halo / ring / related infrastructure highlight 表达
-
-视觉上应避免把算力中心做成“BGP 事件点”那种高频脉冲风格。
-它应该更像长期存在的高价值设施。
-
-## Phases
-
-## Phase 1: Unified Earth Layer
-
-目标：
-
-- 先把算力中心做成 Earth 上可用、可点、可解释的一级图层
-
-工作项：
-
-- 新增统一算力 GeoJSON 接口
-- 新增 `compute-centers.js`
-- 在 `constants.js` 增加 API 路径和视觉配置
-- 在 `controls.js` 增加算力图层开关与启动元数据
-- 在 `layer-startup-tasks.js` 增加算力启动加载任务
-- 在 `main.js` 接入算力拾取、hover、click、锁定态和 HUD 统计
-- 在 `ui.js` / `legend.js` / `index.html` 增加算力统计与图例入口
-- 在 `info-card.js` 提升算力详情字段组织
-- 对无法精确定位、但可按国家或弱线索推测的大概位置，仍然生成地图点位
-- 这类对象必须带显式“估算位置”状态，例如图标问号角标与详情说明
-
-完成标准：
-
-- Earth 上能独立显示/隐藏算力中心
-- 两类对象有可区分的视觉表达
-- hover / click / 详情卡 / 图例 / 统计全部打通
-- 精确位置与估算位置在图标或文案上可区分，不会误导为同一精度
-- 不干扰现有海缆、卫星、BGP 的交互链路
-
-## Phase 2: Relationship Layer
-
-目标：
-
-- 让算力中心不只是“点”，而是和其他基础设施产生上下文关系
-
-工作项：
-
-- 建立算力中心与国家/区域聚合摘要
-- 增加与附近海缆登陆点的关系提示
-- 增加与 BGP 事件/观测范围的空间邻近提示
-- 增加与卫星覆盖或区域连通性的实验性提示
-
-完成标准：
-
-- 点击算力中心时，用户能看到“它和哪些基础设施相关”
-- 信息表达以辅助判断为主，不做夸张推断
-
-## Phase 3: Compute Center Observatory
-
-目标：
-
-- 把算力中心从 Earth 图层扩展成独立专题观测能力
-
-工作项：
-
-- 新增算力中心专题页
-- 提供国家/厂商/类型/容量分布统计
-- 支持列表、筛选、详情和历史快照
-- 预留 AI brief / assessment 入口
-
-完成标准：
-
-- 算力中心不再只是 Earth 上的视觉点位
-- 能作为独立业务上下文进入日常观察与研判
-
-## Phase 4: Alerts And Assessment
-
-目标：
-
-- 在不滥造“假动态告警”的前提下，引入真正有价值的变化感知
-
-候选方向：
-
-- 新增大规模算力中心
-- 既有中心容量显著变化
-- 国家/区域集中度显著变化
-- 高价值中心与关键网络基础设施关系变化
-
-完成标准：
-
-- 告警来自可解释的结构变化
-- 不把静态数据硬做成噪声式实时事件流
-
-## Implementation Notes
-
-建议按下面顺序推进：
-
-1. 先统一 GeoJSON 契约
-2. 再做 Earth 独立模块和图层开关
-3. 再补详情卡、图例和统计
-4. 最后才做关系层和专题页
-
-这样可以避免一开始把范围摊得过大。
-
-## Unknown Location Strategy
-
-由于部分算力数据源不会直接提供经纬度，未知位置补全不能只依赖“继续找 API 字段”。
-更稳妥的方式是做成一条分层富化链路，而不是单一猜测规则。
-
-推荐按下面优先级推进：
-
-1. 直接源信息
-
-- 源记录显式给出 `latitude / longitude`
-- 源记录给出 `city / region / facility / campus / operator`
-- 源页面详情、内嵌 JSON、结构化元数据、新闻稿链接里能抽出地点线索
-
-2. 名称与机构归一化
-
-- 建立 `canonical_name / aliases / operator / facility` 归一化表
-- 把 `cluster name`、`operator`、`campus name` 归一到同一个实体
-- 优先解决同一对象多写法导致的命中失败，而不是先扩大猜测范围
-
-3. 本地位置注册表
-
-- 用仓库内可维护的 registry 保存高价值对象的位置知识
-- 每条记录至少包含：`canonical_name`、`aliases`、`operator`、`country`、`region`、`city`、`lat`、`lon`、`confidence`、`source_note`
-- 转换层优先读取 registry，避免地点知识长期散落在转换代码里
-
-4. 分层回退定位
-
-- `precise`
-- `estimated_site`
-- `estimated_city`
-- `estimated_region`
-- `estimated_national_hub`
-- `estimated_country`
-
-这里建议把“国家内主要算力城市”作为国家质心之前的一层。
-例如没有美国精确位置时，优先考虑已知的主要算力/数据中心城市候选，而不是直接落在几何质心。
-
-5. 候选证据富化
-
-- 如果源 API 无地点信息，可以允许采集链路读取公开辅助证据
-- 例如机构官网、数据中心介绍页、新闻稿、百科型页面、公开 PDF
-- 但只提取“地点线索”，不把外部页面上的经纬度当真值直接写回
-
-6. 人工校验闭环
-
-- 对高价值且仍然未知的对象输出待核验清单
-- 把人工确认结果回写到位置注册表
-- 后续采集继续优先复用这层人工确认结果
-
-### Additional Solution Paths
-
-除了静态映射表，还可以考虑下面这些办法：
-
-- 基于国家和运营方建立“主要园区候选集”，用稳定散列把同国未知节点分散到若干可信城市，而不是全部压到一个点
-- 基于数据中心/云厂商公开 region 列表建立 `operator -> city set` 候选映射，用于云 GPU 集群类对象
-- 把“估算依据”结构化，例如 `matched_alias`、`matched_operator`、`matched_city_text`、`fallback_country_hub`
-- 给位置补全增加 `last_verified_at`，便于后续按时间重新校验老旧映射
-- 单独维护“不可可靠定位”状态；这类对象仍可在国家级聚合统计中出现，但可以允许用户在地图上过滤掉
-- 后续如果你们愿意投入更多，可把这条链路做成小型 enrichment pipeline，而不是仅在 API 转换时临时判断
-
-## Non-Goals
-
-第一阶段不建议做这些内容：
-
-- 不复制 BGP 巡航模式到算力中心
-- 不先做复杂实时 websocket 推送
-- 不先引入独立 `compute_center_incident` 一类模型
-- 不先做全量 AI 分析面板
-
-原因是算力中心的第一需求是“被看清楚”，不是“被实时播报”。
-但“被看清楚”不等于“只显示精确坐标对象”。
-对于没有精确经纬度、但能推测到国家或区域级位置的算力中心，应优先以上图并标注估算状态的方式处理，而不是直接在地图上消失。
-
-## Acceptance Checklist
-
-- 后端存在统一的算力中心 GeoJSON 出口
-- Earth 有独立算力图层模块，而不是散落在 `main.js`
-- 页面上有清晰的算力开关、图例和统计
-- `supercomputer` 和 `gpu_cluster` 在视觉和详情上都可区分
-- 估算位置对象在地图和详情中都有明确状态提示
-- 现有 BGP / 海缆 / 卫星功能无回归
-- 代码结构上为后续专题页和关系分析留出了明确扩展点
-
-## Summary
-
-这项工作的本质不是“再多画几个点”。
-
-它应该把算力中心从已有数据源，升级成与 BGP 同级的 Earth 观测主题：
-
-- 有独立语义
-- 有独立图层
-- 有独立交互
-- 有后续分析扩展能力
-
-推荐先完成 Phase 1，把算力中心做成真正可用的 Earth 一级模块，再继续推进关系层和专题页。

docs/plans/earth-interactable-layer-plan.md

@@ -1,313 +0,0 @@
-# Earth Interactable Layer Plan
-
-## 背景
-
-状态：Phase 1 已经开始落地，Phase 2 的 BGP 事件 / 观测站迁移和 Phase 3 的算力中心迁移也已完成。`frontend/public/earth/js/interactable.js` 已新增，AIS 船只、BGP 事件、BGP 观测站和算力中心图层已经改为通过 `createInteractableLayer()` 使用通用批量 `Points`、hover / locked overlay、默认 glow、状态更新、asset icon 预加载、屏幕空间 picking、固定 / 距离缩放和跨 Interactable 同坐标避让。登陆点因 `THREE.Points` 边缘深度裁切和贴地层级要求，已退回专用 `THREE.Sprite` 黄色球路径，并与海缆同高度同 renderOrder。后续阶段聚焦把可复用的扩圈 / 雷达扇形动画正式沉淀成 `animations` 扩展。
-
-当前实现说明和接入示例见：
-
-- [earth-interactable-usage.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)
-
-当前 AIS 船只图层已经形成了一个适合作为基准的交互图标模式：
-
-- 普通态使用批量 `THREE.Points` 渲染，避免每个对象一个 `Sprite` 带来的 draw call 和透明排序压力。
-- hover / locked 态使用单点 overlay 叠加 glow，不改变普通批次，交互反馈清晰且成本低。
-- moving / anchored 船只通过 canvas 点纹理表达不同形状，moving 船只还按航向分桶。
-- 拾取走屏幕空间命中，拖动和惯性期间跳过高频 hover picking。
-- 图层高度贴近地表，仅保留很小的深度余量，避免“浮在表面层”的观感。
-
-这个模式不应该只服务船只。后续 BGP 事件、BGP 观测站、算力中心、新闻事件、告警、地面传感器等都可能需要“图标类可交互元素”。如果每个图层继续各写一套 icon、glow、hover、locked、动画、picking 和图例逻辑，视觉会漂移，性能策略也会重复分叉。登陆点已经验证为例外：需要完整贴地且不被球面边缘裁切时，专用 Sprite 路径比通用 `Points` 更合适。
-
-目标是把船只图层的成功做法抽象成一个通用接口：业务图层只描述“要画什么、在哪里、怎么交互”，底层统一负责批量渲染、默认 glow、状态 overlay、动画槽位、拾取和生命周期。
-
-## 目标
-
-1. 建立统一的 Earth 交互图标接口，作为未来地表图标类元素的默认入口。
-2. 以 AIS 船只 glow 为默认 glow 视觉，其它图标默认沿用同一套 glow 质感。
-3. 保留图标颜色、状态颜色、hover 放大、locked 强调、dimmed 聚焦、动画扩展等能力。
-4. 支持 canvas / SVG / image icon，不强行要求所有图标都可重着色。
-5. 保持船只当前性能路线：批量绘制普通态，少量 overlay 处理交互态。
-6. 给 BGP 事件扩圈、BGP 观测站雷达扇形等补充动画留出正式扩展点。
-
-## 非目标
-
-- 不在第一阶段重写所有 Earth 图层。
-- 不把卫星、海缆、国家边界、真实地形这类非图标图层纳入同一个接口。
-- 不为了抽象牺牲业务图标的差异表达，例如船只航向、BGP 事件严重级别、观测站雷达扫掠。
-- 不要求图片图标支持运行时重着色；图片图标只能通过预制多状态图片或 overlay tint 做有限表达。
-
-## 核心设计
-
-建议新增一个通用模块，例如：
-
-```text
-frontend/public/earth/js/interactable.js
-```
-
-它导出一个工厂或注册函数：
-
-```js
-createInteractableLayer({
-  id,
-  earth,
-  renderOrder,
-  altitudeOffset,
-  icon,
-  scale,
-  glow,
-  colors,
-  states,
-  animations,
-  picking,
-  data,
-  getPosition,
-  getKind,
-  getRotation,
-  getPayload,
-});
-```
-
-业务模块仍保留自己的数据加载、图例、详情卡字段和业务语义。例如 `vessels.js` 负责 AIS 数据和船型映射，但 icon 渲染、hover overlay、locked overlay、默认 glow 和屏幕空间 picking 可以逐步迁入 `interactable.js`。
-
-## 参数草案
-
-| 参数 | 类型 / 示例 | 默认值 | 说明 |
-| --- | --- | --- | --- |
-| `id` | `"vessels"` | 必填 | 图层唯一标识，用于 debug、picking、legend 和状态缓存。 |
-| `earth` | `THREE.Object3D` | 必填 | 图层挂载目标，通常是 Earth root。 |
-| `renderOrder` | `4.4` | `4` | 普通 icon 批次和 overlay 的基础渲染顺序。 |
-| `altitudeOffset` | `0.2` | `0.2` | 图层高度，语义为 `CONFIG.earthRadius + altitudeOffset`。地表图标默认贴近真实地形基础层。 |
-| `icon` | `{ type, source, draw, size, bins }` | 必填 | 图标来源。支持 canvas draw、SVG URL、image URL、内置 shape。 |
-| `icon.fitSize` | `60` 或 `{ width: 60, height: 60 }` | `atlasCellSize` | asset 图标在 atlas canvas 内的最大绘制尺寸，默认居中等比 contain。SVG / 图片文件只负责原始形状，不需要为了显示大小手写 transform。 |
-| `scale` | `{ base, min, max }` | `{ base: 1 }` | 基础缩放和距离稳定范围。当前船只可映射到 `VESSEL_POINT_SIZE` / `baseScale`。 |
-| `sizeMode` | `"fixed" / "distance"` | `"fixed"` | 是否固定屏幕像素尺寸；非 fixed 时按相机到地表距离做比例缩放。 |
-| `sizeScale` | `{ min, max, referenceFov }` | `{ min: 0.12, max: 3, referenceFov: 75 }` | `sizeMode !== "fixed"` 时的缩放限制和参考视角。 |
-| `glow.enabled` | `true / false` | `true` | 是否启用默认 glow。默认 glow 以船只 hover / locked overlay 为基准。 |
-| `glow.intensity` | `0.0 - 2.0` | `1` | glow 强度，内部映射到 canvas `shadowBlur`、opacity 或 shader uniform。 |
-| `glow.colorMode` | `"state" / "icon" / "fixed"` | `"state"` | glow 颜色来源，默认跟随状态颜色。 |
-| `hover.scale` | `1.0 - 2.0` | `1.18` | hover 放大倍率。当前船只保持同尺寸 glow overlay，接口仍保留放大能力供其它图层使用。 |
-| `hover.mode` | `"scale" / "glow-only" / "custom"` | `"scale"` | hover 反馈方式。船只可用 `"glow-only"`，其它图标默认放大。 |
-| `colors.normal` | `"#4A90D9"` | icon 原色 | 普通态颜色。只有可上色 icon 生效。 |
-| `colors.hover` | `"#7dd3fc"` | normal | hover 态颜色。 |
-| `colors.locked` | `"#ffffff"` | hover | locked 态颜色。 |
-| `colors.dimmed` | `"#9B9B9B"` | normal | 聚焦其它对象时的弱化颜色。 |
-| `colors.byKind` | `{ cargo: "#4A90D9" }` | `{}` | 按业务类型着色，如船型、BGP 严重级别。 |
-| `colorable` | `true / false` | 由 icon 类型推断 | canvas shape 和 SVG mask 通常可上色；图片默认不可上色。 |
-| `opacity` | `{ normal, hover, locked, dimmed }` | 船只当前值 | 各状态透明度。 |
-| `rotation` | `{ enabled, bins, getAngle }` | disabled | 是否按角度分桶，例如船只按 COG 分 32 桶。 |
-| `animations` | `IconAnimationSpec[]` | `[]` | 补充动画列表，例如扩圈、雷达扇形、脉冲、轨迹尾迹。 |
-| `picking.radiusPx` | `22` | `20` | 屏幕空间命中半径。 |
-| `picking.throttleMs` | `100` | `80` | hover picking 节流。 |
-| `picking.skipWhileDragging` | `true` | `true` | 拖动和惯性期间跳过 hover picking。 |
-| `zIndexPolicy` | `"surface-icon"` | `"surface-icon"` | 预设层级策略，避免每个业务图层手写高度和 renderOrder。 |
-| `avoidance.enabled` | `true / false` | `true` | 是否参与跨 Interactable 的同坐标避让。默认开启，同一经纬度下的图标会沿地表切平面小幅排开，方便辨认和选择。 |
-| `avoidance.radius` | `number` | `1.1` | 同坐标避让的第一圈半径，单位为地球本地坐标单位。 |
-| `avoidance.precision` | `number` | `4` | 经纬度归并精度，默认约等于只处理几乎完全重叠的图标。 |
-| `legend` | `{ label, color, shape }[]` | `[]` | 可选图例声明，业务层也可以继续自己导出。 |
-| `metadata` | object | `{}` | 业务扩展数据，不参与渲染但参与 tooltip / info-card / search。 |
-
-## Icon 规格
-
-图标输入建议分三类：
-
-```js
-{
-  type: "canvas-shape",
-  size: 128,
-  draw(ctx, state) {
-    // draw triangle / dot / custom shape
-  },
-}
-```
-
-```js
-{
-  type: "svg-mask",
-  source: "/earth/assets/icons/bgp-event-dot.svg",
-  colorable: true,
-}
-```
-
-```js
-{
-  type: "image",
-  source: "/earth/assets/icons/vendor-logo.png",
-  colorable: false,
-  stateSources: {
-    hover: "/earth/assets/icons/vendor-logo-hover.png",
-  },
-}
-```
-
-颜色策略：
-
-- `canvas-shape` 默认可上色，适合船只、事件点、雷达站这类符号。
-- `svg-mask` 如果能作为 mask 使用，则可上色；如果是完整多色 SVG，则按图片处理。
-- `image` 默认不可上色；需要状态变化时使用 `stateSources` 或额外 glow / ring。
-
-## 默认 Glow 规范
-
-默认 glow 以当前船只 overlay 为视觉基准：
-
-- 普通态尽量不启用 glow，保持地图干净。
-- hover / locked 态叠加同位置 overlay。
-- glow 颜色默认跟随状态颜色或业务类型颜色。
-- glow blur 应该稳定，不随 camera zoom 夸张膨胀。
-- 允许通过 `glow.intensity` 控制强度，但不要让业务图层各自发明完全不同的光晕语言。
-
-建议内部把 glow 拆成两个层次：
-
-1. `textureGlow`：canvas texture 里的 `shadowBlur`，适合小图标 hover / locked。
-2. `effectGlow`：额外 ring / halo / pulse，适合告警、BGP 事件和锁定强调。
-
-## 状态模型
-
-通用状态至少包含：
-
-| 状态 | 触发 | 默认表现 |
-| --- | --- | --- |
-| `normal` | 普通显示 | 批量 Points，使用 normal 颜色和 opacity。 |
-| `hover` | 指针悬停 | 默认放大并显示 glow；船只可配置为同尺寸 glow-only。 |
-| `locked` | 点击锁定 / 详情打开 | 强 glow、更高 opacity，可选 ring 或 pulse。 |
-| `dimmed` | 聚焦其它对象 | 降低 opacity，保留上下文。 |
-| `hidden` | 图层关闭或过滤 | 不参与绘制和 picking。 |
-| `alert` | 业务告警 | 可叠加动画，不替代 locked 状态。 |
-
-状态更新需要增量化：只在 hover 目标、locked 目标、过滤条件、数据版本或相机距离阈值变化时更新，不在每帧遍历全部 icon 写材质属性。
-
-## 动画扩展
-
-动画不直接塞进 icon 基础参数，而是作为 `animations` 列表注册。每个动画声明自己的 geometry / material / update 策略：
-
-```js
-{
-  type: "expanding-ring",
-  when: ["alert", "locked"],
-  color: "state",
-  radiusPx: [10, 42],
-  durationMs: 1400,
-  opacity: [0.8, 0],
-}
-```
-
-```js
-{
-  type: "radar-sweep",
-  when: ["normal", "hover", "locked"],
-  angleDeg: 72,
-  rotationMs: 2600,
-  opacity: 0.36,
-}
-```
-
-首批建议内置动画：
-
-| 动画 | 用例 | 说明 |
-| --- | --- | --- |
-| `pulse-ring` | locked、告警点 | 原地呼吸环，强调选中对象。 |
-| `expanding-ring` | BGP 事件 | 向外扩散的事件波纹。 |
-| `radar-sweep` | BGP 观测站 | 扇形扫描，可持续旋转。 |
-| `orbiting-dot` | 数据流 / collector 活跃态 | 小点绕 icon 环绕，表达活动状态。 |
-| `trail` | 移动目标 | 可选短尾迹，船只或飞机类目标使用。 |
-
-动画必须支持批量或分组绘制，避免为每个对象创建独立的高频更新对象。只有 locked / hover / 少量 alert 对象可以使用单对象 overlay。
-
-## 渲染策略
-
-### 普通态
-
-普通态优先使用分桶 `THREE.Points`：
-
-- 按 icon 类型、可上色策略、旋转分桶、纹理 key 分组。
-- 每组一个 `BufferGeometry`，存 `position`、`color`、必要的 `payloadIndex`。
-- `PointsMaterial.sizeAttenuation = false`，保持屏幕尺寸稳定。
-- `depthTest = true`，`depthWrite = false`，避免遮挡关系破坏地表。
-
-### 交互态
-
-hover / locked 使用少量 overlay：
-
-- overlay 复用 `THREE.Points` 单点对象或小型 ring mesh。
-- overlay texture 从统一 cache 获取。
-- overlay 更新只写当前 hover / locked 的 position、texture、opacity、size。
-
-### 高密度升级
-
-当某类图标超过分桶 Points 的舒适区，才考虑升级：
-
-- `InstancedBufferGeometry` billboard。
-- 自定义 shader 支持 per-instance rotation / scale / opacity。
-- 视口 bbox / LOD / cluster。
-
-这个升级不应该改变业务接口，只替换底层 renderer。
-
-## Picking 策略
-
-沿用船只当前方向：
-
-- 默认屏幕空间 picking，而不是 Three.js 对每个 Sprite / Points 做 raycast。
-- 每个 icon 保留世界坐标和业务 payload。
-- 每次 pointer move 将候选点投影到屏幕，按半径和深度判断命中。
-- 拖动、惯性旋转、相机剧烈变化期间跳过 hover picking。
-- click 时允许做一次更精确的 picking。
-
-后续可以按图层或经纬度网格增加空间索引，减少候选点数量。
-
-## 与现有图层的迁移路径
-
-### Phase 1：抽出船只基准能力
-
-- 从 `vessels.js` 提取 texture cache、canvas icon draw、overlay glow、分桶 Points 创建、状态增量更新。
-- 保持 `vessels.js` 的公开 API 不变：`loadVessels()`、`toggleVessels()`、`getVesselMarkers()` 等继续可用。
-- 新模块先只服务船只，确保视觉没有回退。
-
-### Phase 2：迁移 BGP 事件和观测站
-
-- BGP 事件使用 `canvas-shape`，已接入 `Interactable`。
-- 严重级别映射到 `colors.byKind`，并通过通用 `getPointSizeMultiplier` 保留严重级别尺寸倍率。
-- 当前扩圈效果保留在 BGP 业务动画中，并跟随 `Interactable` marker 位置更新。
-- BGP 观测站主图标已接入 `Interactable`，活跃度映射到颜色和 `getPointSizeMultiplier`。
-- BGP 观测站 halo / 覆盖扇形继续由 BGP 业务动画表达扫描，并跟随 `Interactable` marker 位置更新。
-
-### Phase 3：迁移算力中心并评估登陆点
-
-- 算力中心保留现有业务 icon，但接入统一 hover / locked / glow。（已完成）
-- 登陆点曾接入同一套 `Points` 渲染，但 pin 类 SVG 在地球边缘会被深度测试裁切；当前保留专用 `THREE.Sprite`，并使用 canvas 生成黄色扁平球，贴到海缆层级。
-- TODO：登陆点暂不迁移到完整 Interactable。后续若要统一交互接口，优先考虑 Sprite-backed adapter，只对齐 `getMarkers()`、`getPointerIntersections()`、`setMarkerState()`、`updateVisualState()` 等外观协议，不强行复用 `THREE.Points`、atlas 和跨图层避让。
-- 检查图例、搜索和 info-card 是否只依赖业务 payload，而不是依赖渲染对象类型。
-
-### Phase 4：形成 Earth 图标层规范
-
-- 在 `docs/technical/zh/earth-frontend-context.md` 记录当前实现入口。
-- 在 `docs/technical/zh/earth-layer-style-reference.md` 记录默认 glow、状态颜色、默认高度和动画参数。
-- 在 `docs/technical/zh/earth-render-layer-order.md` 记录 surface icon renderOrder 范围。
-
-## 风险与约束
-
-- 过早抽象可能让船只这种高质量基准被平均化，因此第一阶段必须以船只视觉不回退为验收标准。
-- 图片 icon 不可上色，接口需要明确 `colorable = false` 的行为，避免业务层误以为颜色一定生效。
-- 动画如果默认开启过多，会重新引入 overdraw 和每帧更新压力；默认只给 hover / locked 或少量 alert 使用。
-- 地形开启时，贴地 icon 需要在高度、`depthTest`、`polygonOffset` 和 renderOrder 之间保持平衡。
-- 统一 glow 不等于所有图标一模一样；业务可以调强度和颜色，但不应破坏整体视觉语言。
-
-## 验收标准
-
-1. 船只迁入通用接口后，普通态、hover、locked、航向、颜色、轨迹和 picking 行为保持一致。
-2. 新增一个 BGP 事件示例图层配置，不需要复制船只渲染代码即可得到 icon、glow、hover 和扩圈动画。
-3. 新增一个 BGP 观测站示例图层配置，不需要自写独立动画循环即可得到雷达扇形。
-4. 关闭图层后对应 icon、overlay、动画和 picking 全部停止。
-5. 高密度数据下普通态仍走批量绘制，hover / locked 只更新少量 overlay。
-6. 文档同步说明默认高度、默认 glow、状态模型和动画扩展点。
-
-## 相关文件
-
-| 文件 | 当前角色 | 未来关系 |
-| --- | --- | --- |
-| `frontend/public/earth/js/vessels.js` | 船只基准实现，包含分桶 Points、hover / locked overlay、默认 glow 形态 | Phase 1 的抽象来源 |
-| `frontend/public/earth/js/constants.js` | 保存船只高度、颜色、透明度、轨迹参数 | 后续可加入通用 surface icon 默认配置 |
-| `frontend/public/earth/js/bgp.js` | BGP 事件和观测站视觉逻辑 | BGP 事件和观测站主图标已接入 Interactable；扩圈、halo 和覆盖扇形仍保留业务动画 |
-| `frontend/public/earth/js/compute-centers.js` | 算力中心 icon 和交互 | 已通过 Interactable 接入统一 Points、overlay、glow 和 picking |
-| `frontend/public/earth/js/cables.js` | 登陆点 icon 和海缆线 | 登陆点当前使用专用 `THREE.Sprite` 黄色球，不再走 Interactable；海缆线仍独立渲染 |
-| `frontend/public/earth/js/main.js` | 当前集中处理 hover、click、locked 和 info-card 入口 | 后续需要接入通用 icon picking 结果 |
-| `docs/technical/zh/earth-layer-style-reference.md` | 当前视觉参数参考 | 实现后同步默认 glow 和通用参数 |
-| `docs/technical/zh/earth-render-layer-order.md` | 当前层级参考 | 实现后同步 surface icon 层级范围 |

docs/plans/earth-mobile-drawer-ui-plan.md

@@ -1,400 +0,0 @@
-# Earth Mobile Drawer UI Plan
-
-## 背景
-
-当前 Earth 移动端已经补上了基础触控能力，例如：
-
-- 单指拖拽旋转地球
-- 双指缩放
-- 点击阈值和基础事件隔离
-
-但移动端 UI 仍然存在一个根本问题：
-
-它还在沿用桌面 HUD 的内容切分方式，只是把原来的 panel、modal、toolbar 改位置、改层级、改容器。这样虽然能快速复用旧代码，但手机端体验仍然是生硬的，因为：
-
-- 信息密度和结构是按桌面设计的
-- 面板标题、关闭、折叠、开关项是桌面心智，不是手机心智
-- 很多内容只是“被塞进抽屉”，而不是为抽屉重新设计
-- 设置里仍然带有“显示/隐藏某些 panel”的思路，但移动端本来就不应该存在那些独立 panel
-
-因此本计划进一步收紧：
-
-移动端不只是“底部抽屉化”，而是**重新设计一套 fit 抽屉体系的 mobile-first UI**。
-
-## 新目标
-
-1. 手机端不再使用现有 `toolbar` 作为主入口。
-2. 手机端不再使用现有独立 `panel / modal / sheet` 作为直接 UI 单元。
-3. 手机端统一采用“底部抽屉 + 顶部标题 + tab 切换 + 卡片内容”的单前景模式。
-4. 抽屉内部每个 tab 页面都按移动端重新设计内容结构，而不是直接复用旧 panel 结构。
-5. 设置页移除“显示/隐藏 panel”的桌面遗留配置。
-6. 媒体页拆成两个移动端页面：`新闻` 与 `TV`，都归入抽屉体系。
-7. 桌面端保持现有 HUD 体系，不回退。
-
-## 核心原则
-
-### 1. 只复用数据和状态，不复用桌面 UI 结构
-
-可复用：
-
-- 图层注册表
-- 搜索结果数据
-- BGP / 海缆 / 卫星详情数据
-- 媒体数据
-- 旋转、缩放、选择、高亮等运行时状态
-
-不直接复用：
-
-- 桌面 panel DOM 结构
-- 桌面 panel header / close / collapse 交互
-- 桌面 settings 项里的“显示某 panel”逻辑
-- 桌面媒体面板布局
-
-### 2. 抽屉是唯一主前景层
-
-移动端同一时刻只有一个主前景层：底部抽屉。
-
-抽屉内部切换内容页，而不是多个悬浮层互相覆盖。
-
-### 3. 每个 tab 都是移动端页面，而不是 panel 容器
-
-抽屉中的每一项都应视为一个移动端子页面：
-
-- 有自己的标题
-- 有自己的内容层次
-- 有自己的滚动区域
-- 有自己的主操作
-
-而不是简单挂一个旧面板进去。
-
-### 4. 移动端状态提示不占据屏幕正中
-
-桌面端当前很多通知、状态提示、胶囊消息更适合在屏幕上方居中出现，但移动端不应继续沿用这套布局。
-
-移动端统一改为：
-
-- 通知栏放在右上角安全区
-- 胶囊提示放在右上角堆叠
-- 不遮挡地球中心视野
-- 不与底部抽屉主交互区冲突
-
-## 交互模型
-
-### 默认态
-
-移动端默认只显示：
-
-- 地球主画布
-- 底部半露出的抽屉头部
-
-不再单独显示上箭头按钮。
-
-### 展开态
-
-用户从底边直接上拉抽屉，或点击抽屉头部展开。
-
-展开后显示：
-
-- 当前页面标题
-- tab 导航
-- 当前页面内容
-
-### 收起态
-
-用户下拉抽屉头部收起，或点击背景收起。
-
-## 信息架构
-
-移动端抽屉内的一级页面重定为：
-
-1. 图层
-2. 搜索
-3. 态势
-4. 新闻
-5. TV
-6. 设置
-7. 详情（按需出现，不固定常驻 tab）
-
-其中 `新闻` 和 `TV` 不再共享同一个移动端媒体面板。
-
-## 页面重设计要求
-
-### 图层页
-
-目标：
-
-- 成为移动端最核心的控制页
-- 强调快速开关，不强调桌面 panel 感
-
-内容建议：
-
-- 顶部摘要：当前已启用图层数量
-- 图层列表卡片
-- 每个图层项只保留：
-  - 图标
-  - 中文名
-  - 英文副标题
-  - 开关
-- 去掉桌面式 header / collapse / close 结构
-
-### 搜索页
-
-目标：
-
-- 成为抽屉中的完整搜索页
-- 避免看起来像桌面 modal 被塞进抽屉
-
-内容建议：
-
-- 顶部搜索输入框
-- 搜索提示文案
-- 结果列表
-- 结果项更适合手指点击
-- 结果点击后：
-  - 聚焦地球对象
-  - 自动切换到详情页
-
-### 态势页
-
-目标：
-
-- 合并原来的 `stats + legend` 思路
-- 成为移动端全局态势页
-
-内容建议：
-
-- 顶部核心统计卡
-  - 海缆数量
-  - 登陆点数量
-  - 卫星数量
-  - BGP 事件数量
-- 当前关注层图例
-- BGP 状态摘要
-- 不再出现独立 legend 面板和独立 stats 面板
-
-### 新闻页
-
-目标：
-
-- 从原媒体面板中拆出单独的移动端新闻页
-
-内容建议：
-
-- 当前区域焦点
-- 新闻源数量
-- 新闻卡片列表
-- 卡片内显示标题、来源、时间、区域
-- 外链操作更清晰
-
-### TV 页
-
-目标：
-
-- 从原媒体面板中拆出单独的移动端 TV 页
-
-内容建议：
-
-- 顶部频道选择
-- 直播状态
-- 当前频道说明
-- 视频播放器区域
-- 刷新和外链按钮
-
-不再保留桌面式“新闻/TV tab 共处一个 panel”的结构。
-
-### 设置页
-
-目标：
-
-- 只保留对移动端仍有意义的系统配置
-
-必须移除：
-
-- 图层控制 panel 显示/隐藏
-- 图例 panel 显示/隐藏
-- 全球态势 panel 显示/隐藏
-- 媒体 panel 显示/隐藏
-
-保留项建议：
-
-- 旋转模式
-- 日夜模式
-- 地球默认大小
-- 地形透明度
-- 系统入口
-
-原因：
-
-移动端已经没有这些独立 panel 了，所以继续保留这些开关会制造错误心智。
-
-### 详情页
-
-目标：
-
-- 成为海缆 / BGP / 卫星对象的统一移动端详情页
-
-内容建议：
-
-- 标题区
-- 类型标签
-- 关键属性列表
-- 相关对象摘要
-- 相关图层或态势提示
-
-行为建议：
-
-- 点击对象后自动切入详情页
-- 搜索结果点击后也切入详情页
-
-## 阶段重定义
-
-### 阶段 2：抽屉壳层
-
-目标：
-
-1. 实现底部抽屉基本壳层。
-2. 支持上拉展开、下拉收起、背景点击关闭。
-3. `mobile` 模式下隐藏旧 toolbar。
-4. `mobile` 模式下不再直接显示旧 panel。
-
-完成标准：
-
-1. 手机端只有地球主视图和抽屉。
-2. 抽屉开合稳定。
-
-### 阶段 3：基础页面重做
-
-目标：
-
-1. 重新设计并实现图层页。
-2. 重新设计并实现搜索页。
-3. 重新设计并实现设置页。
-
-完成标准：
-
-1. 这三个页面不再是旧 panel 原样移植。
-2. 设置页已移除 panel 可见性开关。
-
-### 阶段 4：态势与详情重做
-
-目标：
-
-1. 将 stats 和 legend 合并为新的态势页。
-2. 实现统一详情页。
-3. 对象点击与搜索结果点击都可切入详情页。
-
-完成标准：
-
-1. 不再存在移动端独立 legend / stats 面板。
-2. 详情页成为统一对象信息入口。
-
-### 阶段 5：媒体拆分重做
-
-目标：
-
-1. 将原媒体面板拆成两个移动端页面：新闻页、TV 页。
-2. 分别重做这两个页面的布局。
-3. 保留各自必要操作，但不继续共享桌面 panel 结构。
-
-完成标准：
-
-1. 新闻与 TV 各自成为独立移动端页面。
-2. 不再使用桌面媒体 panel 的 tab 结构作为移动端主体。
-
-### 阶段 6：手感与真机修正
-
-目标：
-
-1. 调整抽屉高度、节奏、手势阈值。
-2. 调整 tab 密度与文字层级。
-3. 优化 iPhone / Android 安全区。
-4. 优化抽屉滚动与地球拖拽边界。
-
-完成标准：
-
-1. 抽屉和地球不会抢手势。
-2. 手机端各页面信息层次清晰。
-3. 真机下无遮挡、无死层、无错误交互心智。
-
-## 技术落点调整
-
-### [frontend/public/earth/index.html](/home/ray/dev/linkong/planet/frontend/public/earth/index.html)
-
-职责：
-
-- 只保留移动端抽屉壳层
-- 为各页面提供新的页面容器
-
-不再把旧 panel 作为最终结构直接塞进抽屉。
-
-### [frontend/public/earth/js/controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js)
-
-职责：
-
-- 管理抽屉开合
-- 管理 tab 切换
-- 管理详情页切入
-- 管理 mobile / desktop 分流
-
-### [frontend/public/earth/js/search.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/search.js)
-
-职责：
-
-- 保留搜索能力和结果逻辑
-- 输出给新的移动端搜索页
-
-### [frontend/public/earth/js/info-card.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/info-card.js)
-
-职责：
-
-- 从桌面 info-card 逻辑中提取可复用的数据层
-- 服务新的移动端详情页
-
-### [frontend/public/earth/js/tv.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/tv.js)
-
-职责：
-
-- 为新的 TV 页面提供数据和状态
-- 不再直接主导移动端媒体 panel 壳层
-
-### [frontend/public/earth/js/news.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/news.js)
-
-职责：
-
-- 为新的新闻页面提供列表和区域焦点数据
-
-### CSS
-
-需要新增真正的移动端页面样式，而不是继续在旧 panel class 上堆条件分支：
-
-- 图层页样式
-- 搜索页样式
-- 态势页样式
-- 新闻页样式
-- TV 页样式
-- 设置页样式
-- 详情页样式
-- 移动端右上角通知 / 胶囊提示样式
-
-## 验收标准
-
-1. `mobile` 模式下不再显示旧 toolbar。
-2. `mobile` 模式下不再把旧 panel 直接作为最终 UI。
-3. 图层、搜索、态势、新闻、TV、设置都是重新设计的移动端页面。
-4. 设置页不再包含移动端无意义的 panel 显示/隐藏项。
-5. 新闻与 TV 已拆分为两个移动端页面。
-6. legend / stats 已整合为态势页。
-7. 详情页成为统一对象详情入口。
-8. 移动端通知栏和胶囊提示已统一放到右上角安全区，而不是屏幕正中。
-
-## 结论
-
-本计划进一步明确：
-
-移动端目标不是“把桌面 HUD 放进抽屉”，而是“以抽屉为载体，重做一套适合手机端的信息页面”。
-
-后续开发必须以此为准：
-
-- 复用数据
-- 重做界面
-- 清除桌面遗留心智

docs/plans/earth-news-cruise-summary-plan.md

@@ -1,366 +0,0 @@
-# Earth 新闻巡航摘要增强计划
-
-## 背景
-
-`Earth` 的新闻巡航模式目前直接消费 `/api/v1/news/earth-feed` 返回的 `items[].summary`。这个字段主要来自 RSS/Atom 的 `description`、`summary` 或 `content`，再经过 HTML 清理与长度截断。
-
-这个实现足够轻量，但在巡航展示里有三个问题：
-
-- 不是所有新闻源都会提供摘要，部分源只返回标题和链接。
-- 聚合源的摘要质量不稳定，可能只是重复标题、来源署名或片段文本。
-- 巡航模式需要更稳定的“态势说明”，否则新闻卡片的 `SUMMARY` 区域会显得空或信息密度不足。
-
-目标不是把所有新闻都交给大模型，而是建立一个分层摘要管线：能用新闻源自带内容时零成本处理，需要增强时优先用本地模型，云端 LLM 只作为可控兜底。
-
-## 当前相关实现
-
-| 文件 | 作用 |
-| --- | --- |
-| `backend/app/services/earth_news.py` | 拉取 RSS/Atom 新闻源、解析标题/摘要、按区域聚合并返回 Earth 新闻 payload |
-| `backend/app/api/v1/news.py` | 暴露 `/api/v1/news/earth-feed` |
-| `frontend/public/earth/js/news.js` | 拉取新闻 payload 并渲染媒体面板新闻列表 |
-| `frontend/public/earth/js/news-cruise-adapter.js` | 将新闻条目映射为巡航事件，并把 `summary` 传给信息卡 |
-| `frontend/public/earth/js/info-card.js` | 展示新闻巡航卡片中的 `SUMMARY` |
-
-当前摘要生成逻辑集中在 `earth_news.py`：
-
-```python
-summary = _extract_item_text(node, "description", "content")
-clean_summary = _truncate(_strip_html(summary), 180)
-```
-
-这意味着后端还没有区分“摘要来自哪里”“质量是否足够”“是否需要异步增强”。
-
-## 总体方案
-
-采用四级摘要来源：
-
-| 优先级 | 来源 | 成本 | 适用情况 | 风险 |
-| --- | --- | --- | --- | --- |
-| 1 | 新闻源自带 summary / description | 低 | RSS/Atom 已提供可读摘要 | 字段可能为空或重复标题 |
-| 2 | 本地规则提取 | 低 | 有正文片段但没有可靠摘要 | 只能抽取，不能真正概括 |
-| 3 | 本地 Gemma/Ollama 摘要 | 中 | 巡航会展示且前两级质量不足 | 本地模型质量与机器性能相关 |
-| 4 | 云端 LLM 兜底 | 高 | 用户手动增强、重点新闻、失败补偿 | 成本与网络依赖 |
-
-推荐默认策略：
-
-```text
-provider summary -> extractive summary -> cached local model summary -> async local model summary -> optional cloud LLM
-```
-
-巡航 UI 永远先展示已有摘要，不等待模型调用。模型摘要在后台补齐，写入缓存后下一轮巡航或刷新时使用。
-
-## 数据结构
-
-后端应把原来的 `summary: str` 升级为可追踪的摘要元信息，同时为了兼容前端保留顶层 `summary` 字段。
-
-建议新增结构：
-
-```json
-{
-  "summary": "短摘要文本",
-  "summary_meta": {
-    "source": "provider",
-    "quality": "good",
-    "generated_at": "2026-04-29T00:00:00Z",
-    "content_hash": "sha256:...",
-    "model": null,
-    "language": "zh-CN"
-  }
-}
-```
-
-字段说明：
-
-| 字段 | 可选值 | 说明 |
-| --- | --- | --- |
-| `source` | `provider` / `extractive` / `local_llm` / `cloud_llm` / `fallback` | 摘要来源 |
-| `quality` | `good` / `partial` / `poor` | 后端对摘要可用性的判断 |
-| `generated_at` | ISO 时间 | 模型或规则生成时间 |
-| `content_hash` | SHA-256 | 用于缓存命中和判断内容变化 |
-| `model` | 字符串或 `null` | 例如 `gemma3:4b` |
-| `language` | 语言代码 | 默认 `zh-CN`，也可跟随新闻语言 |
-
-前端第一阶段不需要显示 `summary_meta`，但可以用于后续调试面板或质量标记。
-
-## 后端设计
-
-### NewsSummaryService
-
-新增 `backend/app/services/news_summary.py`，提供统一入口：
-
-```python
-async def resolve_news_summary(item: ParsedNewsItem, *, mode: str) -> NewsSummaryResult:
-    ...
-```
-
-核心职责：
-
-- 标准化新闻输入：标题、URL、来源、发布时间、摘要片段、正文片段。
-- 判断 provider summary 是否可用。
-- 生成本地规则摘要。
-- 查询模型摘要缓存。
-- 在允许时调用本地 Ollama/Gemma。
-- 在增强模式或手动触发时调用云端 LLM。
-- 返回摘要文本与 `summary_meta`。
-
-### 摘要质量判断
-
-第一版可以用轻量规则：
-
-- 少于 30 个字符：`poor`
-- 与标题高度重复：`partial`
-- 包含明显来源署名或聚合噪声：`partial`
-- 60-180 个字符且不重复标题：`good`
-
-伪代码：
-
-```python
-def score_summary(title: str, summary: str) -> SummaryQuality:
-    if len(summary.strip()) < 30:
-        return "poor"
-    if normalized_overlap(title, summary) > 0.75:
-        return "partial"
-    if looks_like_source_attribution(summary):
-        return "partial"
-    return "good"
-```
-
-### 本地规则摘要
-
-如果新闻源没有摘要，但有 `content`、`description`、`snippet` 或正文片段：
-
-- 清理 HTML。
-- 去掉标题重复内容。
-- 去掉来源署名、发布时间、图片说明。
-- 优先取前 1-2 个完整句子。
-- 控制在 80-140 个中文字符或 40-80 个英文词。
-
-### 本地 Gemma/Ollama Provider
-
-不要在业务里写死 Gemma，抽象为 `LocalLLMSummaryProvider`，默认可以指向 Ollama：
-
-```env
-NEWS_SUMMARY_PROVIDER=ollama
-OLLAMA_BASE_URL=http://localhost:11434
-NEWS_SUMMARY_MODEL=gemma3:4b
-NEWS_SUMMARY_TIMEOUT_SECONDS=20
-NEWS_SUMMARY_MAX_INPUT_CHARS=5000
-NEWS_SUMMARY_MAX_PER_HOUR=60
-```
-
-Ollama 请求示例：
-
-```http
-POST /api/generate
-Content-Type: application/json
-
-{
-  "model": "gemma3:4b",
-  "prompt": "...",
-  "stream": false,
-  "options": {
-    "temperature": 0.2,
-    "num_predict": 180
-  }
-}
-```
-
-摘要 prompt 要强调“只基于原文”，避免模型补事实：
-
-```text
-你是新闻摘要器。只根据输入新闻内容生成摘要，不要添加原文没有的信息。
-输出中文，1-2 句话，80-140 字。
-如果原文信息不足，只概括已知事实，不要推测。
-
-标题：{title}
-来源：{source}
-发布时间：{published_at}
-正文或片段：
-{content}
-```
-
-### 缓存
-
-需要缓存模型摘要，避免重复花时间和费用。
-
-缓存 key：
-
-```text
-sha256(url + title + published_at + normalized_content)
-```
-
-建议新增表或复用系统设置缓存。若要可查询与清理，推荐独立表：
-
-```text
-news_summary_cache
-- id
-- cache_key
-- url
-- title
-- content_hash
-- summary
-- source
-- quality
-- provider
-- model
-- generated_at
-- expires_at
-- failure_count
-- last_error
-```
-
-缓存策略：
-
-- 同一 `cache_key` 命中后直接返回。
-- `provider` / `extractive` 可以短期缓存。
-- `local_llm` / `cloud_llm` 可以长缓存，内容 hash 变化才重算。
-- LLM 失败后记录 `failure_count`，短时间内不重复调用。
-
-## 前端与巡航行为
-
-前端第一阶段只需要继续使用 `item.summary`，不阻塞现有逻辑。
-
-后续可选增强：
-
-- `news.js` 在渲染新闻列表时，如果 `summary_meta.quality === "poor"`，可以用更紧凑的标题卡样式。
-- `news-cruise-adapter.js` 选择巡航项时，可以优先选择 `summary_meta.quality !== "poor"` 的新闻。
-- `info-card.js` 不显示“AI 生成中”这类文案，避免把系统内部状态暴露给用户。
-
-如果后端异步生成完成，可以通过下一次 `/api/v1/news/earth-feed` 刷新自然更新。第一版不需要 WebSocket。
-
-## 调用策略
-
-默认使用“省钱模式”：
-
-- 只处理本次 payload 中即将进入巡航队列的前 N 条。
-- `provider` 或 `extractive` 达到 `good` 时不调用模型。
-- 本地模型失败时不影响新闻 payload。
-- 云端 LLM 默认关闭，只允许手动增强或后台配置开启。
-
-推荐限制：
-
-| 配置 | 默认值 | 说明 |
-| --- | --- | --- |
-| `NEWS_SUMMARY_MODE` | `economy` | `off` / `economy` / `enhanced` / `manual` |
-| `NEWS_SUMMARY_CRUISE_PREFETCH_LIMIT` | `10` | 每次新闻 payload 预热多少条巡航摘要 |
-| `NEWS_SUMMARY_MAX_PER_HOUR` | `60` | 本地模型每小时最多处理数量 |
-| `NEWS_SUMMARY_CLOUD_MAX_PER_DAY` | `20` | 云端 LLM 每天最多处理数量 |
-| `NEWS_SUMMARY_TIMEOUT_SECONDS` | `20` | 单条模型摘要超时 |
-| `NEWS_SUMMARY_MAX_INPUT_CHARS` | `5000` | 输入截断上限 |
-
-## Gemma 本地部署建议
-
-Gemma 适合作为“本地省钱层”，但不应成为强绑定依赖。建议通过 Ollama 接入，未来可切换 Qwen、Llama 或其他本地模型。
-
-开发环境：
-
-```bash
-ollama pull gemma3:4b
-ollama serve
-```
-
-集成原则：
-
-- 后端只依赖 Ollama HTTP API，不直接依赖 Gemma SDK。
-- 模型名称来自配置，不写死在代码里。
-- 健康检查访问 `/api/tags` 或执行一条极短测试 prompt。
-- 如果 Ollama 不可用，摘要管线自动退回 `provider` / `extractive`。
-
-中文新闻较多时，需要单独评估 Gemma 与 Qwen 系本地模型的中文摘要质量。不要只看单条效果，至少抽样 50 条新闻比较：
-
-- 事实准确性
-- 中文自然度
-- 长度稳定性
-- 延迟
-- 是否会补充原文没有的信息
-
-## 云端 LLM 兜底
-
-云端 LLM 不作为默认路径，只用于：
-
-- 用户点击“增强摘要”。
-- 管理员开启增强模式。
-- 本地模型连续失败且新闻进入重点巡航队列。
-
-云端结果同样写入 `news_summary_cache`，并受每日限额控制。
-
-## 分阶段实施
-
-### 第一阶段：零成本摘要质量增强
-
-- 在 `earth_news.py` 中引入 `summary_meta`。
-- 增加 provider summary 质量判断。
-- 增加本地规则摘要兜底。
-- `/api/v1/news/earth-feed` 保持兼容，继续返回顶层 `summary`。
-- 前端无需大改。
-
-验收标准：
-
-- 没有摘要的新闻也能尽量得到短摘要。
-- `summary_meta.source` 和 `summary_meta.quality` 可用于调试。
-- 现有新闻面板和巡航模式不破坏。
-
-### 第二阶段：本地 Gemma/Ollama 摘要
-
-- 新增 `LocalLLMSummaryProvider`。
-- 接入 Ollama `/api/generate`。
-- 添加超时、输入截断、错误退避。
-- 增加模型摘要缓存。
-- 巡航 payload 后台预热前 N 条摘要。
-
-验收标准：
-
-- Ollama 可用时，低质量摘要能被本地模型增强。
-- Ollama 不可用时，新闻接口仍然正常返回。
-- 同一新闻不会重复调用模型。
-
-### 第三阶段：设置与可观测性
-
-- 在设置中增加新闻摘要模式：
-  - `关闭`
-  - `省钱模式`
-  - `增强模式`
-  - `仅手动`
-- 增加本地模型连通性检查。
-- 暴露缓存命中率、模型调用次数、失败次数。
-- 日志记录摘要来源和失败原因。
-
-验收标准：
-
-- 用户可以不改环境变量就知道本地摘要服务是否可用。
-- 管理员能看出成本和失败情况。
-
-### 第四阶段：云端 LLM 兜底
-
-- 接入现有 AI Provider 或新增 cloud summary provider。
-- 增加每日限额与手动增强入口。
-- 对云端生成结果落缓存。
-
-验收标准：
-
-- 云端调用可控、可关闭、可限流。
-- 云端失败不影响巡航。
-
-## 风险与防护
-
-| 风险 | 防护 |
-| --- | --- |
-| 本地模型生成不存在的事实 | prompt 明确禁止扩写；摘要只作为原文概括；保留来源链接 |
-| 本地模型慢导致新闻接口卡住 | 模型摘要异步化；接口先返回已有摘要 |
-| 成本失控 | 默认不启用云端；按小时/天限流；缓存命中优先 |
-| 摘要语言不一致 | 配置目标语言，默认 `zh-CN` |
-| 新闻源正文不足 | 只概括标题和片段，不强行扩写 |
-| 模型服务不可用 | 自动回退，不影响巡航主流程 |
-
-## 推荐优先级
-
-先做第一阶段和第二阶段的最小闭环：
-
-1. `summary_meta` + 质量判断。
-2. 本地规则摘要。
-3. Ollama provider。
-4. 缓存。
-5. 巡航前 N 条异步预热。
-
-云端 LLM 和设置页可以后置。这样能先验证“摘要缺失比例、本地模型质量、实际延迟”三个关键问题，再决定是否投入更重的 UI 与云端增强。

docs/plans/earth-predicted-orbit-plan.md

@@ -96,11 +96,3 @@ GEO 轨道点数高，采样率需要按轨道类型分层。
 2. 解锁后轨道立即清除
 3. 不同轨道类型下点数可控
 4. 页面切换回来不会闪出旧轨道残留
-
-## Satellite Footprint Follow-Up Items
-
-从技术文档迁出的 footprint 后续项，作为卫星覆盖能力的计划 backlog：
-
-1. 为 `iridium-next` 新建独立 footprint adapter。
-2. 在 UI 上补一个只读提示，让用户知道当前卫星是否支持 footprint。
-3. 如果未来拿到 GEO beam contour / operator metadata，再为 GEO 开 operator-specific footprint。

docs/plans/earth-renderer-architecture-separation-plan.md

@@ -25,49 +25,6 @@
 - 状态和渲染更新散落在多个模块
 - 后续再加新图层时容易复制旧逻辑
 
-## Current High-Frequency Risks
-
-### 1. Visual state and business state drift apart
-
-Earth 里最常见的 bug 不是“没渲染”，而是状态没有一起收口：
-
-- 图层关了，tooltip 还在
-- 锁定对象隐藏了，info card 还在
-- legend 没跟图层切换
-- loading 已结束，但按钮还像没开
-
-后续架构治理需要把这类同步责任从临时 UI patch 转为统一状态流。
-
-### 2. HUD layout fixes skip structure analysis
-
-Earth HUD 历史上反复出现：
-
-- 面板只剩一条缝
-- markdown 被裁掉
-- tabs / iframe 被 `overflow: hidden` 吃掉
-
-这类问题应纳入布局治理计划，而不是散落在单个功能改动里临时修。
-
-### 3. Transitional paths keep accumulating
-
-Earth 已经经历过多轮 HUD、toolbar、media panel 重构，容易留下：
-
-- 旧 helper
-- 旧 class
-- 旧 fallback 逻辑
-- 已废弃变体
-
-架构分离阶段需要把 cleanup pass 作为计划项，而不是让技术上下文承担提醒职责。
-
-### 4. Cruise logic and business events couple too deeply
-
-巡航相关风险是通用巡航层继续混入业务事件细节，导致 BGP、新闻、卫星、海缆各自复制一套状态机。
-
-架构目标应保持：
-
-- 通用巡航层管理目标、队列、focus、停留、隐藏和切换
-- 业务模块只提供队列、坐标、卡片内容和高亮副作用
-
 ## Target Architecture
 
 Earth 对每类对象都尽量拆成三层：

docs/plans/earth-vessel-ais-aggregation-plan.md

@@ -1,389 +0,0 @@
-# AIS 多源采集、冲突记录与聚合接口计划
-
-**状态**：v0-v3 已实现，v4+ 规划中
-**创建日期**：2026-04-30  
-**核心原则**：采集器只写原始观测；去重、合并、冲突解释放在聚合接口中完成
-
-## 已确认决策
-
-| 项目 | 决策 |
-|-----|------|
-| AISStream 接入方式 | 单独实现 WebSocket 采集器，不塞进现有 BarentsWatch HTTP collector |
-| 采集器职责 | 连接上游、标准化字段、写入原始观测，不直接决定最终展示值 |
-| 去重合并位置 | 放在聚合服务和聚合 API 中，而不是散落在每个 collector 的保存逻辑里 |
-| 冲突处理 | 先记录冲突事实和当前选择原因，后续再开放用户规则配置 |
-| 默认可信度 | 同类 AIS 数据源优先按 `delivery_mode` 评估：`realtime_stream` 优于 `batch_stream`，再优于 `polling` 和 `snapshot` |
-| 过期保护 | 实时流源断流超过 freshness 窗口后，不能仅凭“实时源”身份压过更新的轮询数据 |
-| 源健康状态 | 聚合时必须参考采集器健康状态，不能只看配置中的理论优先级 |
-| 媒体富化 | 船只图片等媒体信息不进入 AIS 实时聚合主链路，后续单独做 enrichment |
-
-## 背景
-
-当前 AIS 链路以 BarentsWatch 为主。它是 HTTP polling 模式，覆盖挪威附近海域，适合作为稳定的免费起点，但不适合承担全球实时船只数据的全部职责。后续接入 AISStream 后，会出现同一个 MMSI 被多个来源同时上报的情况：
-
-- 位置、航速、航向可能在多个来源之间存在秒级差异。
-- 船名、IMO、呼号、船型、尺寸等静态字段可能不完整，甚至互相冲突。
-- WebSocket 或其他实时流通常更接近实时，但也可能断流或批量延迟。
-- 如果每个 collector 自己做去重合并，规则会分散、不可审计，也很难让用户后续配置“某个字段信任哪个来源”。
-
-因此第一阶段不应让采集器直接覆盖最终船只表。更稳的方式是先保留观测事实，再由聚合接口统一给出当前展示视图。
-
-## 目标架构
-
-```mermaid
-flowchart LR
-    A[BarentsWatch HTTP collector] --> D[AIS raw observations]
-    B[AISStream WebSocket collector] --> D
-    C[Custom mapped vessel_ais sources] --> D
-    D --> E[AIS aggregation service]
-    E --> F[Conflict records]
-    E --> G[GeoJSON vessels API]
-    E --> H[Vessel detail API]
-    I[Aggregation strategy config] --> E
-```
-
-### 原始观测层
-
-原始观测层保存每个来源看到的事实。建议模型包含：
-
-| 字段 | 用途 |
-|-----|------|
-| `target_schema` | 例如 `vessel_ais` |
-| `source` | 例如 `barentswatch_vessels`、`aisstream_vessels` |
-| `entity_key` | AIS 使用 MMSI |
-| `delivery_mode` | `realtime_stream`、`batch_stream`、`polling`、`snapshot` |
-| `transport` | `websocket`、`sse`、`http`、`file` 等 |
-| `observed_at` | 上游数据时间，优先使用 AIS 消息时间 |
-| `collected_at` | 本系统接收或采集时间 |
-| `source_message_id` | 上游消息 ID 或可推导 ID，没有则为空 |
-| `observation_hash` | 幂等去重指纹，用于防止同一来源重复写入同一条观测 |
-| `normalized_payload` | 标准化后的 AIS JSON |
-| `raw_payload` | 可选，保存原始或裁剪后的上游记录 |
-| `quality_flags` | 观测级质量标记，例如 `stale`、`position_jump`、`future_timestamp` |
-
-`delivery_mode` 和 `transport` 不应混为一谈。WebSocket 是传输方式；streaming 是交付模式。聚合可信度主要看 `delivery_mode`，`transport` 只作为辅助信息。
-
-原始观测层需要做存储级幂等去重，但这里的去重不是业务合并。推荐使用 `source + entity_key + message_type + observed_at + payload_hash` 或上游稳定消息 ID 作为唯一约束，避免 WebSocket 重连、HTTP 重试或批量回放导致同一事实重复入库。
-
-### 源健康状态
-
-每个采集器应维护独立的健康状态，供聚合服务读取：
-
-| 字段 | 用途 |
-|-----|------|
-| `source` | 采集器标识 |
-| `connection_state` | `connected`、`reconnecting`、`disconnected`、`disabled` 等 |
-| `last_seen_at` | 最近收到上游消息或响应的时间 |
-| `last_success_at` | 最近成功写入观测的时间 |
-| `last_error` | 最近错误摘要 |
-| `message_rate` | 最近窗口内的消息速率 |
-| `lag_seconds` | 上游观测时间与本系统接收时间的延迟 |
-
-聚合优先级不能只看 `source_priority`。例如 `aisstream_vessels` 默认优先于 `barentswatch_vessels`，但如果它处于 `disconnected` 或 `lag_seconds` 超过 freshness 窗口，则动态字段应回退到更新的可用来源。
-
-### 身份键边界
-
-v1 可以继续用 MMSI 作为 `entity_key`，因为它是 AIS 动态消息里最稳定、最容易获得的主键。但文档和模型都要为后续扩展留出口：MMSI 可能复用、填错或缺少静态信息，后续身份解析应结合 `mmsi + imo + callsign + name + dimensions` 判断是否需要拆分或合并实体。
-
-### 冲突记录层
-
-聚合服务发现同一个实体、同一个字段存在多个非空不同值时，写入冲突记录。冲突记录不代表错误，只代表“有多个可用候选值”。
-
-```json
-{
-  "target_schema": "vessel_ais",
-  "entity_key": "257123000",
-  "field": "name",
-  "candidates": {
-    "barentswatch_vessels": "OSLO TRADER",
-    "aisstream_vessels": "OSLO TRADER II"
-  },
-  "selected_source": "aisstream_vessels",
-  "selected_value": "OSLO TRADER II",
-  "selected_reason": "delivery_mode_priority",
-  "resolved_by": "system",
-  "status": "open"
-}
-```
-
-第一阶段只需要记录冲突和当前选择原因，不需要做人工逐条确认。后续 UI 的目标也不是让用户处理每条冲突，而是把冲突沉淀成字段级规则。
-
-## 聚合规则
-
-### 字段分类
-
-| 类型 | 字段 | 默认策略 |
-|-----|------|----------|
-| 动态位置 | `lat`、`lon`、`sog`、`cog`、`heading`、`nav_status` | 优先最新 `observed_at`，同时间再按来源优先级 |
-| 静态身份 | `name`、`callsign`、`imo`、`flag` | 非空优先，再按字段策略或来源优先级 |
-| 静态规格 | `vessel_type`、`vessel_type_name`、`length`、`width`、`draught` | 非空优先；冲突时记录候选值 |
-| 轨迹点 | `track_points` | 按时间线合并；同一时间窗口内相近点去重；保留点级 `source` |
-| 元信息 | `field_sources`、`conflict_count`、`selected_reasons`、`quality_flags` | 聚合接口生成，便于调试和后续 UI 展示 |
-
-### 默认优先级
-
-默认优先级应使用两个维度：
-
-```yaml
-delivery_mode_priority:
-  - realtime_stream
-  - batch_stream
-  - polling
-  - snapshot
-
-transport_priority:
-  - websocket
-  - sse
-  - http
-  - file
-```
-
-`delivery_mode_priority` 是主判断。比如 AISStream 如果提供实时推送，应标记为 `realtime_stream + websocket`；BarentsWatch 当前是 `polling + http`。
-
-### 断流保护
-
-实时流不能永久凭身份占优。聚合时需要 freshness 窗口：
-
-```yaml
-freshness:
-  realtime_stream_seconds: 900
-  polling_seconds: 3600
-```
-
-如果 `aisstream_vessels` 最近 15 分钟没有该 MMSI 的新观测，而 BarentsWatch 轮询源有更新位置，则位置类字段应采用 BarentsWatch 的更新观测，并记录选择原因 `newest_observation` 或 `freshness_fallback`。
-
-### 异常位置保护
-
-多源 AIS 接入后，聚合服务必须过滤或降权明显异常的位置观测：
-
-- 经纬度必须在合法范围内。
-- `observed_at` 不能明显来自未来。
-- 同一 MMSI 短时间内跨越不合理距离时，标记 `position_jump`，默认不直接采用该点。
-- 当异常点来自当前优先源时，应记录 `selected_reason = anomaly_rejected`，再回退到其他可用来源。
-
-异常保护不应静默丢弃事实。原始观测仍应保留，聚合结果通过 `quality_flags` 和冲突记录解释为什么没有采用它。
-
-### 轨迹聚合
-
-轨迹接口不能简单拼接所有来源，否则前端会出现折返、抖动和重复点。默认规则：
-
-- 以 `observed_at` 排序，生成统一时间线。
-- 同一来源的完全重复点通过 `observation_hash` 去重。
-- 多来源在短时间窗口内上报的相近位置视为同一轨迹点，优先选择 freshness 和 source priority 更高的一条。
-- 每个轨迹点保留 `source`、`selected_reason` 和必要的 `quality_flags`。
-- 对被判定为 `position_jump` 的点，默认不进入展示轨迹，但可通过调试参数查看。
-
-## 聚合接口
-
-现有展示接口应逐步改为消费聚合服务，而不是自己直接拼 `VesselPosition + VesselStatic`。
-
-```text
-GET /api/v1/visualization/geo/vessels
-GET /api/v1/visualization/vessels/{mmsi}
-GET /api/v1/visualization/vessels/{mmsi}/track
-GET /api/v1/visualization/vessels/{mmsi}/conflicts
-```
-
-GeoJSON properties 建议增加：
-
-```json
-{
-  "mmsi": 257123000,
-  "name": "OSLO TRADER",
-  "lat": 59.91,
-  "lon": 10.73,
-  "received_at": "2026-04-30T10:00:00Z",
-  "field_sources": {
-    "name": "aisstream_vessels",
-    "lat": "aisstream_vessels",
-    "lon": "aisstream_vessels",
-    "vessel_type": "barentswatch_vessels"
-  },
-  "selected_reasons": {
-    "name": "delivery_mode_priority",
-    "lat": "newest_observation",
-    "vessel_type": "non_empty_priority"
-  },
-  "quality_flags": [],
-  "conflict_count": 2
-}
-```
-
-## 开放配置计划
-
-### Phase 1 — 内置默认策略和只读解释
-
-- 实现后端默认策略。
-- 聚合接口返回 `field_sources`、`selected_reasons`、`conflict_count`。
-- 冲突记录可查询，但不允许用户修改。
-- 保持现有前端船只图层接口形状基本兼容，新增字段只作为调试和后续 UI 输入。
-
-### Phase 2 — 系统设置中的 JSON/YAML 策略配置
-
-新增系统设置项，例如：
-
-```yaml
-collector_aggregation:
-  vessel_ais:
-    source_priority:
-      - aisstream_vessels
-      - barentswatch_vessels
-    field_rules:
-      name:
-        mode: source_priority
-      vessel_type:
-        mode: source_priority
-        source_priority:
-          - barentswatch_vessels
-          - aisstream_vessels
-      lat:
-        mode: newest
-      lon:
-        mode: newest
-```
-
-配置校验要求：
-
-- 未知 source 只警告，不阻断保存，便于先配置后启用。
-- 未知 field 必须拒绝，避免拼写错误悄悄失效。
-- 动态位置字段默认不允许被固定来源永久锁死，除非显式开启高级选项。
-- 空值不覆盖非空值是全局保护，不建议开放关闭。
-
-### Phase 3 — 冲突治理 UI
-
-基于冲突记录提供页面或 drawer：
-
-- 查看某个 MMSI 的冲突字段。
-- 查看每个字段的候选来源和值。
-- 查看当前选择原因。
-- 将一次人工选择保存成字段规则，而不是只处理单条冲突。
-- 支持恢复默认策略。
-
-## AISStream 采集器计划
-
-AISStream 采集器单独实现，建议命名为 `aisstream_vessels`。它的职责是：
-
-- 维护 WebSocket 连接、订阅范围和重连。
-- 将上游 AIS 消息标准化为 `vessel_ais` payload。
-- 标记 `delivery_mode = realtime_stream`，`transport = websocket`。
-- 写入原始观测层。
-- 不直接 upsert 最终展示数据。
-
-配置应放入采集器设置，而不是硬编码：
-
-```yaml
-aisstream_vessels:
-  api_key: "${AISSTREAM_API_KEY}"
-  bounding_boxes:
-    - [[-180, -90], [180, 90]]
-  message_types:
-    - PositionReport
-    - ShipStaticData
-```
-
-默认不建议直接订阅全球范围。AISStream 采集器应支持以下订阅策略：
-
-- 使用配置的固定 `bounding_boxes`。
-- 后续支持按 Earth 当前视口或关注区域动态调整订阅范围。
-- 支持限制 `message_types`，避免静态信息、位置报告和扩展消息全量涌入。
-- 断线后使用指数退避重连，并把连接状态写入源健康状态。
-- 重连后可能收到重复或回放消息，因此必须依赖原始观测层的幂等去重。
-
-### 媒体富化边界
-
-VesselFinder 等服务里的船只图片不属于 AIS 实时数据本身。图片、船籍详情、公司信息等后续应作为独立 enrichment 链路：
-
-- 通过 MMSI、IMO、船名等字段异步查询。
-- 使用独立缓存和授权配置。
-- 不阻塞 `vessel_ais` 实时观测入库。
-- 聚合接口只暴露已经缓存好的媒体引用，不在请求链路中现场抓取。
-
-## 版本拆分
-
-计划按 5 个版本推进：
-
-### v0 — 聚合基础设施（已实现）
-
-目标是不改变前端展示行为，先把数据底座铺好。
-
-1. 新增原始观测模型、冲突记录模型和源健康状态模型。
-2. 为现有 BarentsWatch collector 写入原始观测，同时保留现有 `vessel_position` / `vessel_static` 兼容写入。
-3. 实现存储级 `observation_hash` 幂等去重。
-4. 补基础管理命令或调试接口，用于查看某个 MMSI 的原始观测和冲突候选。
-
-### v1 — 聚合读接口（已实现）
-
-目标是让展示接口开始消费聚合结果，但前端形状保持兼容。
-
-1. 实现 AIS 聚合服务，先兼容读取现有表，再逐步切换到原始观测层。
-2. 将 `/geo/vessels` 和 `/vessels/{mmsi}` 改为走聚合服务。
-3. 将 `/vessels/{mmsi}/track` 改为走轨迹聚合逻辑。
-4. 返回 `field_sources`、`selected_reasons`、`quality_flags`、`conflict_count`。
-5. 加入 freshness fallback 和异常位置保护。
-
-### v2 — AISStream WebSocket collector（已实现）
-
-目标是接入第二个真实 AIS 来源，并验证多源冲突和回退逻辑。
-
-1. 实现 `aisstream_vessels` collector。
-2. 支持 API key、订阅范围、消息类型、重连和限流配置。
-3. 将 AISStream 写入原始观测层，不直接 upsert 最终展示表。
-4. 接入源健康状态和 message rate 统计。
-5. 提供 AISStream API Key 获取教程、设置页入口和连接验证支持。
-6. 为重复消息、断流回退、WS 优先级写集成测试。
-
-### v3 — AISStream 可用性与配置体验（已实现）
-
-目标是让 AISStream 从“能采集”变成日常可观察、可调试、可配置的数据源。
-
-1. 设置页展示 AISStream 运行状态：连接状态、最近收到、最近成功、本轮消息数、延迟和最近错误。
-2. AISStream 设置页提供常用采集范围 preset，并保留自定义 Bounding Boxes JSON。
-3. 聚合结果返回 `source_summary`，展示每艘船的来源、观测数量、最新观测时间、传输模式和消息类型。
-4. 保留 `field_sources` 和 `selected_reasons`，用于解释动态字段来自实时流、静态字段来自可用非空来源。
-5. 船名标准化会读取 AISStream `MetaData.ShipName`；船型展示会从 `vessel_type_name` 和 AIS 数字 `vessel_type` 共同归一化，保证 marker 颜色、详情卡、hover 和搜索结果一致。
-6. `/geo/vessels` 不再默认限制 5000 艘；不传 `limit` 或传 `limit=0` 表示全量返回，前端默认也不再二次裁剪到 5000。
-
-### v4 — 策略配置
-
-目标是开放系统级配置，但仍以安全默认值兜底。
-
-1. 接入系统设置中的聚合策略配置。
-2. 支持 source priority、字段级规则、freshness 窗口和高级保护开关。
-3. 保存配置时校验未知字段、非法模式和危险动态字段锁定。
-4. 聚合接口返回当前命中的配置版本，方便排查。
-
-### v5 — 船舶资料 enrichment 与冲突治理
-
-目标是把 AIS 实时流里不稳定或低频出现的静态信息，补成可缓存、可审计的船舶资料层，同时把冲突解释变成可操作能力。
-
-1. 做冲突治理 UI。
-2. 支持把人工选择沉淀成字段级规则。
-3. 支持恢复默认策略。
-4. 设计 `vessel_profile_enrichment`，按 `mmsi + imo + name + callsign` 异步补充船名、船型细分、AIS 大类、旗国、尺寸、建造年份、运营方等静态资料。
-5. 设计 `vessel_media_enrichment`，异步补充船只图片和外部详情缓存。
-6. enrichment 结果必须带 `source`、`fetched_at`、`expires_at`、`confidence` 和原始引用，不覆盖 AIS 原始观测。
-7. 聚合接口只读取已缓存 enrichment；请求链路不现场抓取第三方页面，避免慢请求和授权风险。
-8. 前端船只详情面板展示已缓存资料和媒体，并标注字段来源，不阻塞 AIS 实时链路。
-
-## 测试计划
-
-- 同一来源同一 `mmsi + observed_at + lat + lon` 重复记录只聚合一次。
-- 多来源同一 MMSI 的位置字段优先选择最新观测。
-- 实时流和轮询源同时间冲突时，实时流优先。
-- 实时流过期后，更新的轮询源可以接管动态字段。
-- 实时流源健康状态异常时，动态字段可以回退到更新的可用来源。
-- 静态字段不会被空值覆盖。
-- 静态字段冲突会写入冲突记录。
-- 明显异常位置不会进入默认展示轨迹，并会留下 `quality_flags`。
-- 同一时间窗口内多来源相近轨迹点只展示一个点。
-- AISStream 重连或回放导致的重复消息不会重复进入聚合结果。
-- 字段级配置可以覆盖默认来源优先级。
-- 聚合接口在没有冲突表时仍可返回兼容 GeoJSON。
-
-## 相关文件
-
-- [实时船只监控系统计划](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-tracking-plan.md)
-- [自定义 API 数据源与 LLM 映射系统计划](/home/ray/dev/linkong/planet/docs/plans/datasource-custom-api-mapping-plan.md)
-- [BarentsWatch AIS collector](/home/ray/dev/linkong/planet/backend/app/services/collectors/vessel_ais.py)
-- [船只模型](/home/ray/dev/linkong/planet/backend/app/models/vessel.py)
-- [可视化 API](/home/ray/dev/linkong/planet/backend/app/api/v1/visualization.py)

docs/plans/earth-vessel-rendering-performance-plan.md

@@ -1,249 +0,0 @@
-# Earth Vessel Rendering Performance Plan
-
-## 当前状态
-
-该计划的前端核心部分已经在 `0.44.1` 落地，但最终实现不是原文设想的 `InstancedBufferGeometry` quad，而是更稳的分桶 `THREE.Points` 方案：
-
-- 普通船只按 moving / anchored 和 `VESSEL_COURSE_BINS` 航向分桶，使用 `PointsMaterial` 批量绘制。
-- 航行船只仍是带方向的三角形，停泊或低速船只仍是圆点。
-- hover / locked 不再放大成世界尺寸 Sprite，而是在原点位叠加同尺寸单点 glow overlay。
-- picking 改为屏幕空间命中，拖动和惯性期间跳过 hover picking。
-- 普通态关闭 glow，交互态才显示 glow，降低 overdraw 并让默认地图更干净。
-
-后续如果需要全球 AIS 或更高船只密度，再评估是否从分桶 `Points` 升级到真正 instanced quad 或视口 bbox / LOD。
-
-## 背景
-
-Earth 船只图层已经形成了一套较好的视觉语言：
-
-- 航行船只使用三角形标记
-- 标记按航向旋转
-- 停泊或低速船只使用圆点
-- 不同船型使用不同颜色
-- hover / locked 状态有 glow、透明度和聚焦反馈
-- 标记带有轻微 glow / soft edge，和 Earth HUD 的观感一致
-
-当前性能问题不应通过降级成无方向、无船型语义的普通小点来解决。目标是在保留现有观赏性的前提下，把底层从“每艘船一个 Sprite 对象”优化为批量绘制和轻量交互。
-
-## 当前问题判断
-
-卫星图层能承载几万个对象，是因为它主要走 `THREE.Points` / `BufferGeometry` / instanced trail 路径。船只图层目前每艘船创建一个 `THREE.Sprite` 和独立 `SpriteMaterial`，这会带来：
-
-- draw call 随船只数量增长
-- 透明 sprite 排序和 overdraw 成本上升
-- 每帧遍历所有船只更新 opacity / scale / visible
-- pointer move 时对船只 sprite 做对象级 raycast
-- hover reset 时全量遍历 marker
-
-因此，即使免费 BarentsWatch AIS 只开放挪威周边数据，前端仍可能因为对象级 sprite、raycast 和每帧全量更新出现地球拖动卡顿。
-
-## 目标
-
-1. 保留当前船只标记的视觉质量。
-2. 保留 hover tooltip、点击详情、lock、轨迹等交互。
-3. 显著降低 draw call、每帧 JS 遍历和 pointer picking 成本。
-4. 为后续全球 AIS 或更高船只数量预留扩展空间。
-
-## 非目标
-
-- 不把船只降级为普通无方向 `Points`。
-- 不取消船型颜色、航向三角和停泊圆点。
-- 不为了短期性能直接删除 hover / click 交互。
-
-## Phase 1：交互路径止血
-
-这一阶段不改视觉，只减少 pointer move 和 hover 状态开销。
-
-### 1. 拖动和惯性期间跳过船只 picking
-
-地球拖动时用户主要关注视角变化，不需要每个 pointer move 都命中船只。
-
-处理方式：
-
-- `isDragging === true` 时跳过船只 hover picking。
-- 惯性旋转期间也跳过船只 hover picking。
-- 拖动结束后再恢复 hover 检测。
-
-### 2. vessel hover picking 节流
-
-对船只 hover 命中增加节流，例如 `80ms ~ 120ms` 一次。鼠标高速移动时复用上一次 hover 状态，不在每个 pointer event 上都做 raycast。
-
-### 3. hover reset 从全量遍历改为增量更新
-
-当前 `resetTransientVesselStates()` 会遍历所有船只。改为记录：
-
-- `hoveredVessel`
-- `lockedObject`
-
-当 hover 目标变化时，只更新旧 hover 和新 hover。
-
-### 4. 点击路径只在 click 时做一次精确 picking
-
-点击仍保留精确命中，但只在 click 事件里执行，不参与拖动和高频 pointer move。
-
-## Phase 2：每帧更新减负
-
-这一阶段仍保留 `Sprite` 外观，但减少每帧对全部 marker 的写操作。
-
-### 1. `updateVesselVisualState()` 增量化
-
-当前每帧都会遍历船只并写：
-
-- `marker.material.opacity`
-- `marker.scale`
-- `marker.visible`
-
-优化方向：
-
-- 图层关闭时直接 return。
-- 没有船只时直接 return。
-- 只有以下状态变化时才更新 marker：
-  - show/hide 变化
-  - hover 变化
-  - locked 变化
-  - camera zoom / distance scale 变化超过阈值
-  - focus dim 状态变化
-
-### 2. 缓存 distance scale
-
-`getDistanceScale(camera)` 可以按 camera distance 或 zoom 阈值缓存。缩放没有明显变化时，不必每帧重设所有船只 scale。
-
-### 3. 降低透明 overdraw
-
-在不破坏视觉的前提下微调：
-
-- marker 基础尺寸
-- glow blur 半径
-- 最大 size stabilization
-
-目标是减少屏幕空间重叠面积，而不是改变符号设计。
-
-## Phase 3：保留视觉的批量渲染（已落地为分桶 Points）
-
-原设想是把每艘船的视觉从 `THREE.Sprite` 迁移为 instanced sprite batch。实际落地时选择了更稳的分桶 `THREE.Points`：
-
-- 不依赖自定义 shader。
-- 不依赖 `Points` 自带 raycaster。
-- 用 canvas 纹理保留三角、圆点、船型颜色和航向。
-- 用 hover / locked 单点 overlay 保留交互 glow。
-
-如果未来全球 AIS 导致分桶 `Points` 仍不够，再升级到 instanced quad。
-
-### 1. 原候选方案：instanced quad
-
-每艘船仍然显示为带贴图/软边的 billboard，但底层使用：
-
-- `THREE.InstancedBufferGeometry`
-- 每类船只一个或少量 material
-- per-instance attributes
-
-可按形状和船型拆 batch：
-
-- moving cargo
-- moving tanker
-- moving passenger
-- moving fishing
-- moving military
-- moving other
-- anchored / slow dot
-
-这样 draw call 从“每艘船一个”变为“每类船只一个”。
-
-### 2. 原候选方案：per-instance attributes
-
-每个 instance 存：
-
-- position
-- color
-- rotation
-- scale
-- opacity
-- state
-- mmsi / data index
-
-hover、locked、dimmed 可通过更新少量 instance attribute 实现，不再逐个修改 material。
-
-### 3. 当前落地方案：分桶 `THREE.Points`
-
-当前实现按以下方式复刻视觉：
-
-- moving 船只按 `VESSEL_COURSE_BINS` 做航向分桶。
-- anchored / slow 船只使用圆点分桶。
-- 每个分桶生成一组 `THREE.Points`，共享 `PointsMaterial` 和 canvas 点纹理。
-- `VESSEL_CONFIG.colors` 仍通过 vertex colors 表示船型颜色。
-- hover / locked 在原位置叠加同尺寸单点 overlay，普通态不带 glow，交互态才带 glow。
-
-这样 draw call 从“每艘船一个”变为“每个形状 / 航向分桶一组”，同时避免自定义 shader 的兼容风险。
-
-### 4. 复刻当前视觉
-
-视觉上继续使用当前 canvas texture 或等效 shader：
-
-- moving 使用三角形纹理
-- anchored 使用圆点纹理
-- 保留 soft glow
-- 保留航向 rotation
-- 保留 hover / locked 放大
-
-因此用户看到的效果应与当前船只图层基本一致。
-
-## Phase 4：picking 改造
-
-批量渲染后不再适合对所有 sprite object 做 `raycaster.intersectObjects()`。
-
-### 1. 屏幕空间 picking
-
-参考卫星 picking：
-
-1. 过滤背面船只。
-2. 将候选船只世界坐标投影到屏幕。
-3. 用鼠标位置计算距离。
-4. 取距离最近且小于半径阈值的船只。
-
-### 2. 可选空间索引
-
-如果后续船只数量明显上升，可增加轻量空间索引：
-
-- 经纬度网格 bucket
-- 屏幕空间 bucket
-- viewport bbox 过滤
-
-第一阶段不必引入复杂索引。
-
-## Phase 5：数据层和 LOD
-
-当接入全球 AIS 或船只数量显著增加时，再做数据层优化。
-
-### 1. 请求视口范围
-
-前端请求 `/api/v1/visualization/geo/vessels` 时带上当前视口 `bbox`，减少无关船只。
-
-### 2. 后端排序策略
-
-从单纯 `received_at desc` 改为综合排序：
-
-- 数据新鲜度
-- 船型优先级
-- 当前视口相关性
-- 是否正在航行
-
-### 3. 远景聚合
-
-远景可显示聚合或 top N，近景展开单船。
-
-## 验收指标
-
-1. 船只视觉效果保持当前质量：三角、圆点、颜色、航向、hover、lock 都保留。
-2. 开启船只图层后拖动地球不应明显掉帧。
-3. pointer move 不应因为船只 hover 导致卡顿。
-4. 船只数量达到 `1000` 级别时仍可顺畅旋转地球。
-5. `renderer.info.render.calls` 相比 Sprite 版本显著下降。
-6. hover / click 命中体验不低于当前版本。
-
-## 建议落地顺序
-
-1. 先做 Phase 1，快速恢复地球拖动手感。
-2. 再做 Phase 2，减少每帧 JS 写操作。
-3. Phase 3 和 Phase 4 已按分桶 `THREE.Points` + 屏幕空间 picking 落地。
-4. Phase 5 等全球船只数据或数量压力出现后再推进。
-5. 如果分桶 `THREE.Points` 达到瓶颈，再评估 instanced quad。

docs/plans/earth-vessel-tracking-plan.md

@@ -1,281 +0,0 @@
-# 实时船只监控系统 — 实施计划
-
-**状态**：规划中
-**创建日期**：2026-04-27
-**优先数据源**：BarentsWatch AIS（免费但需要 OAuth client credentials）→ AISHub / MarineTraffic（TODO，付费）
-
-## 已确认决策
-
-| 项目 | 决策 |
-|-----|------|
-| 数据源 | BarentsWatch 先行；AISHub / MarineTraffic TODO |
-| 船只规模 | BarentsWatch 阶段全部显示；全球数据接入后按需加船型过滤（默认 Cargo + Tanker + Passenger） |
-| 更新频率 | 准实时：前端 5 分钟轮询，后端 Collector 每分钟拉取写库 |
-| 历史轨迹 | 保留（`vessel_position` 表保留 24h，后期按需扩展） |
-| 推送方式 | 前端展示仍可先用 HTTP 拉取聚合结果；AISStream 等实时源应单独实现 WebSocket 采集器 |
-
----
-
-## 一、技术背景
-
-船只通过 AIS（自动识别系统）每 2–10 秒广播位置、航速、航向、目的地等信息。全球约 50 万艘持证船只在线，实时数据通过以下方式获取：
-
-| 来源类型 | 典型服务 | 覆盖范围 | 成本 | 状态 |
-|---------|---------|---------|------|------|
-| **BarentsWatch AIS API** | live.ais.barentswatch.no | 挪威海域实时 | 免费，需要 AIS API client credentials | **当前使用** |
-| **AISHub** | aishub.net | 全球实时 | 免费/小额 | TODO：付费接入 |
-| **MarineTraffic API** | marinetraffic.com | 全球实时 | $50–$500/月 | TODO：评估 tier |
-| **VesselFinder API** | vesselfinder.com | 全球实时 | $50–$300/月 | TODO：备选 |
-| **自建 SDR 接收** | RTL-SDR + AIS-catcher | 仅本地 30–50km | 硬件 $30 | 不考虑 |
-| **NOAA 历史数据** | Marine Cadastre | 美国近海历史 | 免费 | 可用于冷启动 |
-
-### BarentsWatch AIS API
-
-- 端点：`https://live.ais.barentswatch.no/v1/latest/combined`
-- 需要在 BarentsWatch developer portal 创建 `AIS - API` client，通过 client credentials 获取 `scope=ais` 的 access token 后请求 AIS endpoint
-- 字段：mmsi, lat, lon, sog, cog, heading, nav_status, name, vessel_type, flag
-- 刷新频率：数据约 30–60s 更新一次，可随意轮询
-
-### TODO：多源 AIS 与实时流接入
-
-- [ ] 接入 AISStream WebSocket 采集器，作为 BarentsWatch 覆盖不足的实时补充
-- [ ] 将 BarentsWatch、AISStream、自定义 `vessel_ais` 映射源统一写入原始观测层
-- [ ] 通过聚合接口做去重、字段合并、冲突记录和默认来源选择
-- [ ] 开放字段级聚合策略配置，让用户决定不同字段优先信任哪个来源
-- [ ] 评估 AISHub 订阅（全球覆盖，约 $30/月），接入全球实时流
-- [ ] 评估 MarineTraffic API tier，对比 AISHub 数据质量与成本
-- [ ] 实现多数据源适配器，通过 `datasource_config` 切换
-- [ ] 真实高频 AIS 稳定接入后，评估将 `vessel_position` 迁移为 TimescaleDB hypertable（保留 Postgres 原生分区作为备选）
-
-多源 AIS 的详细设计见 [AIS 多源采集、冲突记录与聚合接口计划](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-ais-aggregation-plan.md)。
-
----
-
-## 二、实施计划
-
-### Phase 0 — 数据源验证与链路打通（1–2 天）
-
-- 接入 BarentsWatch AIS API，验证 OAuth token、数据格式与字段
-- 构建全球 mock 数据生成器（用于前端渲染压测，补充 BarentsWatch 的地域限制）
-- 确认前端可渲染船只点，整条链路走通
-
-### Phase 1 — 后端基础设施（3–4 天）
-
-#### 1.1 数据库 Schema
-
-```sql
--- 船只静态信息（每 6h 刷新一次）
-CREATE TABLE vessel_static (
-    mmsi          BIGINT PRIMARY KEY,
-    name          VARCHAR(128),
-    callsign      VARCHAR(16),
-    vessel_type   SMALLINT,
-    vessel_type_name VARCHAR(64),
-    flag          VARCHAR(4),         -- ISO 国家码
-    length        FLOAT,
-    width         FLOAT,
-    draught       FLOAT,
-    imo           BIGINT,
-    updated_at    TIMESTAMPTZ
-);
-
--- 船只实时位置（高频写入，保留 24h 轨迹）
-CREATE TABLE vessel_position (
-    id            BIGSERIAL PRIMARY KEY,
-    mmsi          BIGINT NOT NULL,
-    lat           FLOAT NOT NULL,
-    lon           FLOAT NOT NULL,
-    sog           FLOAT,              -- Speed over ground（节）
-    cog           FLOAT,              -- Course over ground（度）
-    heading       SMALLINT,           -- 真北航向
-    nav_status    SMALLINT,           -- 0=航行 1=锚泊 5=停靠 ...
-    received_at   TIMESTAMPTZ NOT NULL DEFAULT NOW()
-);
-
-CREATE INDEX idx_vessel_pos_mmsi_time ON vessel_position(mmsi, received_at DESC);
-CREATE INDEX idx_vessel_pos_time      ON vessel_position(received_at DESC);
-
--- 最新位置物化视图（地图渲染主数据源，避免全表扫描）
-CREATE MATERIALIZED VIEW vessel_latest AS
-SELECT DISTINCT ON (mmsi)
-    vp.*, vs.name, vs.vessel_type_name, vs.flag, vs.length
-FROM vessel_position vp
-LEFT JOIN vessel_static vs USING (mmsi)
-ORDER BY mmsi, received_at DESC;
-
-CREATE UNIQUE INDEX ON vessel_latest(mmsi);
-```
-
-> 后期如需完整历史轨迹查询，迁移 `vessel_position` 到 TimescaleDB 或按天分区。
-
-#### 1.2 Collector：VesselAISCollector
-
-文件：`backend/app/services/collectors/vessel_ais.py`
-
-- 继承 `BaseCollector`，注册到 `collector_registry`
-- 轮询间隔：30–60s（由数据源限速决定）
-- 支持多数据源切换，通过 `datasource_config` 配置 URL + API Key
-- 写入逻辑：upsert `vessel_latest`，append `vessel_position`
-- 接入现有调度系统（`scheduler.py`）
-
-#### 1.3 API 端点
-
-```
-GET /api/v1/visualization/geo/vessels
-    ?bbox=lon_min,lat_min,lon_max,lat_max   # 视口裁剪
-    ?type=cargo,tanker,passenger             # 船型过滤
-    ?limit=0                                 # 可选；不传或 0 表示不裁剪数量
-→ GeoJSON FeatureCollection（Point）
-
-GET /api/v1/visualization/vessels/{mmsi}           # 单船详情
-GET /api/v1/visualization/vessels/{mmsi}/track     # 历史轨迹（默认 6h）
-    ?hours=6
-→ GeoJSON LineString
-```
-
-GeoJSON Feature 格式：
-
-```json
-{
-  "type": "Feature",
-  "geometry": { "type": "Point", "coordinates": [lon, lat] },
-  "properties": {
-    "mmsi": 123456789,
-    "name": "EVER GIVEN",
-    "vessel_type": 70,
-    "vessel_type_name": "Cargo",
-    "flag": "PA",
-    "sog": 12.4,
-    "cog": 247.0,
-    "heading": 245,
-    "nav_status": 0,
-    "length": 400,
-    "received_at": "2026-04-27T10:00:00Z"
-  }
-}
-```
-
-#### 1.4 更新机制
-
-**前端聚合结果拉取 + 后端实时采集**：
-
-- 前端 `setInterval(fetchVessels, 5 * 60 * 1000)` 定期拉取最新快照
-- 后端 BarentsWatch collector 继续以 HTTP polling 方式采集
-- AISStream 等实时源以独立 WebSocket collector 写入原始观测层
-- 展示接口从聚合服务读取当前船只视图，而不是由单个 collector 决定最终展示值
-- 前端默认不再给 `/geo/vessels` 传 `limit=5000`，`VESSEL_CONFIG.maxRenderedMarkers = 0` 表示不做前端数量裁剪；后续如性能不足再引入显式 LOD 上限
-- marker 颜色、详情卡、hover 和搜索结果必须共享 `vessel_type_display` 船型归一化结果，避免 AIS 数字类型码已驱动颜色但卡片仍显示 `Other`
-- 前端是否升级为 WebSocket delta push 是独立优化，不影响后端采集器可以使用 WebSocket 接上游实时源
-
----
-
-### Phase 2 — 前端渲染（3–4 天）
-
-文件：`frontend/public/earth/js/vessels.js`
-
-#### 2.1 渲染方案
-
-参考现有卫星系统（`satellites.js`）的 InstancedMesh 模式：
-
-- `THREE.InstancedMesh`：每个实例 = 一艘船，矩阵包含位置 + 旋转（朝向 COG）
-- 行进船：三角箭头图标，朝向 COG 方向
-- 静止/锚泊船：圆点图标
-- SVG 图标输出到 `frontend/public/earth/assets/icons/vessel-arrow.svg` 和 `vessel-dot.svg`
-
-#### 2.2 船型颜色规范
-
-| 船型 | 颜色 |
-|-----|------|
-| 货轮 Cargo | `#4A90D9` 蓝 |
-| 油轮 Tanker | `#E85D04` 橙红 |
-| 客船 Passenger | `#06D6A0` 绿 |
-| 渔船 Fishing | `#FFD166` 黄 |
-| 军舰 Military | `#73797E` 灰 |
-| 其他 | `#9B9B9B` 浅灰 |
-| 锚泊/停靠 | 降低饱和度 0.4x |
-
-#### 2.3 LOD（相机距离细节层次）
-
-| 相机距离 | 渲染策略 |
-|---------|---------|
-| > 400 | 默认渲染当前接口返回的全部船只；如性能不足，再引入可配置 LOD 上限 |
-| 200–400 | 默认渲染当前接口返回的全部船只；如性能不足，再引入可配置 LOD 上限 |
-| < 200 | 渲染当前视口 bbox 内全部船只 |
-
-前端根据相机位置动态计算 bbox，附加到 API 请求中。
-
-#### 2.4 图层集成
-
-接入现有图层系统，新增"船只"图层项，支持：
-- 图层开/关，状态持久化
-- 子过滤（按船型选择显示哪类，可在图例或设置面板中配置）
-- 与海缆、BGP、卫星层级共存（renderOrder 待定，参考现有层级文档）
-
-#### 2.5 Info Card
-
-复用 `showInfoCard` 机制，点击船只弹出：
-
-```
-EVER GIVEN                              🚢
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-MMSI        123456789
-IMO         9811000
-旗帜        巴拿马 🇵🇦
-船型        散货轮
-当前航速     12.4 kn
-航向        247°
-状态        航行中
-目的地      ROTTERDAM
-━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
-[ 查看轨迹 ]   [ MarineTraffic ↗ ]
-```
-
-#### 2.6 轨迹可视化
-
-点击"查看轨迹" → 请求 `/vessels/{mmsi}/track` → 用 `THREE.CatmullRomCurve3` 渲染插值轨迹线，风格与海缆一致。
-
----
-
-### Phase 3 — 功能完善（2–3 天）
-
-| 功能 | 说明 |
-|-----|------|
-| **船只搜索** | 接入现有搜索面板，按名称 / MMSI 搜索 |
-| **统计 HUD** | 显示当前在线船只数、各类型分布 |
-| **密度热图** | 超低 zoom 时切换为 hex-bin 热力图（避免点云爆炸） |
-| **港口标注** | 加载 WorldPorts 数据集，显示主要港口标记 |
-| **关键水道监控** | 马六甲、霍尔木兹、苏伊士等高亮 + 流量统计 |
-
----
-
-### Phase 4 — 性能与生产化（2–3 天）
-
-- `vessel_position` 按天分区，7 天自动清理
-- TODO：真实数据量达到百万级/日后，将 `vessel_position` 升级为 TimescaleDB hypertable，配置 retention policy 与压缩策略
-- GeoJSON endpoint 用 Redis 缓存 15s
-- 若需 bbox 精确查询，引入 PostGIS `geography` + `ST_DWithin`
-- InstancedMesh + frustum culling，目标 5 万船只 60fps
-
----
-
-## 三、工作量估算
-
-| Phase | 内容 | 估计时间 |
-|-------|-----|---------|
-| Phase 0 | 数据源验证、mock | 1–2 天 |
-| Phase 1 | 后端 Schema + Collector + API | 3–4 天 |
-| Phase 2 | 前端渲染（InstancedMesh + 图层 + Info Card） | 3–4 天 |
-| Phase 3 | 搜索 + 统计 + 轨迹 | 2–3 天 |
-| Phase 4 | 性能优化 + 生产数据源接入 | 2–3 天 |
-| **合计** | | **约 2–3 周** |
-
----
-
-## 四、参考资料
-
-- BarentsWatch AIS API 文档：https://www.barentswatch.no/en/developer/ais-api/
-- MarineTraffic API：https://www.marinetraffic.com/en/ais-api-services
-- AISHub：https://www.aishub.net/api
-- AIS 导航状态码：ITU-R M.1371-5
-- 船型编码（vessel_type）：ITU/IMO AIS Message 5 Type and Cargo
-- WorldPorts 数据集：https://msi.nga.mil/Publications/WPI

docs/plans/enterprise-logging-system-plan.md

@@ -1,793 +0,0 @@
-# Planet 企业级日志系统实施计划
-
-## Goal
-
-把 Planet 当前“能看一点运行输出”的日志能力，升级为一套真正可用、可定位、可纠错、可追责、可演进的企业级日志系统。
-
-这里的“企业级”不是指一上来就接入很重的外部平台，而是指这套系统需要同时满足下面五件事：
-
-1. 排障可用
-2. 历史可查
-3. 业务可解释
-4. 权限操作可追责
-5. 出错后能够反向定位到请求、任务、模块和操作者
-
-最终目标不是“把更多 stdout 放到日志页里”，而是建立一套统一的日志契约与落地链路：
-
-- 统一日志字段
-- 统一事件命名
-- 统一采集入口
-- 统一查询视图
-- 清晰的实时日志、持久化事件、审计日志分层
-
-## Why
-
-当前仓库已经有一些日志基础，但离真正可用的日志系统还有明显距离。
-
-已有基础：
-
-- 后端运行日志可通过 `/tmp/planet_backend.log` 查看
-- 前端开发服务日志可通过 `/tmp/planet_frontend.log` 查看
-- AI Provider 可从 Docker 容器读取日志
-- Earth 浏览器端关键日志可上报到后端并进入 Redis 缓冲
-- 已有 `system_logs` / `audit_logs` 持久化能力
-- 管理台已有“系统日志”页面，支持来源、级别、日期、搜索
-
-当前缺口：
-
-- 后端日志仍以 `uvicorn` / 文本输出为主，不是统一结构化事件流
-- 不同模块的日志格式不一致，很多地方只有 message，没有 event 语义
-- 还没有统一的后端 logger 封装与字段注入机制
-- 前端虽然能上报错误，但还没有统一 logger API 和统一事件词汇
-- Earth 与管理台之间的错误事件还没有形成可串联的事件链路
-- 历史持久化还偏点状，很多高价值失败并没有系统性落库
-- 系统日志页当前更像“运行输出查看器”，不是“多层日志查询台”
-- 审计日志与运行日志尚未形成明确的产品级联动
-
-所以当前真正的问题不是“有没有日志页”，而是：
-
-**当前系统能看见输出，但还不能稳定回答“发生了什么、影响了谁、在哪条链路上坏了、是否已修复、是谁触发的”。**
-
-## Current State
-
-截至 2026-04-23，当前代码中的日志相关能力大致如下。
-
-### 1. 日志来源
-
-当前系统日志页主要读取以下来源：
-
-- `backend`
-  读取 `/tmp/planet_backend.log`
-- `frontend`
-  读取 `/tmp/planet_frontend.log`
-- `ai-provider`
-  读取 Docker 容器日志
-- `earth-client`
-  读取 Redis 缓冲的浏览器端日志
-
-这些来源定义在：
-
-- [backend/app/services/system_logs.py](/home/ray/dev/linkong/planet/backend/app/services/system_logs.py)
-
-### 2. 当前日志读取模型
-
-当前 `read_log_snapshot()` 的职责是：
-
-- 读取某个来源的最近若干行
-- 解析基础级别与时间
-- 按级别、日期、搜索进行过滤
-- 返回用于日志页展示的快照
-
-这个模型适合“运维查看器”，但不适合企业级日志系统，原因是：
-
-- 读取基于文本尾部扫描，不是基于事件模型
-- 不同来源的结构粒度完全不同
-- 过滤依赖文本解析，准确率有限
-- 没有请求、任务、用户、资源、动作等核心关联字段
-
-### 3. 已有持久化能力
-
-当前已经存在两个持久化入口：
-
-- `record_system_log(...)`
-- `record_audit_log(...)`
-
-位置：
-
-- [backend/app/services/persistent_logs.py](/home/ray/dev/linkong/planet/backend/app/services/persistent_logs.py)
-
-这说明系统并不是从 0 开始，但也说明当前最大的问题是：
-
-**持久化能力存在，但没有成为统一默认路径。**
-
-### 4. 已有 request_id 基础
-
-当前系统已具备 `request_id` 相关基础，部分持久化能力也会尝试写入 `request_id`。
-
-这为后续做：
-
-- 请求链路排障
-- 前后端关联查询
-- 任务执行追踪
-
-提供了很好的基础。
-
-### 5. 当前日志页定位
-
-当前日志页已经具备：
-
-- 来源切换
-- 级别筛选
-- 日期筛选
-- 搜索
-- 文本控制台视图
-
-但它仍然是“单层视图”：
-
-- 上面是筛选器
-- 下面是一块文本控制台
-
-它还不是：
-
-- 运行日志 + 事件日志 + 审计日志 的统一入口
-- 也没有事件详情、关联跳转、纠错建议、链路追踪能力
-
-## Core Principles
-
-这套日志系统后续必须遵循下面几个原则。
-
-### 1. 分层，而不是混存
-
-日志必须拆成三层：
-
-1. 运行日志
-2. 持久化事件日志
-3. 审计日志
-
-它们的用途不同，绝不能继续混成一个概念。
-
-#### 运行日志
-
-用于：
-
-- 实时排障
-- 观察服务运行状态
-- 看 stdout / stderr / exception / collector 输出
-
-特点：
-
-- 数据量大
-- 时效性强
-- 保留周期短
-- 不要求每条都落库
-
-#### 持久化事件日志
-
-用于：
-
-- 记录高价值错误
-- 记录关键业务失败
-- 支撑历史追溯
-- 支撑趋势分析
-
-特点：
-
-- 只持久化有价值事件
-- 必须结构化
-- 必须有统一 event 命名
-
-#### 审计日志
-
-用于：
-
-- 留痕
-- 追责
-- 还原高权限操作
-
-特点：
-
-- 必须单独建模
-- 不与普通运行日志混用
-
-### 2. 结构化优先
-
-正式日志必须可拆字段，不能长期依赖自由文本。
-
-最低要求至少能拿到：
-
-- `timestamp`
-- `level`
-- `service`
-- `module`
-- `event`
-- `message`
-- `request_id`
-- `trace_id`
-- `user_id` / `actor`
-- `context`
-
-### 3. 事件命名优先于 message 命名
-
-人看的 message 可以变化，但机器查询和跨模块关联必须依赖稳定事件名。
-
-例如：
-
-- `collector.run.started`
-- `collector.run.completed`
-- `collector.run.failed`
-- `earth.layer.load_failed`
-- `earth.cruise.route_build_failed`
-- `system.restart_task.failed`
-- `auth.websocket.invalid_token`
-
-### 4. 查询链路必须可串联
-
-企业级日志系统的核心不是“有很多日志”，而是“能串起来”。
-
-最终一条高价值事件，至少要能回链到下面任意几类对象：
-
-- 某个请求
-- 某个任务
-- 某个用户
-- 某个数据源
-- 某个 Earth 模块
-- 某个管理动作
-
-### 5. 默认脱敏
-
-日志体系必须明确禁止记录：
-
-- token
-- password
-- Authorization header
-- cookie
-- session
-- 明文敏感个人信息
-
-并且需要有统一脱敏器，而不是靠调用者自觉。
-
-### 6. “可纠错”不是一句口号
-
-这里的“可纠错”至少包含三层：
-
-1. 日志字段足够解释错误，方便人排查
-2. 系统能识别常见错误模式并给出纠偏建议
-3. 关键错误支持闭环动作，例如重试、重建索引、重新触发采集、跳转到对应对象
-
-也就是说，这套日志系统最终不只是“告诉你出错了”，而要尽量接近“告诉你为什么出错、怎么修、去哪修”。
-
-## Non-Goals
-
-第一阶段不追求：
-
-- 全量接入 ELK / Loki / Datadog / OpenTelemetry 全家桶
-- 做分布式 trace 全链路可视化大屏
-- 把所有历史日志都迁进数据库
-- 先做特别复杂的规则引擎
-
-第一阶段追求的是：
-
-- 在当前仓库和当前部署方式下，先把基础日志体系做正确
-- 再为后续平台化接入预留好接口
-
-## Target Architecture
-
-推荐目标架构如下。
-
-### Layer 1: Runtime Logs
-
-职责：
-
-- 承载后端、前端开发服务、容器输出、浏览器端缓冲事件
-- 提供最近窗口内的实时查看能力
-
-来源：
-
-- 文件
-- Docker
-- Redis 缓冲
-- 后续可扩展到 stdout collector
-
-接口：
-
-- `GET /api/v1/system/logs/sources`
-- `GET /api/v1/system/logs/{source_id}`
-
-这层继续保留，但需要做结构化增强和来源补强。
-
-### Layer 2: Persistent System Events
-
-职责：
-
-- 只存高价值事件
-- 供历史追溯、事件列表、趋势和纠错使用
-
-数据来源：
-
-- 后端关键异常
-- 浏览器端关键失败
-- 采集器/调度器关键失败
-- 业务关键告警与降级事件
-
-接口建议：
-
-- `GET /api/v1/system/events`
-- `GET /api/v1/system/events/{id}`
-- `POST /api/v1/system/events/{id}/actions/...`（后续）
-
-### Layer 3: Audit Logs
-
-职责：
-
-- 留痕高权限操作
-- 记录操作者、对象、结果、请求号
-
-接口建议：
-
-- `GET /api/v1/system/audit-logs`
-
-### Layer 4: Error Intelligence / Triage
-
-职责：
-
-- 对高频错误做归类
-- 对已知错误给出解释与建议动作
-- 对相同错误进行 fingerprint 聚合
-
-这是“可纠错”能力的关键层。
-
-建议字段：
-
-- `fingerprint`
-- `root_cause_type`
-- `known_fix_hint`
-- `runbook_url`
-- `related_resource_type`
-- `related_resource_id`
-
-## Canonical Event Model
-
-推荐统一事件字段模型如下。
-
-### Runtime Log Record
-
-```json
-{
-  "timestamp": "2026-04-23T10:15:30Z",
-  "level": "error",
-  "service": "backend",
-  "module": "app.services.scheduler",
-  "event": "collector.run.failed",
-  "message": "Collector bgp_news failed",
-  "request_id": "req_xxx",
-  "trace_id": "trace_xxx",
-  "user_id": null,
-  "actor": null,
-  "resource_type": "collector",
-  "resource_id": "bgp_news",
-  "context": {
-    "datasource_id": 12,
-    "exception_type": "TimeoutError"
-  }
-}
-```
-
-### Persistent System Event
-
-```json
-{
-  "id": 1024,
-  "event": "earth.layer.load_failed",
-  "level": "error",
-  "source": "earth-client",
-  "service": "earth",
-  "module": "cables",
-  "message": "Failed to load cable layer",
-  "fingerprint": "earth.layer.load_failed:cables:network_timeout",
-  "request_id": "req_xxx",
-  "trace_id": null,
-  "user_id": 1,
-  "resource_type": "earth_layer",
-  "resource_id": "cables",
-  "category": "visualization",
-  "status": "open",
-  "context": {
-    "url": "/api/v1/visualization/geo/cables"
-  },
-  "created_at": "2026-04-23T10:15:30Z"
-}
-```
-
-### Audit Log
-
-```json
-{
-  "id": 88,
-  "action": "system.restart_task.requested",
-  "actor_id": 1,
-  "actor_name": "root",
-  "target_type": "restart_task",
-  "target_id": "restart_20260423_xxx",
-  "result": "success",
-  "request_id": "req_xxx",
-  "ip": "127.0.0.1",
-  "details": {
-    "action": "restart_backend"
-  },
-  "created_at": "2026-04-23T10:15:30Z"
-}
-```
-
-## Implementation Plan
-
-## Phase 0: Logging Inventory And Naming Freeze
-
-目标：
-
-- 先统一“记录什么”和“怎么命名”，避免后面越做越乱
-
-工作项：
-
-- 盘点当前所有 `logging.getLogger` 使用点
-- 盘点裸 `print`
-- 盘点 `record_system_log` / `record_audit_log` 已落点位
-- 建立统一事件命名表
-- 定义 service / module / category / resource 字段枚举
-- 输出日志字段白名单和脱敏规范
-
-完成标准：
-
-- 有一份稳定的事件命名清单
-- 有一份字段规范清单
-- 后续新增日志不再“临时起名”
-
-## Phase 1: Backend Structured Logging Foundation
-
-目标：
-
-- 把后端从“散落 logging + 文本输出”升级成“统一结构化 logger”
-
-工作项：
-
-- 新增统一后端 logger helper，例如 `app/core/logging.py`
-- 自动注入：
-  - `service`
-  - `module`
-  - `request_id`
-  - `trace_id`
-- 增加统一脱敏 filter
-- 把关键模块先切到统一 logger：
-  - API 层
-  - scheduler
-  - collectors
-  - websocket
-  - visualization
-  - system control
-- 约束：
-  - 正式路径禁止裸 `print`
-  - 正式异常优先 `logger.exception(..., extra={...})`
-
-完成标准：
-
-- 后端关键模块都有稳定 `event`
-- request 日志和异常日志能挂上 `request_id`
-- 不再依赖只看 `uvicorn` 原生文本输出来定位问题
-
-## Phase 2: Persistent Event Layer
-
-目标：
-
-- 把“值得长期保留的错误和关键事件”系统性落库
-
-工作项：
-
-- 重新定义 `record_system_log()` 的使用边界
-- 明确哪些事件必须持久化：
-  - API 关键失败
-  - 调度器失败
-  - 采集器失败
-  - Earth 客户端关键错误
-  - 数据源不可用
-  - 业务降级与恢复
-- 补齐字段：
-  - `event`
-  - `resource_type`
-  - `resource_id`
-  - `category`
-  - `fingerprint`
-  - `status`
-- 增加高频错误去重/聚合策略
-
-完成标准：
-
-- 高价值错误不再只存在于运行日志里
-- 能查询最近一周/一月的关键失败事件
-- 相同错误具备聚合基础
-
-## Phase 3: Frontend And Earth Unified Logger
-
-目标：
-
-- 把前端从“点状 error 上报”升级成统一前端事件流
-
-工作项：
-
-- 在前端新增统一 logger API
-- 统一方法：
-  - `debug`
-  - `info`
-  - `warn`
-  - `error`
-- 统一字段：
-  - `page`
-  - `module`
-  - `event`
-  - `message`
-  - `url`
-  - `user_agent`
-  - `context`
-- Earth 模块优先接入：
-  - layer load failed
-  - cruise build failed
-  - popup render failed
-  - connector render failed
-  - websocket dropped
-- 管理台优先接入：
-  - settings save failed
-  - datasource toggle failed
-  - restart task submit failed
-
-完成标准：
-
-- 前端日志事件名与后端可对齐
-- Earth 和管理台关键失败不再只停留在 console
-- 浏览器端关键问题能进入统一系统日志/事件层
-
-## Phase 4: Audit Logging Completion
-
-目标：
-
-- 把管理员与高权限操作真正做成企业级审计
-
-工作项：
-
-- 扩大审计覆盖面：
-  - 系统重启
-  - 数据源启停
-  - 调度规则变更
-  - 配置变更
-  - 人工触发采集
-  - 删除/修改关键配置
-- 增加字段：
-  - actor
-  - target
-  - before / after
-  - request_id
-  - IP
-- 审计页支持：
-  - 动作筛选
-  - 操作者筛选
-  - 时间筛选
-  - 目标对象筛选
-
-完成标准：
-
-- 所有高权限操作都能追到人、时间、对象、结果
-
-## Phase 5: Log Console To Enterprise Observability UI
-
-目标：
-
-- 把当前“系统日志”页升级为真正的多层日志工作台
-
-工作项：
-
-- 将页面拆为三个主视图：
-  1. 运行日志
-  2. 关键事件
-  3. 审计日志
-- 运行日志视图：
-  - 保留大控制台
-  - 支持来源、级别、日期、搜索
-- 关键事件视图：
-  - 列表化展示高价值事件
-  - 支持聚合、状态、指纹、对象筛选
-- 审计视图：
-  - 列表化展示管理员动作
-- 增加详情抽屉：
-  - 原始 message
-  - context
-  - request_id
-  - related resource
-  - recommended action
-
-完成标准：
-
-- 日志页不再只是“终端文本窗口”
-- 运维排障、历史追溯、审计留痕三者分层清晰
-
-## Phase 6: Corrective Intelligence
-
-目标：
-
-- 让系统从“能看日志”进化到“能辅助修错”
-
-工作项：
-
-- 引入错误 fingerprint
-- 对已知错误配置：
-  - 根因类型
-  - 修复建议
-  - runbook 链接
-  - 推荐动作
-- 支持常见纠错动作：
-  - 重试采集任务
-  - 重载配置
-  - 跳转到对应模块/资源
-  - 打开相关日志过滤视图
-- 高频错误支持聚合与静默窗口
-
-完成标准：
-
-- 已知错误能给出明确建议
-- 运维不需要每次都从零猜
-
-## Recommended Module Changes
-
-### Backend
-
-建议新增/增强的模块：
-
-- `backend/app/core/logging.py`
-  - 统一 logger 封装
-  - formatter
-  - filter
-  - request/trace 注入
-- `backend/app/services/persistent_logs.py`
-  - 扩展字段
-  - 统一持久化策略
-- `backend/app/services/system_logs.py`
-  - 逐步从“文本尾部查看器”升级为“运行日志聚合器”
-- `backend/app/services/log_classification.py`
-  - 指纹
-  - 根因分类
-  - 纠错建议
-- `backend/app/api/v1/system_control.py`
-  - 补充事件 / 审计 / 日志多视图接口
-
-### Frontend
-
-建议新增/增强：
-
-- `frontend/src/lib/logger.ts`
-  - 统一前端 logger API
-- `frontend/src/pages/Logs/Logs.tsx`
-  - 升级为多层工作台
-- `frontend/public/earth/js/...`
-  - 各 Earth 模块接入统一事件 logger
-
-## Event Naming Convention
-
-建议采用：
-
-`<domain>.<resource>.<action>.<result>`
-
-示例：
-
-- `collector.datasource.run.started`
-- `collector.datasource.run.failed`
-- `earth.layer.cables.load.failed`
-- `earth.cruise.route.build.failed`
-- `system.restart_task.requested`
-- `system.restart_task.completed`
-- `auth.websocket.connect.failed`
-- `settings.datasource.priority.updated`
-
-规则：
-
-- 不用自然语言句子
-- 不把 ID 塞进 event 名里
-- 资源对象通过字段承载，不通过 event 名承载
-
-## Query Model
-
-最终推荐支持的查询维度：
-
-- 时间范围
-- level
-- source
-- service
-- module
-- event
-- request_id
-- trace_id
-- user_id / actor
-- resource_type / resource_id
-- category
-- fingerprint
-- status
-- full-text search
-
-## Retention Strategy
-
-推荐保留策略：
-
-- 运行日志：
-  - 文件 / 容器 / Redis 缓冲保留短周期
-- 持久化事件：
-  - 保留中长期
-- 审计日志：
-  - 长期保留
-
-初版可以先这样：
-
-- 运行日志：7 到 14 天
-- 关键事件：90 到 180 天
-- 审计日志：180 天以上
-
-后续再根据存储与合规要求调整。
-
-## Security And Compliance
-
-必须落实：
-
-- 敏感字段脱敏
-- 前端上报白名单
-- 防止日志注入
-- 审计日志不可被普通管理员随意篡改
-- 高敏感纠错动作必须再次鉴权
-
-## Success Criteria
-
-当下面这些条件成立时，才算这套日志系统真的“成了”：
-
-1. 一个后端请求失败时，能通过 `request_id` 在运行日志、持久化事件、审计日志之间串联查询
-2. 一个 Earth 前端错误能定位到页面、模块、事件名和上下文
-3. 一个采集器失败能同时看到运行日志、持久化事件和可执行纠错动作
-4. 一个管理员操作能查到操作者、目标对象、结果和 request_id
-5. 日志页不再只是文本控制台，而是完整的“运行日志 / 关键事件 / 审计日志”工作台
-6. 高频已知错误能聚合并给出修复建议
-
-## Delivery Order
-
-推荐严格按下面顺序做，不要乱跳：
-
-1. Phase 0 命名与字段规范冻结
-2. Phase 1 后端结构化 logging 基础
-3. Phase 2 高价值事件持久化
-4. Phase 3 前端 / Earth 统一 logger
-5. Phase 4 审计覆盖补齐
-6. Phase 5 日志工作台 UI 重构
-7. Phase 6 指纹 / 纠错 / runbook
-
-原因：
-
-- 如果不先统一字段和命名，后面 UI 和持久化会越来越乱
-- 如果不先做后端结构化基础，前端上报再多也串不起来
-- 如果不先补持久化层，就只有“实时可看”，没有“历史可查”
-
-## First Actionable Milestone
-
-如果要从明天就开始做，最合理的第一个里程碑是：
-
-### M1: 让后端关键路径全部拥有统一结构化事件
-
-范围：
-
-- API 请求入口/出口
-- scheduler
-- collectors
-- websocket
-- visualization
-- system control
-
-交付物：
-
-- 统一 logger helper
-- 统一 event naming 表
-- 统一 request_id 注入
-- 统一脱敏策略
-- 关键模块替换完成
-
-完成这个里程碑后，Planet 才算真正拥有了“企业级日志系统的地基”。
-

docs/plans/frontend-markdown-renderer-plan.md

@@ -1,97 +0,0 @@
-# Markdown 渲染器完善计划
-
-## 背景
-
-Planet 控制台当前有三类主要 Markdown 使用场景：
-
-- 文档中心：技术文档、计划文档、运行手册。
-- AI Playground：模型回复、分析结果、代码片段。
-- BGP 简报：由系统生成并保存的态势报告。
-
-这些场景都复用 `frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx`。因此 Markdown 能力应该集中在共享渲染器内完成，页面只负责传入内容、链接转换和布局约束，不能让每篇文档或每个页面手写复制按钮、表格样式、列表样式等交互细节。
-
-## 目标
-
-建设一个稳定、可复用、适合技术文档和 AI 输出的 Markdown 渲染器，优先覆盖常用语法、代码块操作和清晰的阅读样式，并为后续语法高亮、锚点导航、内容安全策略留出接口。
-
-## 成功标准
-
-- 代码块支持 fenced language、语言标签、复制按钮、复制成功状态和横向滚动。
-- 常用块语法稳定渲染：标题 1-6、段落、引用、分割线、表格、无序列表、有序列表、任务列表。
-- 常用行内语法稳定渲染：链接、自动链接、图片、行内代码、粗体、斜体、删除线。
-- 文档中心、AI Playground、BGP 简报继续复用同一个组件，不出现页面级重复实现。
-- 样式在普通业务面板和文档中心都有合理表现，文档中心可以通过 `.docs-markdown` 覆盖主题变量。
-- 前端 TypeScript build 通过，`git diff --check` 无空白错误。
-
-## 当前实施范围
-
-### 第一阶段：共享渲染器补齐
-
-- 在 `MarkdownRenderer` 内解析 fenced code block 的语言信息。
-- 引入 `MarkdownCodeBlock` 子组件，负责语言标签、复制按钮和复制状态。
-- 保留现有 `Scrollbar` 横向滚动能力，避免长代码撑破页面。
-- 扩展标题渲染到 h1-h6，并保留 `getHeadingId` 对文档目录的支持。
-- 扩展列表解析，支持 `-`、`*`、`+`、`1.`、`1)` 和 GitHub 风格任务列表。
-- 扩展行内解析，支持图片、自动链接、删除线。
-
-### 第二阶段：样式统一
-
-- 全局 Markdown 样式覆盖业务场景，保持紧凑、清晰、可扫描。
-- 文档中心用 `.docs-markdown` 适配主题变量，避免硬编码颜色破坏明暗主题。
-- 代码块 toolbar 和 copy button 不依赖具体页面。
-- 图片默认响应式展示，避免超出内容区域。
-
-### 第三阶段：验证
-
-- 使用前端 build 验证 TypeScript 和 Vite 构建。
-- 使用 `git diff --check` 验证补丁格式。
-- 手动检查至少一个文档页中代码块复制按钮、语言标签和表格滚动是否出现。
-
-## 后续增强项
-
-### 语法高亮
-
-当前不新增高亮依赖，避免一次性引入过重运行时代码。后续可以在以下方案中二选一：
-
-- `shiki`：适合文档中心，视觉质量高，但包体和初始化成本更高。
-- `highlight.js`：接入简单，覆盖语言广，但样式控制需要额外约束。
-
-建议当文档代码块数量稳定增加后再引入，并做按需加载或懒加载。
-
-### 更完整 CommonMark 支持
-
-当前渲染器覆盖 Planet 常见内容，不追求完整 CommonMark 兼容。后续如果需要完整规范，建议切换到成熟生态：
-
-- `react-markdown`
-- `remark-gfm`
-- `rehype-sanitize`
-- `rehype-slug`
-
-切换前需要评估：链接转换、目录 ID、现有样式、AI 输出安全策略和包体影响。
-
-### 安全策略
-
-目前渲染器不解析原始 HTML，这是正确默认值。后续如需支持 HTML，必须先明确：
-
-- 是否允许用户输入 Markdown。
-- 是否需要 HTML 白名单。
-- 是否需要 `rehype-sanitize`。
-- 图片和链接是否需要域名策略。
-
-### 文档页能力
-
-可继续补齐：
-
-- 标题锚点悬浮复制。
-- Mermaid 图表。
-- 代码块折叠。
-- 文档内搜索结果定位到代码块。
-- 复制按钮埋点，用于判断文档片段是否真正被使用。
-
-## 维护约束
-
-- Markdown 语法能力优先放在共享渲染器，不在具体文档页面散落实现。
-- 文档内容只表达内容，不承载 UI 行为。
-- 新增 Markdown 能力必须同时考虑文档中心、AI Playground、BGP 简报三个调用方。
-- 不解析原始 HTML，除非同步引入明确的 sanitize 策略。
-- 与主题相关的样式优先走页面容器变量覆盖，不在组件内写死文档中心颜色。

docs/plans/frontend-public-docs-site-plan.md

@@ -1,486 +0,0 @@
-# Frontend Public Docs Site Plan
-
-## 目标
-
-新增一个公开访问的 `/docs` 页面，作为 Planet 的开发设计文档与使用手册入口。
-
-这个页面应类似常见开源软件文档站：
-
-- 不需要登录即可访问
-- 与 `/earth` 和 admin 后台平级，但视觉和信息架构独立
-- 直接整理并展示仓库内 `docs/technical` 的 Markdown 文档
-- 支持搜索、分类导航、文档目录和内部跳转
-- 让 `docs/technical` 继续作为文档真源，避免页面内容和仓库文档漂移
-
-## 非目标
-
-本阶段不做：
-
-- 后端全文搜索服务
-- 数据库驱动的 CMS
-- 独立文档构建系统，例如 Docusaurus / VitePress
-- 每篇文档单独手写 React 页面
-- 用户权限、编辑器、在线保存或评论功能
-- 把 `docs/plans`、`docs/deprecated` 全量公开为正式手册
-
-后续可以再决定是否把 plans / deprecated 做成独立的“路线图 / 历史归档”分区。
-
-## 技术路线
-
-### 推荐方案：Markdown 直接渲染
-
-使用 Vite 在前端构建阶段直接加载 `docs/technical/**/*.md`：
-
-```ts
-const modules = import.meta.glob('../../../docs/technical/**/*.md', {
-  query: '?raw',
-  import: 'default',
-})
-```
-
-这样每篇 Markdown 文件仍然留在仓库文档目录中，`/docs` 页面只是读取、索引和渲染这些文档。
-
-当前项目已经满足主要前提：
-
-- 前端使用 Vite + React
-- `frontend/vite.config.ts` 已配置 `server.fs.allow: ['..']`
-- 已有 `MarkdownRenderer` 可作为基础
-- `docs/technical` 文档数量较少，前端本地搜索足够
-
-### 不推荐方案：每篇文档单独写 React
-
-不建议把每篇文档重写成 `.tsx` 页面，因为：
-
-- 文档会出现两份真源
-- 修改技术文档时还要同步 UI 页面
-- 计划文档、技术上下文、变量表这类内容天然适合 Markdown
-- 后续新增文档的成本会变高
-
-只有当某篇文档需要强交互演示、实时图表或复杂 UI 时，才考虑给该文档补充一个 React 组件扩展。
-
-## 信息架构
-
-### 公开路由
-
-新增：
-
-- `/docs`
-- `/docs/:slug`
-
-路由行为：
-
-- `/docs` 默认打开 `docs/technical/README.md`，或打开人工指定的首页文档
-- `/docs/:slug` 打开对应技术文档
-- 未找到文档时显示 docs 专属 404，而不是跳回 admin
-- `/docs` 加入 `App.tsx` 的公开路由白名单
-
-### 文档分类
-
-将 `docs/technical` 中的现有文档整理进以下分组：
-
-#### Overview
-
-- `README.md`
-
-#### Earth
-
-- `earth-frontend-context.md`
-- `earth-layer-style-reference.md`
-- `earth-render-layer-order.md`
-- `earth-satellite-footprint-policy.md`
-- `earth-bgp-context.md`
-- `earth-news-live-streams-collector-format.md`
-
-#### Frontend
-
-- `frontend-admin-frontend-context.md`
-- `frontend-layout-guidelines.md`
-
-#### Backend
-
-- `backend-collectors.md`
-- `backend-system-service-control.md`
-
-#### Agents
-
-- `agents-aiprovider.md`
-
-#### Ops
-
-- `ops-docker-compose-buildx-upgrade.md`
-
-### 页面布局
-
-桌面端：
-
-- 顶部：产品名、搜索框、当前文档标题
-- 左侧：文档分组导航
-- 中间：Markdown 正文
-- 右侧：当前文档目录，也就是 h2 / h3 anchors
-
-移动端：
-
-- 顶部固定搜索入口
-- 导航折叠为抽屉或下拉
-- 正文单列显示
-- 当前文档目录折叠为“本文目录”
-
-视觉风格：
-
-- 像开源软件 docs 页面，清晰、安静、可长时间阅读
-- 不复用 admin 后台的重操作感布局
-- 不做 Earth 的沉浸式深色 HUD 风格
-- 优先阅读性、扫描效率和代码/表格可读性
-
-## 前端实现设计
-
-### 文件结构
-
-建议新增：
-
-```text
-frontend/src/pages/Docs/
-  Docs.tsx
-  docs-content.ts
-  docs-search.ts
-  docs-slugs.ts
-  Docs.css
-```
-
-可选拆分：
-
-```text
-frontend/src/pages/Docs/components/
-  DocsSidebar.tsx
-  DocsSearch.tsx
-  DocsToc.tsx
-  DocsMarkdown.tsx
-```
-
-如果初版代码量不大，可以先保持在 `Docs.tsx` + 少量 helper 文件中，避免过度拆分。
-
-### 文档注册表
-
-创建一个 registry，负责将 Markdown 文件路径映射为文档元信息：
-
-```ts
-interface DocsEntry {
-  slug: string
-  path: string
-  title: string
-  group: string
-  order: number
-  loader: () => Promise<string>
-}
-```
-
-slug 规则：
-
-- `docs/technical/README.md` -> `overview`
-- `docs/technical/earth-layer-style-reference.md` -> `earth-layer-style-reference`
-- 只暴露稳定 slug，不暴露本机绝对路径
-
-标题规则：
-
-- 优先读取 Markdown 第一个 `# heading`
-- 没有 h1 时用人工 registry title
-- 再 fallback 到文件名转换标题
-
-### Markdown 渲染
-
-初版可以复用现有：
-
-- [frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx](/home/ray/dev/linkong/planet/frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx)
-
-但建议增强或包装为 docs 专用渲染：
-
-- heading 生成稳定 `id`
-- 右侧 TOC 使用同一套 heading 解析结果
-- 内部 Markdown 链接转换为 `/docs/:slug`
-- 外部链接保留 `target="_blank" rel="noreferrer"`
-- 表格横向滚动
-- 代码块保留等宽字体和语言标记
-- 支持 GitHub 风格的相对文档链接
-
-内部链接转换示例：
-
-- `earth-render-layer-order.md` -> `/docs/earth-render-layer-order`
-- `./earth-layer-style-reference.md` -> `/docs/earth-layer-style-reference`
-- `/home/ray/dev/linkong/planet/docs/technical/foo.md` -> `/docs/foo`
-
-对非 `docs/technical` 的链接：
-
-- 初版可保留原始链接文本
-- 或显示为不可跳转的 repo path
-- 后续再扩展为跨文档区导航
-
-### 搜索
-
-初版使用纯前端本地搜索。
-
-索引字段：
-
-- title
-- slug
-- group
-- headings
-- markdown 正文纯文本
-
-搜索策略：
-
-- 页面首次加载后异步加载所有 `docs/technical` Markdown
-- 生成内存索引
-- 用户输入时本地过滤
-- 简单打分即可：
-  - 标题命中权重最高
-  - heading 命中其次
-  - 文件名 / slug 命中其次
-  - 正文命中最低
-
-搜索结果展示：
-
-- 文档标题
-- 分组
-- 命中的 heading 或正文摘要
-- 点击跳转到文档
-
-当前只有 13 篇文档，不需要 Lunr、Fuse 或后端搜索。后续文档数量显著增长时，再考虑引入轻量搜索库。
-
-### 路由接入
-
-修改：
-
-- [frontend/src/App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
-
-新增 lazy import：
-
-```ts
-const Docs = lazy(() => import('./pages/Docs/Docs'))
-```
-
-公开路由：
-
-```ts
-const publicPaths = new Set(['/', '/earth', '/docs'])
-```
-
-注意：`/docs/:slug` 不能只用精确匹配 `Set`。
-
-建议改为：
-
-```ts
-const isPublicRoute =
-  window.location.pathname === '/' ||
-  window.location.pathname === '/earth' ||
-  window.location.pathname === '/docs' ||
-  window.location.pathname.startsWith('/docs/')
-```
-
-新增 routes：
-
-```tsx
-<Route path="/docs" element={<Docs />} />
-<Route path="/docs/:slug" element={<Docs />} />
-```
-
-### 样式
-
-建议独立 `Docs.css`，不依赖 admin 页面布局。
-
-核心样式要求：
-
-- 文档正文最大宽度控制在适合阅读的范围
-- 表格横向滚动，不撑破布局
-- 代码块横向滚动
-- 左侧导航固定或 sticky
-- 右侧 TOC sticky
-- 移动端隐藏右侧 TOC，导航折叠
-- 搜索结果浮层或独立面板不遮挡正文阅读
-
-注意：
-
-- 不做营销 hero
-- 不做卡片堆叠式首页
-- 首页第一屏应直接是文档入口和内容，而不是宣传页
-
-## 实施阶段
-
-### Phase 1：基础文档站
-
-目标：
-
-- `/docs` 可公开访问
-- 能看到 `docs/technical` 文档列表
-- 能打开每篇 Markdown
-- 能基本渲染标题、段落、列表、代码块、表格
-
-任务：
-
-- 新增 `Docs` 页面
-- 新增 docs registry
-- 接入 Vite raw Markdown loading
-- 接入 `/docs` 和 `/docs/:slug`
-- 加入公开路由白名单
-- 初版 CSS 布局
-
-验收：
-
-- 未登录访问 `/docs` 不跳转登录
-- `/docs/earth-layer-style-reference` 可打开样式参考文档
-- `/docs/backend-collectors` 可打开后端采集器文档
-- 构建通过：`source ~/.zshrc && bun run build`
-
-### Phase 2：搜索与 TOC
-
-目标：
-
-- 支持本地搜索所有 technical 文档
-- 当前文档右侧显示目录
-- 搜索结果可跳转
-
-任务：
-
-- 实现 heading parser
-- 实现 TOC 组件
-- 实现 search index
-- 搜索结果显示文档标题、分组和摘要
-- 当前文档标题与 active nav 高亮
-
-验收：
-
-- 搜索 `Fresnel` 能找到 Earth 图层样式文档
-- 搜索 `collector` 能找到 backend collectors
-- 点击搜索结果进入对应文档
-- 右侧 TOC 点击后滚动到对应 heading
-
-### Phase 3：链接清理与文档体验
-
-目标：
-
-- Markdown 内部链接在 docs 站内自然跳转
-- 长表格、代码块、绝对路径链接的显示更友好
-
-任务：
-
-- 转换 `docs/technical/*.md` 相对链接
-- 转换 repo 内 technical 文档绝对路径
-- 外链新窗口打开
-- 文件路径链接以代码样式显示
-- 增强空状态和 404
-
-验收：
-
-- 从 `docs/technical/README.md` 点击 technical 文档链接进入 `/docs/:slug`
-- 不支持的 repo 内路径不会导致前端崩溃
-- 外部链接行为正常
-
-### Phase 4：文档内容整理
-
-目标：
-
-- `docs/technical` 的首页适合作为公开手册入口
-- 每篇文档标题、摘要和分类清晰
-
-任务：
-
-- 检查每篇文档是否有唯一 h1
-- 给 README 补公开手册导览
-- 必要时补文档摘要
-- 保持文档内容仍然服务开发维护，不改成营销语气
-
-验收：
-
-- `/docs` 首页能说明各技术文档用途
-- 左侧分类和 README 内容一致
-- 没有明显重复、过期或找不到的主入口
-
-## 需要改动的文件
-
-预计新增：
-
-- `frontend/src/pages/Docs/Docs.tsx`
-- `frontend/src/pages/Docs/Docs.css`
-- `frontend/src/pages/Docs/docs-content.ts`
-- `frontend/src/pages/Docs/docs-search.ts`
-
-预计修改：
-
-- `frontend/src/App.tsx`
-- `frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx` 或新增 docs 专用 wrapper
-- `docs/technical/README.md`
-
-可选修改：
-
-- `frontend/src/index.css`，只放全局极少量 docs shell reset 时才需要
-- `docs/CHANGELOG.md`，实施完成后记录
-- `docs/version-history.md`，若进入版本发布流程再更新
-
-## 风险与注意事项
-
-### 构建路径风险
-
-Vite 从 `frontend/src` 读取 `../../../docs/technical/**/*.md` 时，需要确认开发和生产构建都可解析。
-
-缓解：
-
-- 使用相对路径 glob
-- 构建验证必须跑 `source ~/.zshrc && bun run build`
-- 不使用运行时 `fetch('/docs/...')` 读取仓库文件，避免生产环境缺文件
-
-### Markdown 能力不足
-
-现有 `MarkdownRenderer` 是轻量实现，可能不完整支持所有 GitHub Markdown。
-
-缓解：
-
-- 初版优先覆盖当前 `docs/technical` 实际用到的语法
-- 若后续需要脚注、嵌套列表、复杂代码高亮，再考虑引入 `react-markdown` 等依赖
-
-### Bundle 体积
-
-把所有 Markdown 打进前端 bundle 会增加体积。
-
-当前文档数量少，风险可接受。
-
-缓解：
-
-- 使用 lazy page chunk
-- Markdown loader 保持异步
-- 搜索索引在 `/docs` 页面内初始化，不影响 `/earth` 和 admin 首屏
-
-### 公开内容边界
-
-`docs/technical` 会被公开展示，需要避免包含密钥、内部机器地址、临时方案或不应公开的操作细节。
-
-缓解：
-
-- 实施前快速审阅 `docs/technical`
-- 暂不公开 `docs/plans` 和 `docs/deprecated`
-- 以后如需公开更多文档，先建立 allowlist
-
-## 验收清单
-
-- `/docs` 未登录可访问
-- `/docs/:slug` 未登录可访问
-- `/docs` 不影响 `/earth`
-- 未登录访问 admin 仍然跳登录
-- 左侧导航包含所有 `docs/technical` 文档
-- 文档按 Overview / Earth / Frontend / Backend / Agents / Ops 分类
-- Markdown 表格正常显示并可横向滚动
-- 代码块正常显示并可横向滚动
-- 搜索可搜索标题、heading 和正文
-- 搜索结果点击可跳转
-- 当前文档 TOC 可跳转
-- 不存在的 slug 显示 docs 404
-- `source ~/.zshrc && bun run build` 通过
-
-## 后续增强
-
-- 给文档页面增加复制 heading 链接按钮
-- 给代码块增加复制按钮
-- 增加“上一页 / 下一页”导航
-- 增加最近更新信息
-- 从 git metadata 读取文档更新时间
-- 引入轻量全文搜索库
-- 支持 plans / deprecated 独立分区
-- 增加页面内反馈入口

docs/technical/README.md

@@ -0,0 +1,26 @@
+# Technical Docs
+
+这里放“当前实现和当前结构”的文档，重点回答：
+
+- 现在代码是怎么组织的
+- 当前入口在哪
+- 状态和组件如何工作
+- 后续改动应该沿着哪条实现边界继续走
+
+适合放入这里的内容：
+
+- 前端上下文
+- Earth 前端结构
+- 后端运行控制
+- collector 现状
+- 采集格式约定
+
+不适合放入这里的内容：
+
+- 尚未完成的 roadmap
+- 未来迭代方案
+- 大范围重构计划
+
+这些应放入：
+
+- [docs/plans/README.md](/home/ray/dev/linkong/planet/docs/plans/README.md)

docs/technical/backend-collectors.md

@@ -84,9 +84,6 @@ async def run(self, db):
 | HuggingFace Spaces | space | Demo应用 | 1天 |
 | PeeringDB | ixp/network/facility | 互联网交换点/网络/机房 | 1-2天 |
 | TeleGeography | submarine_cable | 海底光缆信息 | 7天 |
-| Space-Track TLE | satellite_tle | 卫星轨道 TLE 数据 | 依采集器配置 |
-| BarentsWatch AIS | vessel | 船只位置、航速、航向、MMSI 等 AIS 数据 | 依采集器配置 |
-| AISStream Vessels | vessel_ais | AIS WebSocket 实时流，写入原始观测层并由聚合接口展示 | 依采集器配置 |
 
 ## 四、数据格式 (统一存储到 CollectedData 表)
 
@@ -203,30 +200,6 @@ def start_scheduler():
 
 **核心文件**: `backend/app/services/scheduler.py`
 
-### 成功采集与连接状态
-
-内置采集器成功采集后，调度器会记录当前有效配置已通过连接验证：
-
-```python
-if datasource.last_status == "success":
-    effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
-    checksum, _ = await build_builtin_connectivity_checksum(...)
-    await save_connectivity_success(
-        db,
-        datasource.source,
-        checksum,
-        {"status_code": None},
-        connected_by="collection",
-    )
-```
-
-这个记录用于控制台“采集器设置”中的连接状态判断：如果当前配置和成功采集时的 checksum 一致，就视为已连接，不要求用户再手动点击连接按钮。只有 endpoint、请求头、基础配置或凭证指纹变化时，才需要重新验证。
-
-相关实现见：
-
-- [datasource_connectivity.py](/home/ray/dev/linkong/planet/backend/app/services/datasource_connectivity.py)
-- [数据源、采集器设置与连接验证](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)
-
 ## 八、相关代码文件
 
 ```
@@ -238,88 +211,13 @@ backend/app/services/collectors/
 ├── epoch_ai.py          # Epoch AI采集器
 ├── huggingface.py       # HuggingFace采集器
 ├── peeringdb.py         # PeeringDB采集器
-├── telegeraphy.py       # TeleGeography海底光缆采集器
-├── vessel_ais.py        # BarentsWatch AIS 船只采集器
-└── aisstream.py         # AISStream WebSocket 船只采集器
+└── telegeraphy.py      # TeleGeography海底光缆采集器
 
 backend/app/models/
 └── collected_data.py    # 统一数据模型
 ```
 
-## 九、凭证型采集器
-
-部分采集器需要外部服务凭证，例如：
-
-| 采集器 | credential provider | 凭证来源 |
-| --- | --- | --- |
-| `barentswatch_vessels` | `barentswatch` | 控制台采集器设置、环境变量、`~/.zshrc` |
-| `aisstream_vessels` | `aisstream` | 控制台采集器设置、环境变量 |
-| `spacetrack_tle` | `spacetrack` | 环境变量、`~/.zshrc` |
-
-### BarentsWatch AIS
-
-BarentsWatch AIS 的凭证解析统一在：
-
-- [barentswatch.py](/home/ray/dev/linkong/planet/backend/app/services/barentswatch.py)
-
-`VesselAISCollector` 只负责采集和转换 AIS 数据，不再自己读取环境变量或拼 token 请求。它通过：
-
-- `resolve_barentswatch_config()`
-- `fetch_barentswatch_access_token()`
-
-获取运行时配置。
-
-解析优先级：
-
-1. `DataSourceConfig.auth_config`
-2. `DataSourceConfig.config`
-3. 环境变量
-4. `~/.zshrc`
-
-支持变量：
-
-```bash
-export BARENTSWATCH_CLIENT_ID="..."
-export BARENTSWATCH_CLIENT_SECRET="..."
-```
-
-并兼容历史拼写：
-
-```bash
-export BARRENTSWATCH_CLIENT_ID="..."
-export BARRENTSWATCH_CLIENT_SECRET="..."
-```
-
-连接验证会先请求 `https://id.barentswatch.no/connect/token` 获取 `scope=ais` 的 access token，再用 `Authorization: Bearer <token>` 请求 AIS endpoint。
-
-## 十、采集器设置与连接验证
-
-控制台的“采集器设置”页提供所有内置采集器的 endpoint、请求头、超时、重试和凭证配置。连接验证不是只看前端按钮状态，而是由后端计算 checksum：
-
-- endpoint
-- auth type
-- headers
-- config
-- credential provider
-- 凭证指纹
-
-相关 API：
-
-```http
-GET  /api/v1/datasources/configs/all
-POST /api/v1/datasources/configs/builtin/connection-status
-POST /api/v1/datasources/configs/builtin/connect
-POST /api/v1/settings/integrations/barentswatch/connect
-GET  /api/v1/settings/credential-guides/{provider}
-POST /api/v1/settings/credential-guides/{provider}/generate
-POST /api/v1/settings/credential-guides/{provider}/reset
-```
-
-更多细节见：
-
-- [数据源、采集器设置与连接验证](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)
-
-## 十一、数据使用场景
+## 九、数据使用场景
 
 采集的数据最终会：
 
@@ -327,7 +225,7 @@ POST /api/v1/settings/credential-guides/{provider}/reset
 2. **态势分析** - 统计全球算力分布、增长趋势
 3. **告警系统** - 检测重要节点变化
 
-## 十二、采集器注册机制
+## 十、采集器注册机制
 
 采集器在应用启动时自动注册：
 
@@ -349,7 +247,7 @@ collector_registry.register(TeleGeographyCableSystemCollector())
 
 **核心文件**: `backend/app/services/collectors/registry.py`
 
-## 十三、触发采集
+## 十一、触发采集
 
 ### 方式一：定时触发
 系统启动时，APScheduler会自动根据各采集器的`frequency_hours`设置定时任务。

docs/technical/earth-bgp-context.md

@@ -187,7 +187,7 @@ Current reality:
 - that is expected, because incidents are aggregated and de-noised
 - but incident-first rendering makes the Earth view look too quiet unless there is another always-available activity layer
 
-Implementation detail for the recommended `activity layer` is expanded in the [BGP Region Aggregation Plan](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md).
+Implementation detail for the recommended `activity layer` is expanded in [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md).
 
 So the immediate next milestone is: