Compare commits

...

9 Commits

Author SHA1 Message Date
linkong
ac69d5d354 release: bump version to 0.43.0 2026-04-28 16:10:17 +08:00
rayd1o
1cd2dab0ee release: bump version to 0.42.2 2026-04-28 04:35:13 +08:00
rayd1o
42d019af36 release: bump version to 0.42.1 2026-04-28 04:29:44 +08:00
rayd1o
b4e8afb272 release: bump version to 0.42.0 2026-04-28 04:27:18 +08:00
rayd1o
eeee788530 release: bump version to 0.41.2 2026-04-27 23:23:23 +08:00
linkong
655e2a7d2d release: bump version to 0.41.1
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:31:34 +08:00
linkong
3ea99a9529 release: bump version to 0.41.0 2026-04-27 13:58:29 +08:00
rayd1o
f9c1334365 release: bump version to 0.40.5
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-26 05:03:30 +08:00
rayd1o
5f47ec1659 release: bump version to 0.40.4 2026-04-26 01:41:29 +08:00
126 changed files with 16303 additions and 1085 deletions

View File

@@ -12,6 +12,19 @@ allowed-tools: ["Read", "Edit", "Bash", "Grep", "Glob"]
`$ARGUMENTS` 非空,则只检查指定文件/目录;否则检查所有未提交修改(`git diff HEAD`)。
## 节省上下文规则
优先用确定性的 CLI 检查缩小范围,不要一上来把完整文件或大 diff 读入上下文:
```bash
git diff --name-only HEAD
git diff --unified=0 HEAD -- <path>
git diff --check
rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
```
只有 focused diff 不足以安全判断或修改时,才读取完整文件。
## 审查清单
按优先级检查以下问题(只报告在本次 diff 中**新增或修改**的代码里存在的问题):
@@ -59,8 +72,13 @@ git diff HEAD --name-only
### Step 2 — 逐文件阅读并分析
- 用 Read 工具读取完整文件(不只读 diff
- 对照审查清单,记录每个问题:文件名、行号、问题类型、建议修复方式
先从 focused diff 开始:
```bash
git diff --unified=0 HEAD -- <file>
```
`rg``git diff --check`、编译器或 linter 输出确认确定性问题。只有需要上下文时才用 Read 读取完整文件。对照审查清单,记录每个问题:文件名、行号、问题类型、建议修复方式。
### Step 3 — 报告问题清单
@@ -95,6 +113,7 @@ git diff HEAD --name-only
- 只改在审查清单中发现的问题,不做额外优化
- 每次 Edit 只修改确实有问题的行,保持 diff 最小
- 改完后用 `grep` 验证旧的坏代码已消失
- 优先做精确补丁;只有仓库已有对应格式化流程时,才运行格式化工具
### Step 5 — 输出总结

151
.claude/commands/docs.md Normal file
View File

@@ -0,0 +1,151 @@
---
description: 分析本次 git 变更,在 docs/technical/zh/ 中新建或更新对应的技术文档
argument-hint: 可选:指定要记录的主题,或留空自动从 git diff 推断
allowed-tools: ["Read", "Edit", "Write", "Bash", "Glob", "Grep"]
---
# /docs — 技术文档写入工作流
## 目标
根据当前 git 变更(或用户指定主题)在 `docs/technical/zh/` 中写入或更新技术文档,记录**为什么**这样做,而不只是记录做了什么。
## 执行步骤
### Step 1 — 理解变更范围
```bash
git diff HEAD --stat # 变更文件一览
git diff HEAD --name-only # 变更文件列表
git log --oneline -10 # 近期 commit 上下文
```
`$ARGUMENTS` 指定了主题,优先聚焦该主题;否则从文件列表和 diff stat 推断变更主题。不要默认读取完整仓库 diff只对决定文档主题所需的文件读取 focused diff
```bash
git diff HEAD -- <path>
rg -n "class |def |function |export |router|@router|interface |type " <path>
```
### Step 2 — 确认文档范围
分析变更,判断:
1. **应写几篇文档**:单一主题写一篇,跨领域变更可拆分(如后端性能优化 + 运维启动脚本分开写)
2. **是新建还是更新**:检查 `docs/technical/zh/` 中是否已有相关文档
3. **文档命名**:按 `领域-主题-副题.md` 格式,全小写,用连字符,如:
- `backend-datasources-api-performance.md`
- `ops-planet-sh-startup.md`
- `earth-bgp-context.md`
```bash
ls docs/technical/zh/ # 查看现有文档
```
**先输出写作计划供用户确认**(若变更明确且范围小,可直接执行):
```
文档计划:
新建docs/technical/zh/ops-planet-sh-startup.md — planet.sh 启动性能优化
更新docs/technical/zh/backend-datasources-api-performance.md — 补充并行化细节
```
### Step 3 — 写文档
遵循以下原则:
**记录 WHY不只记录 WHAT**
- 好:`将戳文件从 /tmp 移到 ~/.cache/planet/,因为 WSL 重启后 /tmp 被清空`
- 差:`修改了 AI_PROVIDER_BUILD_STAMP_FILE 的值`
**必须包含的内容**
- 背景/问题:改动之前存在什么问题,为什么要改
- 核心设计决策及其理由
- 关键代码片段(用 diff 或 before/after 展示)
- 相关文件列表
**格式要求**
- 使用 `##``###` 分级,不要超过三级
- 代码块注明语言python / bash / typescript / sql
- 表格用于对比多个选项或列出参数
- 中文写作,技术术语保留英文原文
- `docs/technical/zh/` 中的文档不得用英文原文占位;如果存在 `docs/technical/en/` 对应文件,禁止逐字复制成中文文件
- 中文文档内部链接应指向 `docs/technical/zh/...`,除非明确引用英文专属文档
**文档结构模板**
```markdown
# 标题(说明做了什么)
## 背景
为什么要做这个改动,改动前存在什么问题。
## 核心变更
### 子主题一
before/after 或决策说明 + 关键代码
### 子主题二
...
## 相关文件
- `path/to/file.py` — 简短说明
```
### Step 4 — 验证
- 读一遍写好的文档,确认逻辑清晰、代码片段无明显错误
-`rg --files``test -e` 确认文档中的文件路径在项目中真实存在,避免凭记忆判断:
- 检查中文文档没有误复制英文版:
```bash
python - <<'PY'
from pathlib import Path
same = []
for en in sorted(Path("docs/technical/en").glob("*.md")):
zh = Path("docs/technical/zh") / en.name
if zh.exists() and en.read_text() == zh.read_text():
same.append(en.name)
if same:
raise SystemExit("identical en/zh docs: " + ", ".join(same))
print("no identical en/zh docs")
PY
```
- 检查中文文档内部链接没有继续指向无语言目录:
```bash
rg -n "/home/ray/dev/linkong/planet/docs/technical/(?!zh|en)" docs/technical/zh --pcre2
```
```bash
# 对文档中提到的关键路径做快速验证
ls <mentioned_paths>
```
如需检查大量链接,优先用确定性提取:
```bash
rg -n "\]\(([^)]+)\)" docs/technical/zh/<doc>.md
```
### Step 5 — 完成确认
输出摘要:
```
✓ 新建docs/technical/zh/ops-planet-sh-startup.md约 xxx 字)
✓ 更新docs/technical/zh/backend-datasources-api-performance.md
```
## 注意事项
- 不要写流水账式的"改了 A、改了 B、改了 C",要写改动背后的约束和权衡
- 不要在文档中引用 PR 号、issue 号、或当前对话——这些会随时间失效
- 代码片段保持简洁,只保留说明问题的关键部分,省略无关样板代码
- 如果某个变更已有文档记录,优先在原文档中追加,而不是新建
- 文档是给未来的开发者看的,假设读者熟悉项目但不了解这次改动的背景

View File

@@ -72,6 +72,8 @@ Verification
## 执行风格
- 重证据,轻口头判断
- 优先使用确定性工具证据:`rg``git diff --stat``git diff -- <path>`、测试、构建、lint、`curl`、数据库查询等能直接证明成功标准的方式
- 不把大段命令输出粘进回复;保留在工具调用里,回复只总结关键证据
- 重验收,轻自我感觉
- 优先用测试、日志、产物、对比结果来证明完成
- 对长期任务保持“未达标就继续”的节奏

View File

@@ -28,6 +28,19 @@ allowed-tools: ["Read", "Edit", "Bash", "Glob", "Grep"]
- `docs/CHANGELOG.md`
- `docs/version-history.md`
## 节省上下文规则
发版判断应以确定性 CLI 证据为主,优先使用紧凑命令和定点读取:
```bash
git status --short
git diff --stat HEAD
git diff --name-only HEAD
rg -n "version|^## |^Released:|当前开发版本|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
```
除非需要判断某个代码变更是否属于本次发版,否则不要读取完整 diff。
## 执行步骤
### Step 1 — 环境检查
@@ -45,7 +58,7 @@ cat VERSION # 读取当前版本
### Step 2 — 确定发版类型与新版本号
-`$ARGUMENTS` 提供了明确类型(`feature` / `bugfix`),直接使用
- 否则根据当前 `git diff HEAD``git log` 推断
- 否则根据 `git diff --stat HEAD``git diff --name-only HEAD`、必要的 focused diff`git log` 推断
- 计算新版本号(例:`0.26.2` → bugfix → `0.26.3`
- **先输出发版计划供用户确认**
@@ -91,12 +104,13 @@ cat VERSION # 读取当前版本
针对本次变更范围做最小验证:
- Python 文件有修改:`python3 -m py_compile <changed_files>`
- Frontend 文件有修改:运行项目标准检查(若无则跳过并说明)
- Python 文件有修改:先用 `git diff --name-only HEAD -- '*.py'` 列出,再运行 `python3 -m py_compile <changed_files>`
- Frontend 文件有修改:先用 `git diff --name-only HEAD -- frontend` 判断范围,再运行项目标准检查(若无则跳过并说明)
- 版本号一致性检查:用 grep 确认 VERSION、package.json、pyproject.toml 中的版本号完全一致
```bash
grep -h "version" VERSION frontend/package.json pyproject.toml
cat VERSION
rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
```
### Step 7 — 提交前预览

View File

@@ -21,6 +21,19 @@ If the user specifies a file or directory, check only that. Otherwise check all
Only report issues present in **newly added or modified** lines of this diff — do not audit unchanged code.
## Token-Saving Rule
Prefer deterministic CLI checks before reading files into model context:
```bash
git diff --name-only HEAD
git diff --unified=0 HEAD -- <path>
git diff --check
rg -n "TODO|FIXME|console\.log|debugger|print\(" <changed-paths>
```
Read full files only when the focused diff does not provide enough surrounding context to make a safe edit.
## Checklist
### 1. Duplicate Logic
@@ -64,7 +77,13 @@ Filter to the user-specified path if one was provided.
### Step 2 — Read and analyze each file
Read the full file (not just the diff) with the Read tool. For each file, record every issue found: filename, line number, category, and suggested fix.
Start with focused diffs:
```bash
git diff --unified=0 HEAD -- <file>
```
Use `rg`, `git diff --check`, and compiler/linter output for deterministic findings. Read the full file only for files that need surrounding context. For each issue found, record filename, line number, category, and suggested fix.
### Step 3 — Report findings before touching anything
@@ -99,6 +118,7 @@ Principles:
- Only fix issues identified in the checklist — no extra improvements
- Keep each Edit as small as possible
- After fixing, verify the old bad pattern is gone with grep
- Prefer `apply_patch` for targeted edits; use formatters only when the repository already uses them for the touched file type
### Step 5 — Summary

117
.codex/skills/docs/SKILL.md Normal file
View File

@@ -0,0 +1,117 @@
---
name: docs
description: Analyze current Planet repo changes and create or update technical documentation under docs/technical/zh. Use when the user asks to write docs, update technical docs, summarize implementation changes into documentation, or port the Claude docs-codex workflow into Codex.
---
# Docs
Use this skill when the user asks to create or update Planet technical documentation, especially under `docs/technical/zh/`.
## Goal
Write or update technical docs that explain why a change exists, not only what files changed.
Default target directory:
- `docs/technical/zh/`
## Workflow
1. Gather change context:
```bash
git diff HEAD --stat
git diff HEAD --name-only
git log --oneline -10
ls docs/technical/zh/
```
If the user gives a specific topic, focus on that topic. Otherwise infer the documentation topic from the file list and diff stat. Do **not** read the full repository diff by default; inspect focused diffs only for the files that define the doc topic:
```bash
git diff HEAD -- <path>
rg -n "class |def |function |export |router|@router|interface |type " <path>
```
2. Decide document scope:
- Use one document for one coherent topic.
- Split documents when the changes cross meaningful domains, such as backend performance and ops startup behavior.
- Prefer updating an existing relevant doc over creating a duplicate.
- Name new files as lowercase hyphenated `domain-topic-detail.md`, for example:
- `backend-datasources-api-performance.md`
- `ops-planet-sh-startup.md`
- `earth-bgp-context.md`
3. Write the doc in Chinese:
- Write Chinese prose for `docs/technical/zh/`.
- Keep technical identifiers, API paths, config keys, code symbols, and standard product names in English where appropriate.
- Use `##` and `###` headings; avoid going deeper than three levels.
- Use fenced code blocks with language tags.
- Use tables when comparing options or listing parameters.
4. Required content:
- Background/problem: what was wrong before and why the change was needed.
- Core design decisions and rationale.
- Key code snippets, preferably before/after or focused excerpts.
- Related files and what each file contributes.
5. Verification:
- Read the completed doc and check that the reasoning is clear.
- Verify important referenced paths exist.
- Use `rg --files` or `test -e` for path existence instead of relying on memory.
- Run a quick duplicate-language check when editing bilingual docs:
```bash
python - <<'PY'
from pathlib import Path
same = []
for en in sorted(Path("docs/technical/en").glob("*.md")):
zh = Path("docs/technical/zh") / en.name
if zh.exists() and en.read_text() == zh.read_text():
same.append(en.name)
if same:
raise SystemExit("identical en/zh docs: " + ", ".join(same))
print("no identical en/zh docs")
PY
```
Also check that Chinese docs do not link to the old language-less technical docs path:
```bash
rg -n "/home/ray/dev/linkong/planet/docs/technical/(?!zh|en)" docs/technical/zh --pcre2
```
This command should return no matches.
If checking many links, prefer deterministic extraction:
```bash
rg -n "\]\(([^)]+)\)" docs/technical/zh/<doc>.md
```
## Hard Constraints
- A file under `docs/technical/zh/` must not be an English source file copied as a placeholder.
- Do not leave a Chinese doc with only an English title and English first-screen content.
- When an English counterpart exists in `docs/technical/en/`, never duplicate it byte-for-byte into `docs/technical/zh/`.
- Internal links inside `docs/technical/zh/` should point to `docs/technical/zh/...` for Chinese docs, unless intentionally linking to an English-only file.
- Do not reference PR numbers, issue numbers, or the current conversation.
- Do not write changelog-style lists like "changed A, changed B, changed C" without the constraints and tradeoffs behind those changes.
- Keep code snippets concise and relevant.
## Recommended Output
After editing, summarize:
```md
Updated:
- docs/technical/zh/example.md — what changed
Verified:
- no identical en/zh docs
- no language-less docs/technical links in zh docs
```

View File

@@ -72,6 +72,8 @@ In Codex, only use actual subagents when the user explicitly asks for delegation
## Operating Rules
- Prefer objective checks over self-reported completion.
- Prefer deterministic tool evidence over long model summaries: use `rg`, `git diff --stat`, targeted `git diff -- <path>`, tests, builds, linters, `curl`, or database queries when they can prove a criterion.
- Do not paste large command output into the conversation; summarize the evidence and keep raw output in tool calls.
- Do not confuse progress with completion.
- If the worker says "done", verify it.
- If verification fails, continue from the gap instead of restarting blindly.

View File

@@ -18,7 +18,7 @@ Do not use this skill for ordinary commits that are not being released.
## Versioning Rules
- `feature` -> bump `+0.1.0`
- `feature` -> bump minor and reset patch to `0` (`x.y.z``x.(y+1).0`; for example `0.41.2``0.42.0`)
- `bugfix` -> bump `+0.0.1`
- `docs`, `maintenance`, and `refactor` do not bump by default unless the user explicitly wants a release
@@ -35,6 +35,19 @@ Use `git rev-parse --show-toplevel` to get the repo root. All paths are relative
- `docs/CHANGELOG.md`
- `docs/version-history.md`
## Token-Saving Rule
Release work should be driven by deterministic CLI evidence. Prefer compact commands and targeted file reads:
```bash
git status --short
git diff --stat HEAD
git diff --name-only HEAD
rg -n "version|^## |^Released:|current" VERSION frontend/package.json pyproject.toml docs/CHANGELOG.md docs/version-history.md
```
Do not inspect full diffs unless deciding whether changed code belongs in the release.
## Workflow
### Step 1 — Environment check
@@ -52,8 +65,10 @@ If unrelated uncommitted changes exist, list them and ask the user whether to in
### Step 2 — Determine release type and next version
- If the user provided an explicit type (`feature` / `bugfix`), use it
- Otherwise infer from `git diff HEAD` and recent `git log`
- Compute the next version (e.g. `0.26.2` → bugfix → `0.26.3`)
- Otherwise infer from `git diff --stat HEAD`, `git diff --name-only HEAD`, focused diffs for changed code, and recent `git log`
- Compute the next version:
- `feature`: increment minor and reset patch to `0` (e.g. `0.41.2``0.42.0`)
- `bugfix`: increment patch only (e.g. `0.26.2``0.26.3`)
- **Show the release plan before making any changes:**
```
@@ -104,12 +119,13 @@ Get today's date with `date +%Y-%m-%d`.
Run the smallest relevant validation for the changes in scope:
- Python files changed: `python3 -m py_compile <changed_files>`
- Frontend files changed: run the project-standard check if available; otherwise skip and say so
- Python files changed: list changed Python files with `git diff --name-only HEAD -- '*.py'`, then run `python3 -m py_compile <changed_files>`
- Frontend files changed: list changed frontend files with `git diff --name-only HEAD -- frontend`, then run the project-standard check if available; otherwise skip and say so
- Version consistency: confirm VERSION, package.json, pyproject.toml, and uv.lock all show the same version
```bash
grep -h "version" VERSION frontend/package.json pyproject.toml
cat VERSION
rg -n "\"version\":|^version =|version = " frontend/package.json pyproject.toml uv.lock
```
### Step 7 — Pre-commit preview

View File

@@ -1 +1 @@
0.40.3
0.43.0

View File

@@ -1,6 +1,10 @@
FROM python:3.14-slim
ARG PYTHON_IMAGE=python:3.14-slim
ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
FROM ${UV_IMAGE} AS uv
FROM ${PYTHON_IMAGE}
COPY --from=uv /uv /uvx /bin/
WORKDIR /app

View File

@@ -37,8 +37,26 @@ def verify_service_token(x_provider_token: str | None = Header(default=None)) ->
)
def get_provider_service() -> ProviderService:
return ProviderService()
def get_provider_service(
x_ai_provider: str | None = Header(default=None),
x_ai_provider_api: str | None = Header(default=None),
x_ai_base_url: str | None = Header(default=None),
x_ai_api_key: str | None = Header(default=None),
x_ai_model: str | None = Header(default=None),
x_ai_max_tokens: str | None = Header(default=None),
x_ai_anthropic_version: str | None = Header(default=None),
) -> ProviderService:
overrides = {
"provider": x_ai_provider,
"provider_api": x_ai_provider_api,
"base_url": x_ai_base_url,
"api_key": x_ai_api_key,
"model": x_ai_model,
"anthropic_version": x_ai_anthropic_version,
}
if x_ai_max_tokens:
overrides["max_tokens"] = x_ai_max_tokens
return ProviderService({key: value for key, value in overrides.items() if value not in (None, "")})
@app.get("/health")

View File

@@ -46,19 +46,22 @@ def _resolve_provider_api(provider: str, configured_api: str) -> str:
class ProviderService:
def __init__(self) -> None:
self.provider = _normalize_provider(settings.AI_PROVIDER)
def __init__(self, overrides: dict[str, Any] | None = None) -> None:
overrides = overrides or {}
self.provider = _normalize_provider(overrides.get("provider") or settings.AI_PROVIDER)
self.provider_api = _resolve_provider_api(
self.provider,
_normalize_provider_api(settings.AI_PROVIDER_API),
_normalize_provider_api(overrides.get("provider_api") or settings.AI_PROVIDER_API),
)
self.base_url = settings.AI_BASE_URL.rstrip("/")
self.api_key = settings.AI_API_KEY
self.default_model = settings.AI_MODEL
self.base_url = str(overrides.get("base_url") or settings.AI_BASE_URL).rstrip("/")
self.api_key = str(overrides.get("api_key") or settings.AI_API_KEY)
self.default_model = str(overrides.get("model") or settings.AI_MODEL)
self.timeout = settings.AI_TIMEOUT_SECONDS
self.http_retry_attempts = max(settings.AI_HTTP_RETRY_ATTEMPTS, 1)
self.max_tokens = settings.AI_MAX_TOKENS
self.anthropic_version = settings.AI_ANTHROPIC_VERSION
self.max_tokens = int(overrides.get("max_tokens") or settings.AI_MAX_TOKENS)
self.anthropic_version = str(
overrides.get("anthropic_version") or settings.AI_ANTHROPIC_VERSION
)
self.system_prompt = settings.AI_ANALYSIS_SYSTEM_PROMPT
def get_status(self) -> AIProviderStatusResponse:

View File

@@ -1,6 +1,10 @@
FROM python:3.14-slim
ARG PYTHON_IMAGE=python:3.14-slim
ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
COPY --from=ghcr.io/astral-sh/uv:latest /uv /uvx /bin/
FROM ${UV_IMAGE} AS uv
FROM ${PYTHON_IMAGE}
COPY --from=uv /uv /uvx /bin/
WORKDIR /app

View File

@@ -1,20 +1,34 @@
"""DataSourceConfig API for user-defined data sources"""
from typing import Optional
from typing import Any, Optional
from datetime import datetime
import base64
import json
import re
from fastapi import APIRouter, Depends, HTTPException, status
from sqlalchemy import select, func
from sqlalchemy.ext.asyncio import AsyncSession
from pydantic import BaseModel, Field
import httpx
from app.core.target_schema_registry import get_target_schema, list_target_schemas
from app.db.session import get_db
from app.models.user import User
from app.models.datasource_config import DataSourceConfig
from app.models.datasource_mapping import DataSourceMappingTemplate
from app.core.security import get_current_user
from app.core.cache import cache
from app.core.time import to_iso8601_utc
from app.schemas.ai import SituationalAnalysisRequest
from app.services.ai_client import AIProviderClient, get_ai_provider_client
from app.services.datasource_mapping import (
MappingError,
build_heuristic_mapping,
execute_mapping,
persist_mapped_records,
redact_for_llm,
stable_payload_hash,
)
router = APIRouter()
@@ -59,6 +73,44 @@ class DataSourceConfigResponse(BaseModel):
from_attributes = True
class CustomSampleRequest(BaseModel):
datasource_config_id: Optional[int] = None
config: Optional[DataSourceConfigCreate] = None
limit_bytes: int = Field(default=200000, ge=1000, le=1000000)
class MappingProposeRequest(BaseModel):
sample_payload: Any
target_schema: str
use_ai: bool = True
class MappingPreviewRequest(BaseModel):
sample_payload: Any
target_schema: str
mapping_json: dict
limit: int = Field(default=20, ge=1, le=100)
class MappingTemplateCreate(BaseModel):
datasource_config_id: int
target_schema: str
mapping_json: dict
sample_payload: Any | None = None
sample_payload_hash: Optional[str] = None
validation_status: str = Field(default="draft", pattern="^(draft|valid|invalid)$")
is_active: bool = False
class MappingTemplateUpdate(BaseModel):
target_schema: Optional[str] = None
mapping_json: Optional[dict] = None
sample_payload: Any | None = None
sample_payload_hash: Optional[str] = None
validation_status: Optional[str] = Field(default=None, pattern="^(draft|valid|invalid)$")
is_active: Optional[bool] = None
async def test_endpoint(
endpoint: str,
auth_type: str,
@@ -96,6 +148,134 @@ async def test_endpoint(
}
def _build_request_headers(auth_type: str, auth_config: dict, headers: dict) -> dict[str, str]:
request_headers = {str(key): str(value) for key, value in (headers or {}).items()}
auth_type = str(auth_type or "none").lower()
auth_config = auth_config or {}
if auth_type == "bearer" and auth_config.get("token"):
request_headers["Authorization"] = f"Bearer {auth_config['token']}"
elif auth_type == "api_key" and auth_config.get("api_key"):
location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
if location != "query":
key_name = auth_config.get("key_name", "X-API-Key")
request_headers[str(key_name)] = str(auth_config["api_key"])
elif auth_type == "basic":
username = auth_config.get("username", "")
password = auth_config.get("password", "")
credentials = f"{username}:{password}"
encoded = base64.b64encode(credentials.encode()).decode()
request_headers["Authorization"] = f"Basic {encoded}"
return request_headers
def _build_query_params(auth_type: str, auth_config: dict, config: dict) -> dict[str, Any]:
params = {}
candidate = (config or {}).get("params") or (config or {}).get("query_params")
if isinstance(candidate, dict):
params.update(candidate)
auth_type = str(auth_type or "none").lower()
auth_config = auth_config or {}
if auth_type == "api_key" and auth_config.get("api_key"):
location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
if location == "query":
key_name = auth_config.get("key_name") or auth_config.get("param_name") or "api_key"
params[str(key_name)] = auth_config["api_key"]
return params
async def fetch_custom_sample_from_config(config: DataSourceConfig, limit_bytes: int) -> Any:
request_config = config.config or {}
method = str(request_config.get("method") or request_config.get("request_method") or "GET").upper()
if method not in {"GET", "POST"}:
raise HTTPException(status_code=400, detail="Only GET and POST sample requests are supported.")
headers = _build_request_headers(config.auth_type, config.auth_config or {}, config.headers or {})
params = _build_query_params(config.auth_type, config.auth_config or {}, request_config)
timeout = float(request_config.get("timeout", 30))
json_body = request_config.get("json_body")
if json_body is None and str(request_config.get("body_type") or "").lower() in {"json", ""}:
candidate = request_config.get("body")
if isinstance(candidate, (dict, list)):
json_body = candidate
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
response = await client.request(
method,
config.endpoint,
headers=headers,
params=params or None,
json=json_body,
)
response.raise_for_status()
content = response.content[:limit_bytes]
if "application/json" in response.headers.get("content-type", ""):
return json.loads(content.decode(response.encoding or "utf-8"))
return {"text": content.decode(response.encoding or "utf-8", errors="replace")}
def _parse_mapping_from_ai_text(content: str) -> dict[str, Any] | None:
if not content:
return None
candidates = [content]
fenced = re.findall(r"```(?:json)?\s*(\{.*?\})\s*```", content, flags=re.DOTALL)
candidates = fenced + candidates
for candidate in candidates:
try:
parsed = json.loads(candidate)
except json.JSONDecodeError:
continue
if isinstance(parsed, dict) and isinstance(parsed.get("fields"), dict):
return parsed
return None
async def _get_config_for_sample(
payload: CustomSampleRequest,
db: AsyncSession,
) -> DataSourceConfig:
if payload.datasource_config_id is not None:
result = await db.execute(
select(DataSourceConfig).where(DataSourceConfig.id == payload.datasource_config_id)
)
config = result.scalar_one_or_none()
if not config:
raise HTTPException(status_code=404, detail="Configuration not found")
return config
if payload.config is None:
raise HTTPException(status_code=400, detail="datasource_config_id or config is required")
config_data = payload.config
return DataSourceConfig(
name=config_data.name,
description=config_data.description,
source_type=config_data.source_type,
endpoint=config_data.endpoint,
auth_type=config_data.auth_type,
auth_config=config_data.auth_config,
headers=config_data.headers,
config=config_data.config,
)
def serialize_mapping_template(template: DataSourceMappingTemplate) -> dict[str, Any]:
return {
"id": template.id,
"datasource_config_id": template.datasource_config_id,
"target_schema": template.target_schema,
"mapping_json": template.mapping_json,
"sample_payload_hash": template.sample_payload_hash,
"validation_status": template.validation_status,
"version": template.version,
"is_active": template.is_active,
"created_at": to_iso8601_utc(template.created_at),
"updated_at": to_iso8601_utc(template.updated_at),
}
@router.get("/configs")
async def list_configs(
active_only: bool = False,
@@ -345,3 +525,311 @@ async def list_all_datasources(
)
return {"total": len(result), "data": result}
@router.post("/custom/sample")
async def fetch_custom_sample(
payload: CustomSampleRequest,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Fetch a sample payload for a saved or draft custom data source."""
config = await _get_config_for_sample(payload, db)
try:
sample = await fetch_custom_sample_from_config(config, payload.limit_bytes)
except httpx.HTTPStatusError as exc:
raise HTTPException(
status_code=exc.response.status_code,
detail=f"Sample request failed: HTTP {exc.response.status_code}",
) from exc
except httpx.HTTPError as exc:
raise HTTPException(status_code=502, detail=f"Sample request failed: {exc}") from exc
return {
"success": True,
"sample_payload": sample,
"sample_payload_hash": stable_payload_hash(sample),
"redacted_preview": redact_for_llm(sample),
}
@router.get("/target-schemas")
async def get_datasource_target_schemas(
current_user: User = Depends(get_current_user),
):
"""List target schemas available for custom datasource mapping."""
return {"data": list_target_schemas()}
@router.post("/mappings/propose")
async def propose_datasource_mapping(
payload: MappingProposeRequest,
current_user: User = Depends(get_current_user),
ai_client: AIProviderClient = Depends(get_ai_provider_client),
):
"""Generate a mapping draft for a sample payload and target schema."""
schema = get_target_schema(payload.target_schema)
redacted_sample = redact_for_llm(payload.sample_payload)
fallback_mapping = build_heuristic_mapping(redacted_sample, payload.target_schema)
ai_error: str | None = None
mapping = fallback_mapping
generated_by = "heuristic"
if payload.use_ai:
try:
response = await ai_client.analyze(
SituationalAnalysisRequest(
title=f"Generate datasource mapping for {schema.key}",
objective=(
"Return only JSON for a deterministic mapping DSL. "
"The JSON must contain source.items_path and fields. "
"Do not include prose or code."
),
context={
"target_schema": schema.to_dict(),
"sample_payload": redacted_sample,
"mapping_dsl_example": fallback_mapping,
},
observations=[
"Use JSONPath-like paths beginning with $.",
"Never generate executable code.",
"Use field types from the target schema.",
],
constraints=[
"Return a single JSON object.",
"Do not include credentials or secrets.",
"Mark uncertain optional fields with default null.",
],
)
)
parsed = _parse_mapping_from_ai_text(response.content)
if parsed:
mapping = parsed
generated_by = "ai_provider"
else:
ai_error = "AI provider did not return a valid mapping JSON object."
except HTTPException as exc:
ai_error = str(exc.detail)
mapping.setdefault("meta", {})
if isinstance(mapping["meta"], dict):
mapping["meta"].update(
{
"generated_by": generated_by,
"requires_review": True,
"ai_error": ai_error,
}
)
return {
"target_schema": schema.to_dict(),
"mapping_json": mapping,
"sample_payload_hash": stable_payload_hash(payload.sample_payload),
"redacted_sample_payload": redacted_sample,
}
@router.post("/mappings/preview")
async def preview_datasource_mapping(
payload: MappingPreviewRequest,
current_user: User = Depends(get_current_user),
):
"""Preview deterministic mapping output for a sample payload."""
try:
preview = execute_mapping(
payload.sample_payload,
payload.mapping_json,
payload.target_schema,
limit=payload.limit,
)
except (MappingError, ValueError) as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
return {
"success": preview["failed_count"] == 0,
"preview": preview,
"sample_payload_hash": stable_payload_hash(payload.sample_payload),
}
@router.get("/mappings")
async def list_datasource_mappings(
datasource_config_id: Optional[int] = None,
active_only: bool = False,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""List saved mapping templates."""
query = select(DataSourceMappingTemplate).order_by(
DataSourceMappingTemplate.datasource_config_id,
DataSourceMappingTemplate.version.desc(),
)
if datasource_config_id is not None:
query = query.where(DataSourceMappingTemplate.datasource_config_id == datasource_config_id)
if active_only:
query = query.where(DataSourceMappingTemplate.is_active.is_(True))
result = await db.execute(query)
mappings = result.scalars().all()
return {"total": len(mappings), "data": [serialize_mapping_template(item) for item in mappings]}
@router.post("/mappings")
async def create_datasource_mapping(
payload: MappingTemplateCreate,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Save a mapping template for a datasource config."""
get_target_schema(payload.target_schema)
datasource = await db.get(DataSourceConfig, payload.datasource_config_id)
if not datasource:
raise HTTPException(status_code=404, detail="Configuration not found")
if payload.sample_payload is not None:
try:
execute_mapping(payload.sample_payload, payload.mapping_json, payload.target_schema, limit=100)
except (MappingError, ValueError) as exc:
raise HTTPException(status_code=400, detail=f"Mapping validation failed: {exc}") from exc
result = await db.execute(
select(func.max(DataSourceMappingTemplate.version)).where(
DataSourceMappingTemplate.datasource_config_id == payload.datasource_config_id,
DataSourceMappingTemplate.target_schema == payload.target_schema,
)
)
next_version = int(result.scalar() or 0) + 1
if payload.is_active:
await db.execute(
DataSourceMappingTemplate.__table__.update()
.where(DataSourceMappingTemplate.datasource_config_id == payload.datasource_config_id)
.values(is_active=False)
)
template = DataSourceMappingTemplate(
datasource_config_id=payload.datasource_config_id,
target_schema=payload.target_schema,
mapping_json=payload.mapping_json,
sample_payload_hash=payload.sample_payload_hash
or (stable_payload_hash(payload.sample_payload) if payload.sample_payload is not None else None),
validation_status=payload.validation_status,
version=next_version,
is_active=payload.is_active,
)
db.add(template)
await db.commit()
await db.refresh(template)
return {"message": "Mapping template saved successfully", "data": serialize_mapping_template(template)}
@router.put("/mappings/{mapping_id}")
async def update_datasource_mapping(
mapping_id: int,
payload: MappingTemplateUpdate,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Update a mapping template in place."""
template = await db.get(DataSourceMappingTemplate, mapping_id)
if not template:
raise HTTPException(status_code=404, detail="Mapping template not found")
target_schema = payload.target_schema or template.target_schema
mapping_json = payload.mapping_json or template.mapping_json
get_target_schema(target_schema)
if payload.sample_payload is not None:
try:
execute_mapping(payload.sample_payload, mapping_json, target_schema, limit=100)
except (MappingError, ValueError) as exc:
raise HTTPException(status_code=400, detail=f"Mapping validation failed: {exc}") from exc
if payload.is_active is True:
await db.execute(
DataSourceMappingTemplate.__table__.update()
.where(DataSourceMappingTemplate.datasource_config_id == template.datasource_config_id)
.where(DataSourceMappingTemplate.id != template.id)
.values(is_active=False)
)
template.target_schema = target_schema
template.mapping_json = mapping_json
if payload.sample_payload_hash is not None:
template.sample_payload_hash = payload.sample_payload_hash
elif payload.sample_payload is not None:
template.sample_payload_hash = stable_payload_hash(payload.sample_payload)
if payload.validation_status is not None:
template.validation_status = payload.validation_status
if payload.is_active is not None:
template.is_active = payload.is_active
await db.commit()
await db.refresh(template)
return {"message": "Mapping template updated successfully", "data": serialize_mapping_template(template)}
@router.post("/{config_id}/run-mapped")
async def run_mapped_datasource(
config_id: int,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Run a saved custom datasource through its active deterministic mapping."""
datasource = await db.get(DataSourceConfig, config_id)
if not datasource:
raise HTTPException(status_code=404, detail="Configuration not found")
result = await db.execute(
select(DataSourceMappingTemplate)
.where(DataSourceMappingTemplate.datasource_config_id == config_id)
.where(DataSourceMappingTemplate.is_active.is_(True))
.order_by(DataSourceMappingTemplate.version.desc())
.limit(1)
)
mapping = result.scalar_one_or_none()
if not mapping:
raise HTTPException(status_code=404, detail="No active mapping template found")
try:
sample = await fetch_custom_sample_from_config(datasource, 5_000_000)
mapped = execute_mapping(sample, mapping.mapping_json, mapping.target_schema)
except httpx.HTTPStatusError as exc:
raise HTTPException(
status_code=exc.response.status_code,
detail=f"Datasource request failed: HTTP {exc.response.status_code}",
) from exc
except httpx.HTTPError as exc:
raise HTTPException(status_code=502, detail=f"Datasource request failed: {exc}") from exc
except (MappingError, ValueError) as exc:
raise HTTPException(status_code=400, detail=f"Mapping failed: {exc}") from exc
if mapped["failed_count"] > 0:
return {
"status": "failed",
"datasource_config_id": config_id,
"mapping_id": mapping.id,
"mapping_version": mapping.version,
"target_schema": mapping.target_schema,
"mapped_count": mapped["mapped_count"],
"failed_count": mapped["failed_count"],
"errors": mapped["errors"][:20],
}
written_count = await persist_mapped_records(
db,
datasource_name=datasource.name,
datasource_config_id=datasource.id,
target_schema=mapping.target_schema,
records=mapped["records"],
mapping_version=mapping.version,
)
return {
"status": "success",
"datasource_config_id": config_id,
"mapping_id": mapping.id,
"mapping_version": mapping.version,
"target_schema": mapping.target_schema,
"fetched_count": mapped["total_items"],
"mapped_count": mapped["mapped_count"],
"written_count": written_count,
}

View File

@@ -9,6 +9,7 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.core.time import to_iso8601_utc
from app.core.security import get_current_user
from app.core.data_sources import get_data_sources_config
from app.core.datasource_defaults import DEFAULT_DATASOURCES
from app.db.session import get_db
from app.models.collected_data import CollectedData
from app.models.data_snapshot import DataSnapshot
@@ -35,6 +36,17 @@ def format_frequency_label(minutes: int) -> str:
return f"{minutes}m"
def datasource_metadata(source: str) -> dict:
info = DEFAULT_DATASOURCES.get(source, {})
return {
"display_name": info.get("display_name") or info.get("name") or source,
"is_free": bool(info.get("is_free", True)),
"requires_credentials": bool(info.get("requires_credentials", False)),
"credential_provider": info.get("credential_provider"),
"credential_status": info.get("credential_status", "none"),
}
def is_due_for_collection(datasource: DataSource, now: datetime) -> bool:
if datasource.last_run_at is None:
return True
@@ -72,31 +84,6 @@ async def _load_latest_running_tasks(
return {task.datasource_id: task for task in result.scalars().all()}
async def _load_latest_completed_tasks(
db: AsyncSession,
datasource_ids: list[int],
) -> dict[int, CollectionTask]:
if not datasource_ids:
return {}
ranked_tasks = (
select(
CollectionTask.id.label("task_id"),
_task_rank_column(CollectionTask.completed_at),
)
.where(CollectionTask.datasource_id.in_(datasource_ids))
.where(CollectionTask.completed_at.isnot(None))
.where(CollectionTask.status.in_(("success", "failed", "cancelled")))
.subquery()
)
result = await db.execute(
select(CollectionTask)
.join(ranked_tasks, CollectionTask.id == ranked_tasks.c.task_id)
.where(ranked_tasks.c.row_num == 1)
)
return {task.datasource_id: task for task in result.scalars().all()}
async def _load_latest_task_ids(
db: AsyncSession,
datasource_ids: list[int],
@@ -123,21 +110,6 @@ async def _load_latest_task_ids(
return {datasource_id: task_id for datasource_id, task_id in result.all()}
async def _load_datasource_data_counts(
db: AsyncSession,
sources: list[str],
) -> dict[str, int]:
if not sources:
return {}
result = await db.execute(
select(CollectedData.source, func.count(CollectedData.id))
.where(CollectedData.source.in_(sources))
.group_by(CollectedData.source)
)
return {source: count for source, count in result.all()}
async def _load_datasource_endpoint_overrides(
db: AsyncSession,
sources: list[str],
@@ -161,7 +133,7 @@ async def _load_datasource_endpoint_overrides(
async def _load_datasource_list_context(
db: AsyncSession,
datasources: list[DataSource],
) -> tuple[dict[int, CollectionTask], dict[int, CollectionTask], dict[str, int], dict[str, str]]:
) -> tuple[dict[int, CollectionTask], dict[str, str]]:
datasource_ids = [datasource.id for datasource in datasources]
sources = [datasource.source for datasource in datasources]
@@ -185,10 +157,8 @@ async def _load_datasource_list_context(
if stale_datasource_ids:
running_tasks = await _load_latest_running_tasks(db, datasource_ids)
completed_tasks = await _load_latest_completed_tasks(db, datasource_ids)
data_counts = await _load_datasource_data_counts(db, sources)
endpoint_overrides = await _load_datasource_endpoint_overrides(db, sources)
return running_tasks, completed_tasks, data_counts, endpoint_overrides
return running_tasks, endpoint_overrides
async def get_datasource_record(db: AsyncSession, source_id: str) -> Optional[DataSource]:
@@ -401,27 +371,19 @@ async def list_datasources(
collector_list = []
config = get_data_sources_config()
running_tasks, completed_tasks, data_counts, endpoint_overrides = await _load_datasource_list_context(
db,
datasources,
)
running_tasks, endpoint_overrides = await _load_datasource_list_context(db, datasources)
for datasource in datasources:
running_task = running_tasks.get(datasource.id)
last_task = completed_tasks.get(datasource.id)
endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(
datasource.source,
)
data_count = data_counts.get(datasource.source, 0)
last_run_at = datasource.last_run_at or (last_task.completed_at if last_task else None)
last_run = to_iso8601_utc(last_run_at)
last_status = datasource.last_status or (last_task.status if last_task else None)
endpoint = endpoint_overrides.get(datasource.source) or config.get_yaml_url(datasource.source)
last_run_at = datasource.last_run_at
last_status = datasource.last_status
collector_list.append(
{
"id": datasource.id,
"source": datasource.source,
"name": datasource.name,
**datasource_metadata(datasource.source),
"module": datasource.module,
"priority": datasource.priority,
"frequency": format_frequency_label(datasource.frequency_minutes),
@@ -429,11 +391,9 @@ async def list_datasources(
"is_active": datasource.is_active,
"collector_class": datasource.collector_class,
"endpoint": endpoint,
"last_run": last_run,
"last_run": to_iso8601_utc(last_run_at),
"last_run_at": to_iso8601_utc(last_run_at),
"last_status": last_status,
"last_records_processed": last_task.records_processed if last_task else None,
"data_count": data_count,
"is_running": running_task is not None,
"task_id": running_task.id if running_task else None,
"progress": running_task.progress if running_task else None,
@@ -576,6 +536,7 @@ async def get_datasource(
return {
"id": datasource.id,
"name": datasource.name,
**datasource_metadata(datasource.source),
"module": datasource.module,
"priority": datasource.priority,
"frequency": format_frequency_label(datasource.frequency_minutes),

View File

@@ -9,10 +9,19 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.core.security import get_current_user
from app.core.time import to_iso8601_utc
from app.core.config import settings as app_settings
from app.core.data_sources import get_data_sources_config
from app.core.datasource_defaults import DEFAULT_DATASOURCES
from app.db.session import get_db
from app.models.datasource import DataSource
from app.models.datasource_config import DataSourceConfig
from app.models.system_setting import SystemSetting
from app.models.user import User
from app.services.llm_provider_catalog import (
get_fallback_llm_provider_preset,
list_fallback_llm_provider_presets,
refresh_llm_provider_preset,
)
from app.services.scheduler import sync_datasource_job
from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
@@ -39,6 +48,21 @@ DEFAULT_SETTINGS = {
"password_policy": "medium",
},
"tv": DEFAULT_TV_SETTINGS,
"external_integrations": {
"ai_provider": {
"service_url": "",
"service_token": "",
"provider": "minimax",
"provider_api": "anthropic-messages",
"base_url": "https://api.minimaxi.com/anthropic",
"model": "MiniMax-M2.7",
"api_key": "",
"max_tokens": 1200,
"anthropic_version": "2023-06-01",
"timeout_seconds": 60,
"retry_attempts": 2,
}
},
}
@@ -96,6 +120,34 @@ class TVSettingsUpdate(BaseModel):
sources: list[TVStreamSourceUpdate] = Field(default_factory=list)
class AIProviderIntegrationUpdate(BaseModel):
service_url: str = ""
service_token: Optional[str] = None
provider: str = Field(default="minimax", max_length=80)
provider_api: str = Field(default="anthropic-messages", max_length=80)
base_url: str = Field(default="", max_length=500)
model: str = Field(default="", max_length=200)
api_key: Optional[str] = None
max_tokens: int = Field(default=1200, ge=1, le=200000)
anthropic_version: str = Field(default="2023-06-01", max_length=40)
timeout_seconds: int = Field(default=60, ge=5, le=600)
retry_attempts: int = Field(default=2, ge=1, le=10)
clear_service_token: bool = False
clear_api_key: bool = False
class BarentsWatchIntegrationUpdate(BaseModel):
endpoint: str = ""
client_id: str = ""
client_secret: Optional[str] = None
clear_client_secret: bool = False
class ExternalIntegrationsUpdate(BaseModel):
ai_provider: AIProviderIntegrationUpdate
barentswatch: BarentsWatchIntegrationUpdate
def merge_with_defaults(category: str, payload: Optional[dict]) -> dict:
merged = deepcopy(DEFAULT_SETTINGS[category])
if payload:
@@ -146,6 +198,158 @@ async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -
return merge_with_defaults(category, record.payload)
def _mask_secret(value: Optional[str]) -> dict:
if not value:
return {"configured": False, "preview": ""}
text = str(value)
if "-" in text:
prefix = text.split("-", 1)[0] + "-"
preview = prefix + ("*" * max(len(text) - len(prefix), 1))
else:
prefix_len = min(4, len(text))
preview = text[:prefix_len] + ("*" * max(len(text) - prefix_len, 1))
return {"configured": True, "preview": preview}
async def get_runtime_ai_provider_config(db: AsyncSession) -> dict:
runtime_record = await get_setting_record(db, "external_integrations")
payload = merge_with_defaults(
"external_integrations",
runtime_record.payload if runtime_record else None,
)
ai_payload = payload.get("ai_provider") or {}
has_runtime_llm_config = bool(
runtime_record
and isinstance(runtime_record.payload, dict)
and isinstance(runtime_record.payload.get("ai_provider"), dict)
)
return {
"service_url": ai_payload.get("service_url") or app_settings.AI_PROVIDER_SERVICE_URL,
"service_token": ai_payload.get("service_token") or app_settings.AI_PROVIDER_SERVICE_TOKEN,
"timeout_seconds": int(
ai_payload.get("timeout_seconds") or app_settings.AI_PROVIDER_TIMEOUT_SECONDS
),
"retry_attempts": int(
ai_payload.get("retry_attempts") or app_settings.AI_PROVIDER_RETRY_ATTEMPTS
),
"llm_config": {
"provider": ai_payload.get("provider") or "minimax",
"provider_api": ai_payload.get("provider_api") or "anthropic-messages",
"base_url": ai_payload.get("base_url") or "https://api.minimaxi.com/anthropic",
"model": ai_payload.get("model") or "MiniMax-M2.7",
"api_key": ai_payload.get("api_key") or "",
"max_tokens": int(ai_payload.get("max_tokens") or 1200),
"anthropic_version": ai_payload.get("anthropic_version") or "2023-06-01",
} if has_runtime_llm_config else {},
}
async def get_barentswatch_config_record(db: AsyncSession) -> Optional[DataSourceConfig]:
result = await db.execute(
select(DataSourceConfig)
.where(DataSourceConfig.name == "barentswatch_vessels")
.where(DataSourceConfig.is_active.is_(True))
)
return result.scalar_one_or_none()
async def serialize_external_integrations(db: AsyncSession) -> dict:
ai_config = await get_runtime_ai_provider_config(db)
runtime_setting = await get_setting_record(db, "external_integrations")
display_llm_config = ai_config["llm_config"] or DEFAULT_SETTINGS["external_integrations"]["ai_provider"]
barentswatch_record = await get_barentswatch_config_record(db)
yaml_config = get_data_sources_config()
barentswatch_auth = barentswatch_record.auth_config if barentswatch_record else {}
barentswatch_auth = barentswatch_auth or {}
return {
"ai_provider": {
"service_url": ai_config["service_url"],
"service_token": _mask_secret(ai_config["service_token"]),
"provider": display_llm_config.get("provider") or "minimax",
"provider_api": display_llm_config.get("provider_api") or "anthropic-messages",
"base_url": display_llm_config.get("base_url") or "https://api.minimaxi.com/anthropic",
"model": display_llm_config.get("model") or "MiniMax-M2.7",
"api_key": _mask_secret(display_llm_config.get("api_key")),
"max_tokens": int(display_llm_config.get("max_tokens") or 1200),
"anthropic_version": display_llm_config.get("anthropic_version") or "2023-06-01",
"timeout_seconds": ai_config["timeout_seconds"],
"retry_attempts": ai_config["retry_attempts"],
"source": "runtime" if runtime_setting else "env",
},
"barentswatch": {
"endpoint": (
barentswatch_record.endpoint
if barentswatch_record and barentswatch_record.endpoint
else yaml_config.get_yaml_url("barentswatch_vessels")
),
"client_id": barentswatch_auth.get("client_id") or "",
"client_secret": _mask_secret(barentswatch_auth.get("client_secret")),
"source": "datasource_config" if barentswatch_record else "default",
},
}
async def save_external_integrations_payload(
db: AsyncSession,
update: ExternalIntegrationsUpdate,
) -> dict:
current_payload = await get_setting_payload(db, "external_integrations")
current_ai = current_payload.get("ai_provider") or {}
ai_payload = {
"service_url": update.ai_provider.service_url.strip()
or app_settings.AI_PROVIDER_SERVICE_URL,
"service_token": current_ai.get("service_token") or "",
"provider": update.ai_provider.provider.strip() or "minimax",
"provider_api": update.ai_provider.provider_api.strip() or "anthropic-messages",
"base_url": update.ai_provider.base_url.strip(),
"model": update.ai_provider.model.strip(),
"api_key": current_ai.get("api_key") or "",
"max_tokens": update.ai_provider.max_tokens,
"anthropic_version": update.ai_provider.anthropic_version.strip() or "2023-06-01",
"timeout_seconds": update.ai_provider.timeout_seconds,
"retry_attempts": update.ai_provider.retry_attempts,
}
if update.ai_provider.clear_service_token:
ai_payload["service_token"] = ""
elif update.ai_provider.service_token not in (None, ""):
ai_payload["service_token"] = update.ai_provider.service_token
if update.ai_provider.clear_api_key:
ai_payload["api_key"] = ""
elif update.ai_provider.api_key not in (None, ""):
ai_payload["api_key"] = update.ai_provider.api_key
await save_setting_payload(db, "external_integrations", {"ai_provider": ai_payload})
default_endpoint = get_data_sources_config().get_yaml_url("barentswatch_vessels")
barentswatch_record = await get_barentswatch_config_record(db)
if barentswatch_record is None:
barentswatch_record = DataSourceConfig(
name="barentswatch_vessels",
description="BarentsWatch Live AIS credentials",
source_type="api",
endpoint=update.barentswatch.endpoint.strip() or default_endpoint,
auth_type="oauth_client_credentials",
auth_config={},
headers={},
config={},
is_active=True,
)
db.add(barentswatch_record)
current_auth = dict(barentswatch_record.auth_config or {})
if update.barentswatch.clear_client_secret:
current_auth.pop("client_secret", None)
elif update.barentswatch.client_secret not in (None, ""):
current_auth["client_secret"] = update.barentswatch.client_secret
current_auth["client_id"] = update.barentswatch.client_id.strip()
barentswatch_record.endpoint = update.barentswatch.endpoint.strip() or default_endpoint
barentswatch_record.auth_type = "oauth_client_credentials"
barentswatch_record.auth_config = current_auth
await db.commit()
return await serialize_external_integrations(db)
def format_frequency_label(minutes: int) -> str:
if minutes % 1440 == 0:
return f"{minutes // 1440}d"
@@ -155,9 +359,11 @@ def format_frequency_label(minutes: int) -> str:
def serialize_collector(datasource: DataSource) -> dict:
defaults = DEFAULT_DATASOURCES.get(datasource.source, {})
return {
"id": datasource.id,
"name": datasource.name,
"display_name": defaults.get("display_name") or datasource.name,
"source": datasource.source,
"module": datasource.module,
"priority": datasource.priority,
@@ -167,6 +373,10 @@ def serialize_collector(datasource: DataSource) -> dict:
"last_run_at": to_iso8601_utc(datasource.last_run_at),
"last_status": datasource.last_status,
"next_run_at": to_iso8601_utc(datasource.next_run_at),
"is_free": bool(defaults.get("is_free", True)),
"requires_credentials": bool(defaults.get("requires_credentials", False)),
"credential_provider": defaults.get("credential_provider"),
"credential_status": defaults.get("credential_status", "none"),
}
@@ -243,6 +453,46 @@ async def update_tv_settings(
return {"status": "updated", "tv": normalize_tv_settings(saved)}
@router.get("/integrations")
async def get_external_integrations(
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
return {"integrations": await serialize_external_integrations(db)}
@router.get("/integrations/ai-provider/presets")
async def get_ai_provider_presets(
current_user: User = Depends(get_current_user),
):
return {"data": list_fallback_llm_provider_presets()}
@router.post("/integrations/ai-provider/presets/{provider}/refresh")
async def refresh_ai_provider_preset(
provider: str,
current_user: User = Depends(get_current_user),
):
try:
return {"data": await refresh_llm_provider_preset(provider)}
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
except Exception as exc:
fallback = get_fallback_llm_provider_preset(provider)
fallback["refresh_error"] = str(exc)
return {"data": fallback}
@router.put("/integrations")
async def update_external_integrations(
payload: ExternalIntegrationsUpdate,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
saved = await save_external_integrations_payload(db, payload)
return {"status": "updated", "integrations": saved}
@router.get("/collectors")
async def get_collector_settings(
current_user: User = Depends(get_current_user),
@@ -289,6 +539,7 @@ async def get_all_settings(
"notifications": setting_payloads["notifications"],
"security": setting_payloads["security"],
"tv": await get_tv_settings_payload(db),
"integrations": await serialize_external_integrations(db),
"collectors": [serialize_collector(datasource) for datasource in datasources],
"generated_at": to_iso8601_utc(datetime.now(UTC)),
}

View File

@@ -4,7 +4,7 @@ Unified API for all visualization data sources.
Returns GeoJSON format compatible with Three.js, CesiumJS, and Unreal Cesium.
"""
from datetime import UTC, datetime
from datetime import UTC, datetime, timedelta
import math
import httpx
from fastapi import APIRouter, HTTPException, Depends, Query, Response
@@ -20,6 +20,7 @@ from app.db.session import get_db
from app.models.bgp_anomaly import BGPAnomaly
from app.models.bgp_incident import BGPIncident
from app.models.collected_data import CollectedData
from app.models.vessel import VesselPosition, VesselStatic
from app.services.bgp_collectors import build_bgp_collector_coverage
from app.services.cable_graph import build_graph_from_data, CableGraph, haversine_distance
from app.services.collectors.bgp_common import RIPE_RIS_COLLECTOR_COORDS
@@ -511,7 +512,7 @@ def _normalize_capacity_band(capacity_value: Optional[float], capacity_unit: str
if unit in {"pflop/s", "pflops", "pflop"}:
normalized_tflops = capacity_value * 1000
elif unit in {"gflop/s", "gflops", "gflop"}:
normalized_tflops = capacity_value / 1000
normalized_tflops = capacity_value
else:
normalized_tflops = capacity_value
@@ -609,6 +610,108 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
return {"type": "FeatureCollection", "features": features}
VESSEL_TYPE_FILTERS = {
"cargo": lambda props: str(props.get("vessel_type_name", "")).lower() == "cargo"
or 70 <= int(props.get("vessel_type") or -1) <= 79,
"tanker": lambda props: str(props.get("vessel_type_name", "")).lower() == "tanker"
or 80 <= int(props.get("vessel_type") or -1) <= 89,
"passenger": lambda props: str(props.get("vessel_type_name", "")).lower() == "passenger"
or 60 <= int(props.get("vessel_type") or -1) <= 69,
"fishing": lambda props: str(props.get("vessel_type_name", "")).lower() == "fishing"
or int(props.get("vessel_type") or -1) == 30,
"military": lambda props: str(props.get("vessel_type_name", "")).lower() == "military"
or int(props.get("vessel_type") or -1) == 35,
"other": lambda props: str(props.get("vessel_type_name", "")).lower()
not in {"cargo", "tanker", "passenger", "fishing", "military"},
}
def convert_vessels_to_geojson(rows: List[Any]) -> Dict[str, Any]:
features = []
for position, static in rows:
if position.lat is None or position.lon is None:
continue
props = {
"mmsi": position.mmsi,
"name": getattr(static, "name", None) or f"MMSI {position.mmsi}",
"callsign": getattr(static, "callsign", None),
"imo": getattr(static, "imo", None),
"vessel_type": getattr(static, "vessel_type", None),
"vessel_type_name": getattr(static, "vessel_type_name", None) or "Other",
"flag": getattr(static, "flag", None),
"length": getattr(static, "length", None),
"width": getattr(static, "width", None),
"draught": getattr(static, "draught", None),
"sog": position.sog,
"cog": position.cog,
"heading": position.heading,
"nav_status": position.nav_status,
"received_at": to_iso8601_utc(position.received_at),
"data_type": "vessel",
}
features.append(
{
"type": "Feature",
"id": position.mmsi,
"geometry": {
"type": "Point",
"coordinates": [position.lon, position.lat],
},
"properties": props,
}
)
return {"type": "FeatureCollection", "features": features}
def _parse_bbox(value: Optional[str]) -> tuple[float, float, float, float] | None:
if not value:
return None
parts = [part.strip() for part in value.split(",")]
if len(parts) != 4:
raise HTTPException(status_code=400, detail="bbox must be lon_min,lat_min,lon_max,lat_max")
try:
lon_min, lat_min, lon_max, lat_max = [float(part) for part in parts]
except ValueError as exc:
raise HTTPException(status_code=400, detail="bbox values must be numbers") from exc
if lat_min > lat_max:
lat_min, lat_max = lat_max, lat_min
if lon_min > lon_max:
lon_min, lon_max = lon_max, lon_min
return lon_min, lat_min, lon_max, lat_max
def _matches_vessel_type(props: dict[str, Any], requested_types: set[str]) -> bool:
if not requested_types:
return True
for requested_type in requested_types:
predicate = VESSEL_TYPE_FILTERS.get(requested_type)
if predicate and predicate(props):
return True
return False
def _build_vessel_stats(features: List[dict[str, Any]]) -> dict[str, Any]:
by_type: dict[str, int] = {}
underway = 0
anchored_or_moored = 0
for feature in features:
props = feature.get("properties", {})
vessel_type = str(props.get("vessel_type_name") or "Other")
by_type[vessel_type] = by_type.get(vessel_type, 0) + 1
nav_status = props.get("nav_status")
if nav_status in (1, 5):
anchored_or_moored += 1
else:
underway += 1
return {
"total": len(features),
"by_type": by_type,
"underway": underway,
"anchored_or_moored": anchored_or_moored,
}
def convert_bgp_anomalies_to_geojson(
records: List[BGPAnomaly],
geography_hints: Optional[Dict[str, Dict[str, Any]]] = None,
@@ -1298,6 +1401,137 @@ async def get_compute_centers_geojson(
}
@router.get("/geo/vessels")
async def get_vessels_geojson(
bbox: Optional[str] = Query(
None,
description="Viewport bbox as lon_min,lat_min,lon_max,lat_max",
),
type: Optional[str] = Query(
None,
description="Comma-separated vessel types: cargo,tanker,passenger,fishing,military,other",
),
limit: int = Query(5000, ge=1, le=50000),
db: AsyncSession = Depends(get_db),
):
"""Return latest vessel positions as GeoJSON points."""
latest_times = (
select(
VesselPosition.mmsi.label("mmsi"),
func.max(VesselPosition.received_at).label("received_at"),
)
.group_by(VesselPosition.mmsi)
.subquery()
)
stmt = (
select(VesselPosition, VesselStatic)
.join(
latest_times,
(VesselPosition.mmsi == latest_times.c.mmsi)
& (VesselPosition.received_at == latest_times.c.received_at),
)
.outerjoin(VesselStatic, VesselStatic.mmsi == VesselPosition.mmsi)
.order_by(VesselPosition.received_at.desc())
.limit(limit)
)
parsed_bbox = _parse_bbox(bbox)
if parsed_bbox is not None:
lon_min, lat_min, lon_max, lat_max = parsed_bbox
stmt = stmt.where(
VesselPosition.lon >= lon_min,
VesselPosition.lon <= lon_max,
VesselPosition.lat >= lat_min,
VesselPosition.lat <= lat_max,
)
result = await db.execute(stmt)
rows = list(result.all())
geojson = convert_vessels_to_geojson(rows)
requested_types = {
item.strip().lower()
for item in (type or "").split(",")
if item.strip()
}
if requested_types:
geojson["features"] = [
feature
for feature in geojson.get("features", [])
if _matches_vessel_type(feature.get("properties", {}), requested_types)
]
features = geojson.get("features", [])
return {
**geojson,
"count": len(features),
"stats": _build_vessel_stats(features),
}
@router.get("/vessels/{mmsi}")
async def get_vessel_detail(mmsi: int, db: AsyncSession = Depends(get_db)):
latest_position_stmt = (
select(VesselPosition)
.where(VesselPosition.mmsi == mmsi)
.order_by(VesselPosition.received_at.desc())
.limit(1)
)
static = await db.get(VesselStatic, mmsi)
result = await db.execute(latest_position_stmt)
position = result.scalar_one_or_none()
if position is None:
raise HTTPException(status_code=404, detail="Vessel not found")
geojson = convert_vessels_to_geojson([(position, static)])
return {
**(geojson["features"][0]["properties"]),
"latitude": position.lat,
"longitude": position.lon,
}
@router.get("/vessels/{mmsi}/track")
async def get_vessel_track(
mmsi: int,
hours: int = Query(6, ge=1, le=24),
db: AsyncSession = Depends(get_db),
):
cutoff = datetime.now(UTC) - timedelta(hours=hours)
result = await db.execute(
select(VesselPosition)
.where(VesselPosition.mmsi == mmsi)
.where(VesselPosition.received_at >= cutoff)
.order_by(VesselPosition.received_at.asc())
)
positions = list(result.scalars().all())
if not positions:
return {
"type": "FeatureCollection",
"features": [],
"count": 0,
}
return {
"type": "FeatureCollection",
"features": [
{
"type": "Feature",
"geometry": {
"type": "LineString",
"coordinates": [[position.lon, position.lat] for position in positions],
},
"properties": {
"mmsi": mmsi,
"hours": hours,
"point_count": len(positions),
"start_at": to_iso8601_utc(positions[0].received_at),
"end_at": to_iso8601_utc(positions[-1].received_at),
},
}
],
"count": 1,
}
@router.get("/geo/bgp-anomalies")
async def get_bgp_anomalies_geojson(
severity: Optional[str] = Query(None),
@@ -1353,6 +1587,76 @@ async def get_bgp_collectors_geojson(db: AsyncSession = Depends(get_db)):
return {**geojson, "count": len(geojson.get("features", []))}
@router.get("/geo/summary")
async def get_visualization_geo_summary(db: AsyncSession = Depends(get_db)):
"""Return lightweight Earth HUD counts without loading layer GeoJSON payloads."""
records_by_source = await _load_current_collected_data_by_sources(
db,
[
"arcgis_cables",
"arcgis_landing_points",
"celestrak_tle",
"top500",
"epoch_ai_gpu",
],
)
cables = convert_cable_to_geojson(records_by_source.get("arcgis_cables", []))
landing_points = convert_landing_point_to_geojson(
records_by_source.get("arcgis_landing_points", []),
)
satellites = convert_satellite_to_geojson(
_filter_known_records(records_by_source.get("celestrak_tle", [])),
)
compute_centers = convert_compute_centers_to_geojson(
_filter_known_records(
records_by_source.get("top500", [])
+ records_by_source.get("epoch_ai_gpu", []),
),
)
compute_features = compute_centers.get("features", [])
active_incident_result = await db.execute(
select(func.count(BGPIncident.id)).where(BGPIncident.status == "active"),
)
active_anomaly_result = await db.execute(
select(func.count(BGPAnomaly.id)).where(BGPAnomaly.status == "active"),
)
active_incident_count = int(active_incident_result.scalar() or 0)
active_anomaly_count = int(active_anomaly_result.scalar() or 0)
bgp_collectors = await build_bgp_collector_coverage(
db,
source_filter=("ris_live_bgp", "bgpstream_bgp"),
)
vessel_count_result = await db.execute(
select(func.count(func.distinct(VesselPosition.mmsi))),
)
vessel_count = int(vessel_count_result.scalar() or 0)
return {
"generated_at": to_iso8601_utc(datetime.now(UTC)),
"stats": {
"cable_count": len(cables.get("features", [])),
"landing_point_count": len(landing_points.get("features", [])),
"satellite_count": len(satellites.get("features", [])),
"compute_center_count": len(compute_features),
"vessel_count": vessel_count,
"supercomputer_count": sum(
1 for feature in compute_features
if feature.get("properties", {}).get("site_type") == "supercomputer"
),
"gpu_cluster_count": sum(
1 for feature in compute_features
if feature.get("properties", {}).get("site_type") == "gpu_cluster"
),
"bgp_event_count": active_incident_count or active_anomaly_count,
"bgp_incident_count": active_incident_count,
"bgp_anomaly_count": active_anomaly_count,
"bgp_collector_count": len([item for item in bgp_collectors if item.get("collector")]),
},
}
@router.get("/all")
async def get_all_visualization_data(db: AsyncSession = Depends(get_db)):
"""获取所有可视化数据的统一端点

View File

@@ -31,6 +31,7 @@ COLLECTOR_URL_KEYS = {
"opengeofeed_prefix_geo": "opengeofeed.public_csv_url",
"nro_delegated_prefix_geo": "nro.delegated_stats_url",
"news_live_streams": "news_live_streams.channels_url",
"barentswatch_vessels": "barentswatch_vessels.url",
}

View File

@@ -94,3 +94,7 @@ news_live_streams:
streams_url: "https://iptv-org.github.io/api/streams.json"
# IPTV-org 台标 JSON
logos_url: "https://iptv-org.github.io/api/logos.json"
barentswatch_vessels:
# BarentsWatch Live AIS latest combined endpoint. Requires an AIS bearer token.
url: "https://live.ais.barentswatch.no/v1/latest/combined"

View File

@@ -4,163 +4,246 @@ DEFAULT_DATASOURCES = {
"top500": {
"id": 1,
"name": "TOP500 Supercomputers",
"display_name": "TOP500 超算榜单",
"module": "L1",
"priority": "P0",
"frequency_minutes": 240,
"is_free": True,
"requires_credentials": False,
},
"epoch_ai_gpu": {
"id": 2,
"name": "Epoch AI GPU Clusters",
"display_name": "Epoch AI GPU 集群",
"module": "L1",
"priority": "P0",
"frequency_minutes": 360,
"is_free": True,
"requires_credentials": False,
},
"huggingface_models": {
"id": 3,
"name": "HuggingFace Models",
"display_name": "Hugging Face 模型",
"module": "L2",
"priority": "P1",
"frequency_minutes": 720,
"is_free": True,
"requires_credentials": False,
},
"huggingface_datasets": {
"id": 4,
"name": "HuggingFace Datasets",
"display_name": "Hugging Face 数据集",
"module": "L2",
"priority": "P1",
"frequency_minutes": 720,
"is_free": True,
"requires_credentials": False,
},
"huggingface_spaces": {
"id": 5,
"name": "HuggingFace Spaces",
"display_name": "Hugging Face Spaces",
"module": "L2",
"priority": "P2",
"frequency_minutes": 1440,
"is_free": True,
"requires_credentials": False,
},
"peeringdb_ixp": {
"id": 6,
"name": "PeeringDB IXP",
"display_name": "PeeringDB 交换中心",
"module": "L2",
"priority": "P1",
"frequency_minutes": 1440,
"is_free": True,
"requires_credentials": False,
},
"peeringdb_network": {
"id": 7,
"name": "PeeringDB Networks",
"display_name": "PeeringDB 网络",
"module": "L2",
"priority": "P2",
"frequency_minutes": 2880,
"is_free": True,
"requires_credentials": False,
},
"peeringdb_facility": {
"id": 8,
"name": "PeeringDB Facilities",
"display_name": "PeeringDB 设施",
"module": "L2",
"priority": "P2",
"frequency_minutes": 2880,
"is_free": True,
"requires_credentials": False,
},
"telegeography_cables": {
"id": 9,
"name": "Submarine Cables",
"display_name": "海底光缆",
"module": "L2",
"priority": "P1",
"frequency_minutes": 10080,
"is_free": True,
"requires_credentials": False,
},
"telegeography_landing": {
"id": 10,
"name": "Cable Landing Points",
"display_name": "光缆登陆点",
"module": "L2",
"priority": "P2",
"frequency_minutes": 10080,
"is_free": True,
"requires_credentials": False,
},
"telegeography_systems": {
"id": 11,
"name": "Cable Systems",
"display_name": "光缆系统",
"module": "L2",
"priority": "P2",
"frequency_minutes": 10080,
"is_free": True,
"requires_credentials": False,
},
"arcgis_cables": {
"id": 15,
"name": "ArcGIS Submarine Cables",
"display_name": "ArcGIS 海底光缆",
"module": "L2",
"priority": "P1",
"frequency_minutes": 10080,
"is_free": True,
"requires_credentials": False,
},
"arcgis_landing_points": {
"id": 16,
"name": "ArcGIS Landing Points",
"display_name": "ArcGIS 登陆点",
"module": "L2",
"priority": "P1",
"frequency_minutes": 10080,
"is_free": True,
"requires_credentials": False,
},
"arcgis_cable_landing_relation": {
"id": 17,
"name": "ArcGIS Cable-Landing Relations",
"display_name": "ArcGIS 光缆登陆关系",
"module": "L2",
"priority": "P1",
"frequency_minutes": 10080,
"is_free": True,
"requires_credentials": False,
},
"fao_landing_points": {
"id": 18,
"name": "FAO Landing Points",
"display_name": "FAO 登陆点",
"module": "L2",
"priority": "P1",
"frequency_minutes": 10080,
"is_free": True,
"requires_credentials": False,
},
"spacetrack_tle": {
"id": 19,
"name": "Space-Track TLE",
"display_name": "Space-Track 轨道根数",
"module": "L3",
"priority": "P2",
"frequency_minutes": 1440,
"is_free": True,
"requires_credentials": True,
"credential_provider": "spacetrack",
"credential_status": "planned",
},
"celestrak_tle": {
"id": 20,
"name": "CelesTrak TLE",
"display_name": "CelesTrak 轨道根数",
"module": "L3",
"priority": "P2",
"frequency_minutes": 1440,
"is_free": True,
"requires_credentials": False,
},
"ris_live_bgp": {
"id": 21,
"name": "RIPE RIS Live BGP",
"display_name": "RIPE RIS 实时 BGP",
"module": "L3",
"priority": "P1",
"frequency_minutes": 15,
"is_free": True,
"requires_credentials": False,
},
"bgpstream_bgp": {
"id": 22,
"name": "CAIDA BGPStream Backfill",
"display_name": "CAIDA BGPStream 回填",
"module": "L3",
"priority": "P1",
"frequency_minutes": 360,
"is_free": True,
"requires_credentials": False,
},
"iptoasn_prefix_geo": {
"id": 23,
"name": "IPtoASN Prefix Geography",
"display_name": "IPtoASN 前缀地理",
"module": "L3",
"priority": "P1",
"frequency_minutes": 1440,
"is_free": True,
"requires_credentials": False,
},
"opengeofeed_prefix_geo": {
"id": 24,
"name": "OpenGeoFeed Prefix Geography",
"display_name": "OpenGeoFeed 前缀地理",
"module": "L3",
"priority": "P1",
"frequency_minutes": 1440,
"is_free": True,
"requires_credentials": False,
},
"nro_delegated_prefix_geo": {
"id": 25,
"name": "NRO Delegated Prefix Geography",
"display_name": "NRO 分配前缀地理",
"module": "L3",
"priority": "P1",
"frequency_minutes": 1440,
"is_free": True,
"requires_credentials": False,
},
"news_live_streams": {
"id": 26,
"name": "News Live Streams",
"display_name": "新闻直播源",
"module": "L4",
"priority": "P2",
"frequency_minutes": 720,
"is_free": True,
"requires_credentials": False,
},
"barentswatch_vessels": {
"id": 27,
"name": "BarentsWatch AIS Vessels",
"display_name": "BarentsWatch AIS 船舶",
"module": "L4",
"priority": "P1",
"frequency_minutes": 1,
"is_free": True,
"requires_credentials": True,
"credential_provider": "barentswatch",
"credential_status": "supported",
},
}

View File

@@ -0,0 +1,151 @@
"""Registry of target schemas supported by mapped custom data sources."""
from __future__ import annotations
from dataclasses import dataclass
from datetime import datetime
from typing import Any
from pydantic import BaseModel, Field, ValidationError, field_validator
class VesselAISRecord(BaseModel):
mmsi: int = Field(ge=100000000, le=999999999)
lat: float = Field(ge=-90, le=90)
lon: float = Field(ge=-180, le=180)
sog: float | None = None
cog: float | None = Field(default=None, ge=0, le=360)
heading: int | None = Field(default=None, ge=0, le=511)
name: str | None = None
vessel_type: str | int | None = None
received_at: datetime | None = None
class GeoPointRecord(BaseModel):
lat: float = Field(ge=-90, le=90)
lon: float = Field(ge=-180, le=180)
name: str | None = None
type: str | None = None
source_id: str | None = None
observed_at: datetime | None = None
metadata: dict[str, Any] = Field(default_factory=dict)
class GenericRecord(BaseModel):
data: dict[str, Any] = Field(default_factory=dict)
source_id: str | None = None
observed_at: datetime | None = None
@field_validator("data")
@classmethod
def require_payload(cls, value: dict[str, Any]) -> dict[str, Any]:
if not value:
raise ValueError("generic_records requires a non-empty data object")
return value
@dataclass(frozen=True)
class TargetField:
name: str
type: str
required: bool = False
description: str = ""
example: Any = None
def to_dict(self) -> dict[str, Any]:
return {
"name": self.name,
"type": self.type,
"required": self.required,
"description": self.description,
"example": self.example,
}
@dataclass(frozen=True)
class TargetSchema:
key: str
label: str
description: str
fields: tuple[TargetField, ...]
model: type[BaseModel]
destination: str
def to_dict(self) -> dict[str, Any]:
return {
"key": self.key,
"label": self.label,
"description": self.description,
"destination": self.destination,
"fields": [field.to_dict() for field in self.fields],
}
def validate_record(self, record: dict[str, Any]) -> tuple[dict[str, Any] | None, list[str]]:
try:
return self.model.model_validate(record).model_dump(mode="json"), []
except ValidationError as exc:
return None, [
".".join(str(part) for part in error["loc"]) + f": {error['msg']}"
for error in exc.errors()
]
TARGET_SCHEMAS: dict[str, TargetSchema] = {
"vessel_ais": TargetSchema(
key="vessel_ais",
label="船舶 AIS",
description="船只位置、航速、航向、MMSI 等 AIS 数据。",
destination="vessel_position",
model=VesselAISRecord,
fields=(
TargetField("mmsi", "integer", True, "MMSI 九位船舶标识", 257123000),
TargetField("lat", "float", True, "纬度", 59.91),
TargetField("lon", "float", True, "经度", 10.75),
TargetField("sog", "float", False, "对地航速,单位节", 12.4),
TargetField("cog", "float", False, "对地航向0-360 度", 184.5),
TargetField("heading", "integer", False, "船首向0-511", 186),
TargetField("name", "string", False, "船名", "OSLO EXPRESS"),
TargetField("vessel_type", "string", False, "船型", "cargo"),
TargetField("received_at", "datetime", False, "数据接收时间", "2026-04-28T00:00:00Z"),
),
),
"geo_points": TargetSchema(
key="geo_points",
label="通用地理点",
description="带经纬度的通用实体或事件点位。",
destination="generic_geo_points",
model=GeoPointRecord,
fields=(
TargetField("lat", "float", True, "纬度", 1.3),
TargetField("lon", "float", True, "经度", 103.8),
TargetField("name", "string", False, "点位名称", "Singapore"),
TargetField("type", "string", False, "点位类型", "datacenter"),
TargetField("source_id", "string", False, "来源侧 ID", "sg-1"),
TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
TargetField("metadata", "object", False, "扩展字段", {"provider": "example"}),
),
),
"generic_records": TargetSchema(
key="generic_records",
label="通用结构化记录",
description="未知结构数据沉淀,不直接进入 Earth 图层。",
destination="collected_data",
model=GenericRecord,
fields=(
TargetField("data", "object", True, "结构化记录主体", {"raw": "value"}),
TargetField("source_id", "string", False, "来源侧 ID", "record-1"),
TargetField("observed_at", "datetime", False, "观测时间", "2026-04-28T00:00:00Z"),
),
),
}
def list_target_schemas() -> list[dict[str, Any]]:
return [schema.to_dict() for schema in TARGET_SCHEMAS.values()]
def get_target_schema(key: str) -> TargetSchema:
try:
return TARGET_SCHEMAS[key]
except KeyError as exc:
raise ValueError(f"Unsupported target schema: {key}") from exc

View File

@@ -110,6 +110,8 @@ async def init_db():
import app.models.playground_session # noqa: F401
import app.models.playground_message # noqa: F401
import app.models.system_log # noqa: F401
import app.models.vessel # noqa: F401
import app.models.datasource_mapping # noqa: F401
logger.warning_event(
"Database pool settings active",

View File

@@ -12,6 +12,8 @@ from app.models.system_setting import SystemSetting
from app.models.playground_session import PlaygroundSession
from app.models.playground_message import PlaygroundMessage
from app.models.system_log import SystemLog, AuditLog
from app.models.vessel import VesselPosition, VesselStatic
from app.models.datasource_mapping import DataSourceMappingTemplate
__all__ = [
"User",
@@ -29,4 +31,7 @@ __all__ = [
"BGPObservation",
"SystemLog",
"AuditLog",
"VesselPosition",
"VesselStatic",
"DataSourceMappingTemplate",
]

View File

@@ -0,0 +1,32 @@
"""Mapping templates for user-defined data source payloads."""
from sqlalchemy import Boolean, Column, DateTime, ForeignKey, Integer, JSON, String
from sqlalchemy.sql import func
from app.db.session import Base
class DataSourceMappingTemplate(Base):
__tablename__ = "datasource_mapping_templates"
id = Column(Integer, primary_key=True, autoincrement=True)
datasource_config_id = Column(
Integer,
ForeignKey("datasource_configs.id"),
nullable=False,
index=True,
)
target_schema = Column(String(80), nullable=False, index=True)
mapping_json = Column(JSON, nullable=False, default={})
sample_payload_hash = Column(String(64), nullable=True)
validation_status = Column(String(30), nullable=False, default="draft")
version = Column(Integer, nullable=False, default=1)
is_active = Column(Boolean, nullable=False, default=False, index=True)
created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())
def __repr__(self):
return (
f"<DataSourceMappingTemplate {self.id}: "
f"{self.datasource_config_id}/{self.target_schema}/v{self.version}>"
)

View File

@@ -0,0 +1,75 @@
"""Vessel AIS models for live maritime tracking."""
from sqlalchemy import BigInteger, Column, DateTime, Float, Index, Integer, SmallInteger, String
from sqlalchemy.sql import func
from app.core.time import to_iso8601_utc
from app.db.session import Base
class VesselStatic(Base):
"""Slow-changing vessel identity and dimensions."""
__tablename__ = "vessel_static"
mmsi = Column(BigInteger, primary_key=True)
name = Column(String(128), nullable=True)
callsign = Column(String(16), nullable=True)
vessel_type = Column(SmallInteger, nullable=True, index=True)
vessel_type_name = Column(String(64), nullable=True, index=True)
flag = Column(String(4), nullable=True, index=True)
length = Column(Float, nullable=True)
width = Column(Float, nullable=True)
draught = Column(Float, nullable=True)
imo = Column(BigInteger, nullable=True)
updated_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now())
def to_dict(self) -> dict:
return {
"mmsi": self.mmsi,
"name": self.name,
"callsign": self.callsign,
"vessel_type": self.vessel_type,
"vessel_type_name": self.vessel_type_name,
"flag": self.flag,
"length": self.length,
"width": self.width,
"draught": self.draught,
"imo": self.imo,
"updated_at": to_iso8601_utc(self.updated_at),
}
class VesselPosition(Base):
"""Append-only AIS positions retained for short history windows."""
__tablename__ = "vessel_position"
id = Column(Integer, primary_key=True, autoincrement=True)
mmsi = Column(BigInteger, nullable=False, index=True)
lat = Column(Float, nullable=False)
lon = Column(Float, nullable=False)
sog = Column(Float, nullable=True)
cog = Column(Float, nullable=True)
heading = Column(SmallInteger, nullable=True)
nav_status = Column(SmallInteger, nullable=True, index=True)
received_at = Column(DateTime(timezone=True), nullable=False, server_default=func.now(), index=True)
__table_args__ = (
Index("idx_vessel_pos_mmsi_time", "mmsi", "received_at"),
Index("idx_vessel_pos_time", "received_at"),
Index("idx_vessel_pos_lat_lon", "lat", "lon"),
)
def to_dict(self) -> dict:
return {
"id": self.id,
"mmsi": self.mmsi,
"lat": self.lat,
"lon": self.lon,
"sog": self.sog,
"cog": self.cog,
"heading": self.heading,
"nav_status": self.nav_status,
"received_at": to_iso8601_utc(self.received_at),
}

View File

@@ -3,9 +3,11 @@ from __future__ import annotations
import asyncio
import httpx
from fastapi import HTTPException, status
from fastapi import Depends, HTTPException, status
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import settings
from app.db.session import get_db
from app.schemas.ai import (
AIProviderStatusResponse,
SituationalAnalysisRequest,
@@ -14,11 +16,27 @@ from app.schemas.ai import (
class AIProviderClient:
def __init__(self) -> None:
self.service_url = settings.AI_PROVIDER_SERVICE_URL.rstrip("/")
self.service_token = settings.AI_PROVIDER_SERVICE_TOKEN
self.timeout = settings.AI_PROVIDER_TIMEOUT_SECONDS
self.retry_attempts = max(settings.AI_PROVIDER_RETRY_ATTEMPTS, 1)
def __init__(
self,
*,
service_url: str | None = None,
service_token: str | None = None,
timeout: int | None = None,
retry_attempts: int | None = None,
llm_config: dict | None = None,
) -> None:
self.service_url = (
service_url if service_url is not None else settings.AI_PROVIDER_SERVICE_URL
).rstrip("/")
self.service_token = (
service_token if service_token is not None else settings.AI_PROVIDER_SERVICE_TOKEN
)
self.timeout = timeout if timeout is not None else settings.AI_PROVIDER_TIMEOUT_SECONDS
self.retry_attempts = max(
retry_attempts if retry_attempts is not None else settings.AI_PROVIDER_RETRY_ATTEMPTS,
1,
)
self.llm_config = llm_config or {}
def _headers(self, request_id: str | None = None) -> dict[str, str]:
headers = {"Content-Type": "application/json"}
@@ -26,6 +44,19 @@ class AIProviderClient:
headers["X-Provider-Token"] = self.service_token
if request_id:
headers["X-Request-ID"] = request_id
llm_header_map = {
"provider": "X-AI-Provider",
"provider_api": "X-AI-Provider-API",
"base_url": "X-AI-Base-URL",
"api_key": "X-AI-API-Key",
"model": "X-AI-Model",
"max_tokens": "X-AI-Max-Tokens",
"anthropic_version": "X-AI-Anthropic-Version",
}
for key, header_name in llm_header_map.items():
value = self.llm_config.get(key)
if value not in (None, ""):
headers[header_name] = str(value)
return headers
async def get_status(self, request_id: str | None = None) -> AIProviderStatusResponse:
@@ -105,5 +136,14 @@ class AIProviderClient:
)
def get_ai_provider_client() -> AIProviderClient:
return AIProviderClient()
async def get_ai_provider_client(db: AsyncSession = Depends(get_db)) -> AIProviderClient:
from app.api.v1.settings import get_runtime_ai_provider_config
runtime_config = await get_runtime_ai_provider_config(db)
return AIProviderClient(
service_url=runtime_config["service_url"],
service_token=runtime_config["service_token"],
timeout=runtime_config["timeout_seconds"],
retry_attempts=runtime_config["retry_attempts"],
llm_config=runtime_config.get("llm_config") or {},
)

View File

@@ -36,6 +36,7 @@ from app.services.collectors.iptoasn import IPtoASNPrefixGeoCollector
from app.services.collectors.opengeofeed import OpenGeoFeedPrefixGeoCollector
from app.services.collectors.nro_delegated import NRODelegatedPrefixGeoCollector
from app.services.collectors.news_live_streams import NewsLiveStreamsCollector
from app.services.collectors.vessel_ais import VesselAISCollector
collector_registry.register(TOP500Collector())
collector_registry.register(EpochAIGPUCollector())
@@ -63,3 +64,4 @@ collector_registry.register(IPtoASNPrefixGeoCollector())
collector_registry.register(OpenGeoFeedPrefixGeoCollector())
collector_registry.register(NRODelegatedPrefixGeoCollector())
collector_registry.register(NewsLiveStreamsCollector())
collector_registry.register(VesselAISCollector())

View File

@@ -0,0 +1,326 @@
"""BarentsWatch AIS collector for vessel tracking."""
from datetime import UTC, datetime, timedelta
import os
from typing import Any
import httpx
from sqlalchemy import delete, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.models.vessel import VesselPosition, VesselStatic
from app.services.collectors.base import BaseCollector
BARENTSWATCH_LATEST_URL = "https://live.ais.barentswatch.no/v1/latest/combined"
BARENTSWATCH_TOKEN_URL = "https://id.barentswatch.no/connect/token"
VESSEL_TYPE_NAMES = {
30: "Fishing",
35: "Military",
60: "Passenger",
70: "Cargo",
80: "Tanker",
}
class VesselAISCollector(BaseCollector):
"""Collect latest AIS positions and append them to vessel tables."""
name = "barentswatch_vessels"
priority = "P1"
module = "L4"
frequency_hours = 1
data_type = "vessel_ais"
@property
def base_url(self) -> str:
return self._resolved_url or BARENTSWATCH_LATEST_URL
async def _load_datasource_config(self) -> dict[str, Any]:
if not self._db_session:
return {}
try:
from sqlalchemy import select
from app.models.datasource_config import DataSourceConfig
result = await self._db_session.execute(
select(DataSourceConfig)
.where(DataSourceConfig.name == self.name)
.where(DataSourceConfig.is_active.is_(True))
)
datasource_config = result.scalar_one_or_none()
except Exception:
return {}
if not datasource_config:
return {}
return {
"auth_config": datasource_config.auth_config or {},
"config": datasource_config.config or {},
}
async def _get_access_token(self, client: httpx.AsyncClient) -> str | None:
datasource_config = await self._load_datasource_config()
auth_config = datasource_config.get("auth_config") or {}
config = datasource_config.get("config") or {}
client_id = (
auth_config.get("client_id")
or config.get("client_id")
or os.getenv("BARENTSWATCH_CLIENT_ID")
or os.getenv("BARRENTSWATCH_CLIENT_ID")
)
client_secret = (
auth_config.get("client_secret")
or config.get("client_secret")
or os.getenv("BARENTSWATCH_CLIENT_SECRET")
or os.getenv("BARRENTSWATCH_CLIENT_SECRET")
)
if not client_id or not client_secret:
return None
response = await client.post(
BARENTSWATCH_TOKEN_URL,
data={
"client_id": client_id,
"client_secret": client_secret,
"scope": "ais",
"grant_type": "client_credentials",
},
headers={"Content-Type": "application/x-www-form-urlencoded"},
)
response.raise_for_status()
payload = response.json()
token = payload.get("access_token")
return str(token) if token else None
async def fetch(self) -> list[dict[str, Any]]:
async with httpx.AsyncClient(timeout=60.0) as client:
headers: dict[str, str] = {}
token = await self._get_access_token(client)
if token:
headers["Authorization"] = f"Bearer {token}"
response = await client.get(self.base_url, headers=headers)
if response.status_code == 401 and not token:
return self._get_sample_data()
response.raise_for_status()
payload = response.json()
if isinstance(payload, list):
return [item for item in payload if isinstance(item, dict)]
if isinstance(payload, dict):
for key in ("features", "data", "items", "vessels"):
value = payload.get(key)
if isinstance(value, list):
if key == "features":
return [
{
**(item.get("properties") or {}),
"geometry": item.get("geometry"),
}
for item in value
if isinstance(item, dict)
]
return [item for item in value if isinstance(item, dict)]
return self._get_sample_data()
def transform(self, raw_data: list[dict[str, Any]]) -> list[dict[str, Any]]:
transformed = []
for item in raw_data:
record = self._normalize_record(item)
if record:
transformed.append(record)
return transformed
async def _save_data(
self,
db: AsyncSession,
data: list[dict[str, Any]],
task_id: int | None = None,
snapshot_id: int | None = None,
) -> int:
now = datetime.now(UTC)
records_added = 0
for index, item in enumerate(data):
static = await db.get(VesselStatic, item["mmsi"])
if static is None:
static = VesselStatic(mmsi=item["mmsi"])
db.add(static)
for field in (
"name",
"callsign",
"vessel_type",
"vessel_type_name",
"flag",
"length",
"width",
"draught",
"imo",
):
value = item.get(field)
if value not in (None, ""):
setattr(static, field, value)
static.updated_at = now
db.add(
VesselPosition(
mmsi=item["mmsi"],
lat=item["lat"],
lon=item["lon"],
sog=item.get("sog"),
cog=item.get("cog"),
heading=item.get("heading"),
nav_status=item.get("nav_status"),
received_at=item.get("received_at") or now,
)
)
records_added += 1
if (index + 1) % 1000 == 0:
await self.update_progress(index + 1, commit=True)
await db.execute(
delete(VesselPosition).where(VesselPosition.received_at < now - timedelta(hours=24))
)
await db.commit()
await self.update_progress(records_added, force=True)
return records_added
def _normalize_record(self, item: dict[str, Any]) -> dict[str, Any] | None:
mmsi = _as_int(_pick(item, "mmsi", "MMSI", "Mmsi"))
lat = _as_float(_pick(item, "lat", "latitude", "Latitude"))
lon = _as_float(_pick(item, "lon", "lng", "longitude", "Longitude"))
geometry = item.get("geometry")
coordinates = geometry.get("coordinates") if isinstance(geometry, dict) else None
if (lat is None or lon is None) and isinstance(coordinates, list) and len(coordinates) >= 2:
lon = _as_float(coordinates[0])
lat = _as_float(coordinates[1])
if mmsi is None or lat is None or lon is None:
return None
if not (-90 <= lat <= 90 and -180 <= lon <= 180):
return None
vessel_type = _as_int(_pick(item, "vessel_type", "shipType", "ship_type", "ShipType"))
vessel_type_name = (
_pick(item, "vessel_type_name", "shipTypeName", "ship_type_name", "VesselTypeName")
or _vessel_type_name(vessel_type)
)
received_at = _parse_datetime(_pick(item, "received_at", "timestamp", "time", "msgtime"))
return {
"mmsi": mmsi,
"name": _pick(item, "name", "shipName", "ship_name", "Name"),
"callsign": _pick(item, "callsign", "callSign", "CallSign"),
"vessel_type": vessel_type,
"vessel_type_name": vessel_type_name,
"flag": _pick(item, "flag", "country", "Flag"),
"length": _as_float(_pick(item, "length", "shipLength", "Length")),
"width": _as_float(_pick(item, "width", "shipWidth", "Width")),
"draught": _as_float(_pick(item, "draught", "draft", "Draught")),
"imo": _as_int(_pick(item, "imo", "IMO", "imoNumber")),
"lat": lat,
"lon": lon,
"sog": _as_float(_pick(item, "sog", "speedOverGround", "SOG")),
"cog": _as_float(_pick(item, "cog", "courseOverGround", "COG")),
"heading": _as_int(_pick(item, "heading", "trueHeading", "Heading")),
"nav_status": _as_int(_pick(item, "nav_status", "navStatus", "NavigationalStatus")),
"received_at": received_at,
}
def _get_sample_data(self) -> list[dict[str, Any]]:
return [
{
"mmsi": 257123000,
"name": "OSLO TRADER",
"lat": 59.91,
"lon": 10.73,
"sog": 12.4,
"cog": 214,
"heading": 215,
"nav_status": 0,
"vessel_type": 70,
"vessel_type_name": "Cargo",
"flag": "NO",
"length": 185,
},
{
"mmsi": 257456000,
"name": "NORDIC FJORD",
"lat": 60.39,
"lon": 5.32,
"sog": 0.2,
"cog": 82,
"heading": 80,
"nav_status": 1,
"vessel_type": 60,
"vessel_type_name": "Passenger",
"flag": "NO",
"length": 126,
},
]
def _pick(item: dict[str, Any], *keys: str) -> Any:
for key in keys:
if key in item and item[key] not in (None, ""):
return item[key]
return None
def _as_float(value: Any) -> float | None:
try:
if value in (None, ""):
return None
return float(value)
except (TypeError, ValueError):
return None
def _as_int(value: Any) -> int | None:
try:
if value in (None, ""):
return None
return int(float(value))
except (TypeError, ValueError):
return None
def _parse_datetime(value: Any) -> datetime | None:
if isinstance(value, datetime):
return value if value.tzinfo else value.replace(tzinfo=UTC)
if not value:
return None
if isinstance(value, (int, float)):
timestamp = float(value)
if timestamp > 10_000_000_000:
timestamp /= 1000
return datetime.fromtimestamp(timestamp, UTC)
if isinstance(value, str):
try:
parsed = datetime.fromisoformat(value.replace("Z", "+00:00"))
return parsed if parsed.tzinfo else parsed.replace(tzinfo=UTC)
except ValueError:
return None
return None
def _vessel_type_name(vessel_type: int | None) -> str:
if vessel_type is None:
return "Other"
if 70 <= vessel_type <= 79:
return "Cargo"
if 80 <= vessel_type <= 89:
return "Tanker"
if 60 <= vessel_type <= 69:
return "Passenger"
if vessel_type == 30:
return "Fishing"
if vessel_type == 35:
return "Military"
return VESSEL_TYPE_NAMES.get(vessel_type, "Other")

View File

@@ -0,0 +1,358 @@
"""Deterministic mapping support for custom data sources."""
from __future__ import annotations
import hashlib
import json
import re
from datetime import UTC, datetime
from typing import Any
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.target_schema_registry import TargetSchema, get_target_schema
SECRET_KEY_PATTERN = re.compile(
r"(token|secret|password|passwd|authorization|api[_-]?key|client[_-]?secret)",
re.IGNORECASE,
)
class MappingError(ValueError):
"""Raised when a mapping definition cannot be executed."""
def stable_payload_hash(payload: Any) -> str:
encoded = json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str).encode()
return hashlib.sha256(encoded).hexdigest()
def redact_for_llm(value: Any) -> Any:
if isinstance(value, dict):
redacted = {}
for key, item in value.items():
if SECRET_KEY_PATTERN.search(str(key)):
redacted[key] = "[REDACTED]"
else:
redacted[key] = redact_for_llm(item)
return redacted
if isinstance(value, list):
return [redact_for_llm(item) for item in value[:20]]
return value
def extract_path(payload: Any, path: str | None) -> Any:
if not path or path == "$":
return payload
normalized = path.strip()
if normalized.startswith("$."):
normalized = normalized[2:]
elif normalized.startswith("$"):
normalized = normalized[1:]
normalized = normalized.strip(".")
if not normalized:
return payload
current = payload
for raw_segment in normalized.split("."):
segment = raw_segment.strip()
if not segment:
continue
list_all = segment.endswith("[*]")
if list_all:
segment = segment[:-3]
index = None
match = re.fullmatch(r"(.+)\[(\d+)\]", segment)
if match:
segment = match.group(1)
index = int(match.group(2))
if segment:
if isinstance(current, dict):
current = current.get(segment)
else:
return None
if list_all:
return current if isinstance(current, list) else []
if index is not None:
if not isinstance(current, list) or index >= len(current):
return None
current = current[index]
return current
def _convert_value(value: Any, target_type: str | None) -> Any:
if value is None or target_type in (None, "", "any"):
return value
if target_type == "string":
return str(value)
if target_type == "integer":
return int(value)
if target_type == "float":
return float(value)
if target_type == "boolean":
if isinstance(value, bool):
return value
if isinstance(value, str):
return value.strip().lower() in {"1", "true", "yes", "y", "on"}
return bool(value)
if target_type == "datetime":
if isinstance(value, datetime):
return value
if isinstance(value, (int, float)):
return datetime.fromtimestamp(value)
if isinstance(value, str):
return datetime.fromisoformat(value.replace("Z", "+00:00"))
return value
if target_type == "object":
if isinstance(value, dict):
return value
raise ValueError("expected object")
if target_type == "array":
if isinstance(value, list):
return value
raise ValueError("expected array")
return value
def _apply_enum(value: Any, enum_map: Any) -> Any:
if not isinstance(enum_map, dict):
return value
key = str(value)
return enum_map.get(key, enum_map.get(value, value))
def _map_one(item: Any, field_mapping: dict[str, Any]) -> tuple[dict[str, Any], list[str]]:
output: dict[str, Any] = {}
errors: list[str] = []
for field_name, rule in field_mapping.items():
if isinstance(rule, str):
rule = {"path": rule}
if not isinstance(rule, dict):
errors.append(f"{field_name}: mapping rule must be an object or path string")
continue
value = extract_path(item, rule.get("path"))
if value is None and "default" in rule:
value = rule.get("default")
value = _apply_enum(value, rule.get("enum"))
try:
value = _convert_value(value, rule.get("type"))
except (TypeError, ValueError) as exc:
errors.append(f"{field_name}: failed to convert value {value!r}: {exc}")
continue
if value is not None or rule.get("include_null", False):
output[field_name] = value
return output, errors
def execute_mapping(
payload: Any,
mapping_json: dict[str, Any],
target_schema: str | TargetSchema,
*,
limit: int | None = None,
) -> dict[str, Any]:
schema = get_target_schema(target_schema) if isinstance(target_schema, str) else target_schema
source = mapping_json.get("source") or {}
fields = mapping_json.get("fields")
if not isinstance(fields, dict) or not fields:
raise MappingError("mapping_json.fields must be a non-empty object")
items_path = source.get("items_path") or mapping_json.get("items_path") or "$"
items = extract_path(payload, items_path)
if isinstance(items, dict):
items = [items]
elif not isinstance(items, list):
items = []
if limit is not None:
items = items[:limit]
mapped_records: list[dict[str, Any]] = []
errors: list[dict[str, Any]] = []
for index, item in enumerate(items):
mapped, mapping_errors = _map_one(item, fields)
validated, validation_errors = schema.validate_record(mapped)
all_errors = mapping_errors + validation_errors
if all_errors:
errors.append({"index": index, "errors": all_errors, "record": mapped})
continue
if validated is not None:
mapped_records.append(validated)
return {
"target_schema": schema.key,
"total_items": len(items),
"mapped_count": len(mapped_records),
"failed_count": len(errors),
"records": mapped_records,
"errors": errors,
}
def build_heuristic_mapping(sample_payload: Any, target_schema_key: str) -> dict[str, Any]:
schema = get_target_schema(target_schema_key)
items_path = "$"
sample_item = sample_payload
if isinstance(sample_payload, dict):
for key in ("data", "items", "results", "features", "vessels"):
candidate = sample_payload.get(key)
if isinstance(candidate, list) and candidate:
items_path = f"$.{key}[*]"
sample_item = candidate[0]
break
elif isinstance(sample_payload, list) and sample_payload:
items_path = "$"
sample_item = sample_payload[0]
available = _flatten_keys(sample_item if isinstance(sample_item, dict) else {})
fields: dict[str, Any] = {}
for field in schema.fields:
candidate = _best_field_match(field.name, available)
if candidate:
fields[field.name] = {"path": f"$.{candidate}", "type": field.type}
elif field.name == "data" and target_schema_key == "generic_records":
fields[field.name] = {"path": "$", "type": "object"}
elif not field.required:
fields[field.name] = {"path": f"$.{field.name}", "type": field.type, "default": None}
return {
"source": {"items_path": items_path},
"fields": fields,
"meta": {
"generated_by": "heuristic",
"requires_review": True,
},
}
def _flatten_keys(payload: dict[str, Any], prefix: str = "") -> list[str]:
keys: list[str] = []
for key, value in payload.items():
dotted = f"{prefix}.{key}" if prefix else str(key)
keys.append(dotted)
if isinstance(value, dict):
keys.extend(_flatten_keys(value, dotted))
return keys
def _best_field_match(field_name: str, candidates: list[str]) -> str | None:
aliases = {
"lat": ("lat", "latitude", "y"),
"lon": ("lon", "lng", "longitude", "x"),
"mmsi": ("mmsi",),
"sog": ("sog", "speed", "speedOverGround"),
"cog": ("cog", "course", "courseOverGround"),
"received_at": ("received_at", "timestamp", "time", "updated_at"),
"observed_at": ("observed_at", "timestamp", "time", "updated_at"),
"source_id": ("id", "source_id", "uuid"),
}.get(field_name, (field_name,))
lowered = {candidate.lower(): candidate for candidate in candidates}
for alias in aliases:
if alias.lower() in lowered:
return lowered[alias.lower()]
for candidate in candidates:
tail = candidate.split(".")[-1].lower()
if tail in {alias.lower() for alias in aliases}:
return candidate
return None
def _parse_datetime(value: Any) -> datetime | None:
if value is None:
return None
if isinstance(value, datetime):
return value
if isinstance(value, str):
return datetime.fromisoformat(value.replace("Z", "+00:00"))
return None
async def persist_mapped_records(
db: AsyncSession,
*,
datasource_name: str,
datasource_config_id: int,
target_schema: str,
records: list[dict[str, Any]],
mapping_version: int,
) -> int:
"""Persist validated mapped records to the destination for a target schema."""
if target_schema == "vessel_ais":
from app.models.vessel import VesselPosition
for record in records:
db.add(
VesselPosition(
mmsi=record["mmsi"],
lat=record["lat"],
lon=record["lon"],
sog=record.get("sog"),
cog=record.get("cog"),
heading=record.get("heading"),
received_at=_parse_datetime(record.get("received_at")) or datetime.now(UTC),
)
)
await db.commit()
return len(records)
from app.models.collected_data import CollectedData
collected_at = datetime.now(UTC)
for index, record in enumerate(records):
if target_schema == "geo_points":
source_id = record.get("source_id") or f"{datasource_config_id}:{index}"
name = record.get("name")
metadata = {
"latitude": record.get("lat"),
"longitude": record.get("lon"),
"type": record.get("type"),
"mapping_version": mapping_version,
"target_schema": target_schema,
**(record.get("metadata") or {}),
}
reference_date = _parse_datetime(record.get("observed_at"))
else:
source_id = record.get("source_id") or f"{datasource_config_id}:{index}"
name = None
metadata = {
"data": record.get("data") or {},
"mapping_version": mapping_version,
"target_schema": target_schema,
}
reference_date = _parse_datetime(record.get("observed_at"))
db.add(
CollectedData(
source=datasource_name,
source_id=str(source_id),
entity_key=f"{datasource_name}:{source_id}",
data_type=target_schema,
name=name,
title=name,
extra_data=metadata,
collected_at=collected_at,
reference_date=reference_date,
is_valid=1,
is_current=True,
change_type="created",
change_summary={},
)
)
await db.commit()
return len(records)

View File

@@ -0,0 +1,150 @@
"""LLM provider presets used by Settings and the runtime AI provider bridge."""
from __future__ import annotations
from typing import Any
import httpx
MODELS_DEV_URL = "https://models.dev/api.json"
FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
"minimax": {
"provider": "minimax",
"label": "MiniMax",
"provider_api": "anthropic-messages",
"base_url": "https://api.minimaxi.com/anthropic",
"model": "MiniMax-M2.7",
"models": ["MiniMax-M2.7", "MiniMax-M2.7-highspeed", "MiniMax-M2.5", "MiniMax-M2"],
"api_key_env": "MINIMAX_API_KEY",
"source": "fallback",
},
"openai": {
"provider": "openai",
"label": "OpenAI",
"provider_api": "openai-completions",
"base_url": "https://api.openai.com/v1",
"model": "gpt-5.1",
"models": ["gpt-5.1", "gpt-5.1-codex", "gpt-4.1", "gpt-4o"],
"api_key_env": "OPENAI_API_KEY",
"source": "fallback",
},
"anthropic": {
"provider": "anthropic",
"label": "Anthropic",
"provider_api": "anthropic-messages",
"base_url": "https://api.anthropic.com/v1",
"model": "claude-sonnet-4-6",
"models": ["claude-sonnet-4-6", "claude-opus-4-5", "claude-3-5-haiku-20241022"],
"api_key_env": "ANTHROPIC_API_KEY",
"source": "fallback",
},
"deepseek": {
"provider": "deepseek",
"label": "DeepSeek",
"provider_api": "openai-completions",
"base_url": "https://api.deepseek.com/v1",
"model": "deepseek-chat",
"models": ["deepseek-chat", "deepseek-reasoner"],
"api_key_env": "DEEPSEEK_API_KEY",
"source": "fallback",
},
"alibaba": {
"provider": "alibaba",
"label": "Alibaba Qwen / DashScope",
"provider_api": "openai-completions",
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
"model": "qwen3-max",
"models": ["qwen3-max", "qwen3.5-plus", "qwen-max", "qwen-plus"],
"api_key_env": "DASHSCOPE_API_KEY",
"source": "fallback",
},
"moonshotai": {
"provider": "moonshotai",
"label": "Moonshot AI / Kimi",
"provider_api": "openai-completions",
"base_url": "https://api.moonshot.ai/v1",
"model": "kimi-k2.5",
"models": ["kimi-k2.5", "kimi-k2-thinking", "kimi-k2-turbo-preview"],
"api_key_env": "MOONSHOT_API_KEY",
"source": "fallback",
},
"openrouter": {
"provider": "openrouter",
"label": "OpenRouter",
"provider_api": "openai-completions",
"base_url": "https://openrouter.ai/api/v1",
"model": "openai/gpt-5.1",
"models": ["openai/gpt-5.1", "anthropic/claude-sonnet-4.5", "qwen/qwen3-max"],
"api_key_env": "OPENROUTER_API_KEY",
"source": "fallback",
},
"ollama": {
"provider": "ollama",
"label": "Ollama Local",
"provider_api": "ollama-generate",
"base_url": "http://127.0.0.1:11434",
"model": "qwen2.5:7b",
"models": ["qwen2.5:7b", "llama3.1:8b", "mistral:7b"],
"api_key_env": "",
"source": "fallback",
},
}
MODELS_DEV_PROVIDER_KEYS = {
"minimax": "minimax",
"openai": "openai",
"anthropic": "anthropic",
"deepseek": "deepseek",
"alibaba": "alibaba",
"moonshotai": "moonshotai",
"openrouter": "openrouter",
}
def list_fallback_llm_provider_presets() -> list[dict[str, Any]]:
return [dict(value) for value in FALLBACK_LLM_PROVIDER_PRESETS.values()]
def get_fallback_llm_provider_preset(provider: str) -> dict[str, Any]:
key = provider.strip().lower()
if key not in FALLBACK_LLM_PROVIDER_PRESETS:
raise ValueError(f"Unsupported LLM provider preset: {provider}")
return dict(FALLBACK_LLM_PROVIDER_PRESETS[key])
async def refresh_llm_provider_preset(provider: str) -> dict[str, Any]:
fallback = get_fallback_llm_provider_preset(provider)
models_dev_key = MODELS_DEV_PROVIDER_KEYS.get(fallback["provider"])
if not models_dev_key:
return fallback
async with httpx.AsyncClient(timeout=15.0, follow_redirects=True) as client:
response = await client.get(
MODELS_DEV_URL,
headers={"User-Agent": "Planet/1.0"},
)
response.raise_for_status()
catalog = response.json()
upstream = catalog.get(models_dev_key)
if not isinstance(upstream, dict):
return fallback
upstream_models = upstream.get("models") if isinstance(upstream.get("models"), dict) else {}
model_ids = list(upstream_models.keys())[:80]
base_url = upstream.get("api") or fallback["base_url"]
if fallback["provider"] == "deepseek" and base_url == "https://api.deepseek.com":
base_url = "https://api.deepseek.com/v1"
refreshed = {
**fallback,
"label": upstream.get("name") or fallback["label"],
"base_url": base_url,
"model": model_ids[0] if model_ids else fallback["model"],
"models": model_ids or fallback["models"],
"api_key_env": (upstream.get("env") or [fallback["api_key_env"]])[0],
"source": MODELS_DEV_URL,
}
return refreshed

View File

@@ -0,0 +1,199 @@
from types import SimpleNamespace
import pytest
from httpx import ASGITransport, AsyncClient
from app.api.v1.datasource_config import get_ai_provider_client
from app.core.security import get_current_user
from app.core.target_schema_registry import get_target_schema, list_target_schemas
from app.main import app
from app.models.user import User
from app.services.datasource_mapping import execute_mapping, persist_mapped_records, redact_for_llm
SAMPLE_AIS = {
"data": [
{
"mmsi": "257123000",
"latitude": "59.91",
"longitude": "10.75",
"speedOverGround": "12.4",
"timestamp": "2026-04-28T00:00:00Z",
"api_token": "secret-value",
}
]
}
def test_registry_exposes_v1_target_schemas():
keys = {schema["key"] for schema in list_target_schemas()}
assert {"vessel_ais", "geo_points", "generic_records"}.issubset(keys)
assert get_target_schema("vessel_ais").destination == "vessel_position"
def test_mapping_engine_maps_and_validates_vessel_ais():
mapping = {
"source": {"items_path": "$.data[*]"},
"fields": {
"mmsi": {"path": "$.mmsi", "type": "integer"},
"lat": {"path": "$.latitude", "type": "float"},
"lon": {"path": "$.longitude", "type": "float"},
"sog": {"path": "$.speedOverGround", "type": "float"},
"received_at": {"path": "$.timestamp", "type": "datetime"},
},
}
result = execute_mapping(SAMPLE_AIS, mapping, "vessel_ais")
assert result["mapped_count"] == 1
assert result["failed_count"] == 0
assert result["records"][0]["mmsi"] == 257123000
assert result["records"][0]["lat"] == 59.91
def test_mapping_engine_reports_schema_errors():
mapping = {
"source": {"items_path": "$.data[*]"},
"fields": {
"mmsi": {"path": "$.mmsi", "type": "integer"},
"lat": {"path": "$.missing_lat", "type": "float"},
"lon": {"path": "$.longitude", "type": "float"},
},
}
result = execute_mapping(SAMPLE_AIS, mapping, "vessel_ais")
assert result["mapped_count"] == 0
assert result["failed_count"] == 1
assert any("lat" in error for error in result["errors"][0]["errors"])
def test_redact_for_llm_masks_secret_like_fields():
redacted = redact_for_llm(SAMPLE_AIS)
assert redacted["data"][0]["api_token"] == "[REDACTED]"
@pytest.mark.asyncio
async def test_persist_mapped_records_writes_generic_records():
class FakeDB:
def __init__(self):
self.added = []
self.committed = False
def add(self, value):
self.added.append(value)
async def commit(self):
self.committed = True
db = FakeDB()
count = await persist_mapped_records(
db,
datasource_name="custom_weather",
datasource_config_id=42,
target_schema="generic_records",
records=[{"source_id": "row-1", "data": {"temp": 25}}],
mapping_version=3,
)
assert count == 1
assert db.committed is True
assert db.added[0].source == "custom_weather"
assert db.added[0].data_type == "generic_records"
assert db.added[0].extra_data["mapping_version"] == 3
@pytest.mark.asyncio
async def test_mapping_preview_api_uses_deterministic_engine():
def override_get_current_user():
return User(
id=1,
username="testuser",
email="test@example.com",
password_hash="hashed",
role="admin",
is_active=True,
)
app.dependency_overrides = {get_current_user: override_get_current_user}
transport = ASGITransport(app=app)
try:
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.post(
"/api/v1/datasources/mappings/preview",
json={
"sample_payload": SAMPLE_AIS,
"target_schema": "vessel_ais",
"mapping_json": {
"source": {"items_path": "$.data[*]"},
"fields": {
"mmsi": {"path": "$.mmsi", "type": "integer"},
"lat": {"path": "$.latitude", "type": "float"},
"lon": {"path": "$.longitude", "type": "float"},
},
},
},
)
finally:
app.dependency_overrides.clear()
assert response.status_code == 200
payload = response.json()
assert payload["success"] is True
assert payload["preview"]["records"][0]["mmsi"] == 257123000
@pytest.mark.asyncio
async def test_mapping_propose_api_redacts_sample_before_ai():
seen_context = {}
class FakeAIClient:
async def analyze(self, request, request_id=None):
seen_context.update(request.context)
return SimpleNamespace(
content=(
'{"source":{"items_path":"$.data[*]"},"fields":{'
'"mmsi":{"path":"$.mmsi","type":"integer"},'
'"lat":{"path":"$.latitude","type":"float"},'
'"lon":{"path":"$.longitude","type":"float"}}}'
)
)
def override_get_current_user():
return User(
id=1,
username="testuser",
email="test@example.com",
password_hash="hashed",
role="admin",
is_active=True,
)
def override_ai_client():
return FakeAIClient()
app.dependency_overrides = {
get_current_user: override_get_current_user,
get_ai_provider_client: override_ai_client,
}
transport = ASGITransport(app=app)
try:
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.post(
"/api/v1/datasources/mappings/propose",
json={
"sample_payload": SAMPLE_AIS,
"target_schema": "vessel_ais",
"use_ai": True,
},
)
finally:
app.dependency_overrides.clear()
assert response.status_code == 200
payload = response.json()
assert payload["mapping_json"]["meta"]["generated_by"] == "ai_provider"
assert seen_context["sample_payload"]["data"][0]["api_token"] == "[REDACTED]"

View File

@@ -0,0 +1,105 @@
from datetime import datetime, timedelta, timezone
import pytest
from httpx import ASGITransport, AsyncClient
from app.api.v1.visualization import convert_vessels_to_geojson
from app.db.session import get_db
from app.main import app
from app.models.vessel import VesselPosition, VesselStatic
from app.services.collectors.vessel_ais import VesselAISCollector
def test_vessel_collector_transforms_barentswatch_like_records():
collector = VesselAISCollector()
records = collector.transform(
[
{
"mmsi": "257123000",
"lat": "59.91",
"lon": "10.73",
"sog": 12.4,
"cog": 214,
"nav_status": 0,
"shipType": 70,
"name": "OSLO TRADER",
},
{"mmsi": "bad", "lat": 120, "lon": 10},
]
)
assert len(records) == 1
assert records[0]["mmsi"] == 257123000
assert records[0]["vessel_type_name"] == "Cargo"
assert records[0]["lat"] == pytest.approx(59.91)
def test_convert_vessels_to_geojson():
position = VesselPosition(
mmsi=257123000,
lat=59.91,
lon=10.73,
sog=12.4,
cog=214,
heading=215,
nav_status=0,
received_at=datetime(2026, 4, 28, 1, 0, tzinfo=timezone.utc),
)
static = VesselStatic(
mmsi=257123000,
name="OSLO TRADER",
vessel_type=70,
vessel_type_name="Cargo",
flag="NO",
length=185,
)
payload = convert_vessels_to_geojson([(position, static)])
assert payload["type"] == "FeatureCollection"
assert payload["features"][0]["geometry"]["coordinates"] == [10.73, 59.91]
assert payload["features"][0]["properties"]["mmsi"] == 257123000
assert payload["features"][0]["properties"]["vessel_type_name"] == "Cargo"
@pytest.mark.asyncio
async def test_vessels_geojson_endpoint_filters_type_and_bbox():
now = datetime(2026, 4, 28, 1, 0, tzinfo=timezone.utc)
rows = [
(
VesselPosition(mmsi=1, lat=59.9, lon=10.7, received_at=now),
VesselStatic(mmsi=1, name="Cargo Ship", vessel_type=70, vessel_type_name="Cargo"),
),
(
VesselPosition(mmsi=2, lat=60.3, lon=5.3, received_at=now - timedelta(minutes=1)),
VesselStatic(mmsi=2, name="Passenger Ship", vessel_type=60, vessel_type_name="Passenger"),
),
]
class _Result:
def all(self):
return rows
class _FakeSession:
async def execute(self, _query):
return _Result()
async def override_get_db():
yield _FakeSession()
app.dependency_overrides[get_db] = override_get_db
transport = ASGITransport(app=app)
try:
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get(
"/api/v1/visualization/geo/vessels",
params={"bbox": "0,50,20,70", "type": "cargo"},
)
assert response.status_code == 200
data = response.json()
assert data["count"] == 1
assert data["features"][0]["properties"]["name"] == "Cargo Ship"
assert data["stats"]["by_type"]["Cargo"] == 1
finally:
app.dependency_overrides.clear()

View File

@@ -215,3 +215,133 @@ async def test_compute_centers_geojson_endpoint_returns_stats():
assert data["features"][0]["properties"]["data_type"] == "compute_center"
finally:
app.dependency_overrides.clear()
@pytest.mark.asyncio
async def test_visualization_geo_summary_returns_counts(monkeypatch):
records = [
_build_record(
record_id=1,
source="arcgis_cables",
data_type="submarine_cable",
name="Test Cable",
country="",
city="",
latitude=0,
longitude=0,
metadata={
"route_coordinates": [[[0, 0], [1, 1]]],
"status": "active",
},
),
_build_record(
record_id=2,
source="arcgis_landing_points",
data_type="landing_point",
name="Test Landing",
country="United States",
city="New York",
latitude=40.7,
longitude=-74.0,
metadata={"city_id": 10},
),
_build_record(
record_id=3,
source="celestrak_tle",
data_type="satellite_tle",
name="TESTSAT",
country="",
city="",
latitude=0,
longitude=0,
metadata={
"norad_cat_id": 12345,
"tle_line1": "1 12345U 98067A 24001.00000000 .00000000 00000-0 00000-0 0 9991",
"tle_line2": "2 12345 51.6000 100.0000 0001000 10.0000 20.0000 15.50000000 01",
},
),
_build_record(
record_id=4,
source="top500",
data_type="supercomputer",
name="Frontier",
country="United States",
city="Oak Ridge",
latitude=35.93,
longitude=-84.31,
metadata={"rank": 1, "rmax": 1102000.0},
),
_build_record(
record_id=5,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name="Colossus",
country="United States",
city="Memphis",
latitude=35.15,
longitude=-90.05,
metadata={"value": "20000", "unit": "TFlop/s"},
),
]
class _ScalarResult:
def __init__(self, rows=None, scalar_value=None):
self._rows = rows or []
self._scalar_value = scalar_value
def scalar(self):
return self._scalar_value
def scalars(self):
class _Scalars:
def __init__(self, rows):
self._rows = rows
def all(self):
return self._rows
return _Scalars(self._rows)
class _FakeSession:
async def execute(self, query):
query_text = str(query)
if "bgp_incidents" in query_text:
return _ScalarResult(scalar_value=2)
if "bgp_anomalies" in query_text:
return _ScalarResult(scalar_value=3)
return _ScalarResult(rows=records)
async def override_get_db():
yield _FakeSession()
async def _fake_build_bgp_collector_coverage(*_args, **_kwargs):
return [
{"collector": "rrc00"},
{"collector": "rrc01"},
]
monkeypatch.setattr(
"app.api.v1.visualization.build_bgp_collector_coverage",
_fake_build_bgp_collector_coverage,
)
app.dependency_overrides[get_db] = override_get_db
transport = ASGITransport(app=app)
try:
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/api/v1/visualization/geo/summary")
assert response.status_code == 200
stats = response.json()["stats"]
assert stats["cable_count"] == 1
assert stats["landing_point_count"] == 1
assert stats["satellite_count"] == 1
assert stats["compute_center_count"] == 2
assert stats["supercomputer_count"] == 1
assert stats["gpu_cluster_count"] == 1
assert stats["bgp_event_count"] == 2
assert stats["bgp_incident_count"] == 2
assert stats["bgp_anomaly_count"] == 3
assert stats["bgp_collector_count"] == 2
finally:
app.dependency_overrides.clear()

View File

@@ -18,6 +18,9 @@ services:
build:
context: .
dockerfile: aiprovider/Dockerfile
args:
PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
container_name: planet_aiprovider
ports:
- "8010:8010"

View File

@@ -5,6 +5,9 @@ services:
build:
context: .
dockerfile: aiprovider/Dockerfile
args:
PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
container_name: planet_aiprovider
ports:
- "8010:8010"

View File

@@ -5,6 +5,9 @@ services:
build:
context: .
dockerfile: aiprovider/Dockerfile
args:
PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
env_file:
- ./aiprovider/.env
container_name: planet_aiprovider

View File

@@ -8,8 +8,115 @@ This project follows the repository versioning rule:
- `improvement` -> `+0.0.1`bugfix + 小功能混合)
- `bugfix` -> `+0.0.1`
## [0.43.0] — 2026-04-28
### ✨ Highlights
- 新增 Earth 船舶追踪链路,接入 BarentsWatch AIS 凭证配置、采集器、后端 vessel 模型/API 与前端 Earth 船舶图层
- 新增自定义数据源映射流程,支持样本抓取、目标 schema、AI 辅助生成映射、预览校验和映射执行
- Settings 拆分 AI Provider 与采集器凭证配置DataSources 只保留采集状态、运行参数和必要引导
### 🔧 Improvements
- AI Provider 支持运行时 LLM 配置、provider preset 下拉与刷新,并在 Playground 中引导到 AI 配置页
- Markdown 渲染器补齐代码块复制按钮、语言标签、任务列表、图片、自动链接、删除线和文档主题样式
- Docs 公开导航改为显式元数据白名单,避免开发任务文档自动出现在“其他”分组
- 将 Codex/Claude cleanup、docs、goal-driven、release 流程补充 CLI-first 约束,并把 `rules.md` 整理成可按模块加载的工程规则
---
## [0.42.2] — 2026-04-28
### 🐛 Fixes
- Docs 中文模式下补齐左侧分组、文档标题、页头分类与搜索结果分类翻译,并更新文档站品牌标题/副标题文案
---
## [0.42.1] — 2026-04-28
### 🐛 Fixes
- 修正 release skill 的 feature 版本计算规则minor 进位时 patch 必须重置为 `0`,例如 `0.41.2` 应发布为 `0.42.0`
---
## [0.42.0] — 2026-04-28
### ✨ Highlights
- 新增公开 `/docs` 文档站支持中英文技术文档、使用手册、Quickstart、搜索、目录锚点与浅色/深色/跟随系统主题
- Earth 在无高清材质时新增轻量 Fresnel 边缘提示,并调整卫星覆盖默认显示与地表材质可读性
### 🔧 Improvements
- 将技术文档整理为 `docs/technical/zh``docs/technical/en`,并补充控制台、`planet.sh`、Earth 与公共组件使用说明
- 新增 `SegmentedControl` 公共滑块组件,支持缩放参数,复用到 docs 语言与主题切换
- Markdown 渲染器接入自定义滚动条,表格与代码块在深色模式和 overflow 场景下保持可读
- Docs 搜索结果支持内部滚动、点击外部关闭、重新聚焦恢复上次搜索结果
- Earth 工具栏展开状态与设置持久化版本迁移继续收口,改善默认面板和快捷关闭行为
---
## [0.41.2] — 2026-04-27
### 🔧 Improvements
- `planet.sh` 启动链路新增 verbose 滚动输出窗口,并在后端端口占用时打印目标地址和监听进程诊断
- Docker 构建支持通过 build args 覆盖 Python 与 uv 镜像,方便 Docker Hub 不稳定时切换镜像源
### 🐛 Fixes
- Earth 海缆登陆点改为基于相机射线与地球遮挡判断可见性,修复旋转后 pin 可见性滞后一帧的问题
### 🔧 Improvements
- `docker-compose*.yml` 为 AI Provider 构建传入 `PYTHON_IMAGE` / `UV_IMAGE` 参数,默认仍使用官方镜像
- 后端启动失败遇到 `Address already in use` 时输出 `lsof``ss` 与 PID 命令行信息
- verbose 模式下 AI Provider build、后端与前端启动日志会在 spinner 下方保留最新 5 行滚动展示
---
## [0.41.1] — 2026-04-27
### 🐛 Fixes
- 修复新闻直播面板设置项持久化失效:`closeTransientMobileOverlays` 通过旁路路径隐藏面板导致下次 persist 快照到错误状态,改为不重新从 DOM 读取面板可见性
- 修复登陆点 pin 在地球侧面被半截遮挡改为在接近地平线前dot < 0.05)主动隐藏,避免深度测试切片
### 🔧 Improvements
- 将所有画布绘制的图标抽取为 SVG存入 `frontend/public/earth/assets/icons/`,新增图标规范到 `rules.md`
---
## [0.41.0] — 2026-04-27
### ✨ Highlights
- Earth 图层系统完成地表到天空的注册顺序与关注优先的面板顺序拆分支持基座海陆色块、国界、高清材质、云图、地形、算力、BGP、卫星、轨迹与海缆的稳定层级
- 国界层新增真实行政区轮廓交互与中国/台湾联动高亮修复高清材质、地形、footprint、卫星与经纬线之间的遮挡和 hover 竞争
### 🔧 Improvements
- 新增无轮廓基座地图,所有图层关闭时仍保留 `#010609` 海洋与 `#080f1b` 陆地色块
- 将大气云图抽象为独立图层并接入桌面/移动端图层开关、持久化状态与启动同步
- 高清材质改为独立纹理覆盖层,地形显示在高清材质上方,并在高清材质关闭/恢复时保持原地形开关意图
- 补充 Earth 渲染层级与图层样式文档,记录正式图层名、变量名、材质颜色、线宽与 renderOrder
---
## [0.40.5] — 2026-04-26
### 🔧 Improvements
- 卫星拖尾改用 Instanced screen-space ribbon单 draw call 渲染所有轨迹段,支持像素级宽度控制
- Iridium 地面覆盖重写为球面投影径向网格,修复填充光晕不可见问题;新增外圈 LineLoop
- 搜索面板打开时改用双 rAF 延迟聚焦输入框,确保 CSS 过渡完成后焦点可靠触发
- 代码清理:提取 `IRIDIUM_OVERLAY_COLOR``IRIDIUM_REFERENCE_ALTITUDE_KM` 常量,消除重复三角函数调用
---
## [0.39.0] — 2026-04-24
## [0.40.4] — 2026-04-26
### 🔧 Improvements
- 新增页面可见性恢复处理,页面从后台切回前台时主动刷新卫星位置,避免累积后台时间在下一帧一次性回放
- 抽出卫星轨迹状态与轨迹几何清理 helper统一后台恢复与清空数据时的轨迹重置路径
### 🐛 Fixes
- 修复页面在后台停留较久后恢复前台时,卫星轨迹因超大 `deltaTime` 突然跳变、拖尾异常拉长的问题
- 修复后台恢复后首帧仍沿用旧轨迹缓存,导致轨迹与当前卫星位置短时错位的问题
---
## [0.40.3] — 2026-04-25
### 🔧 Improvements

View File

@@ -19,10 +19,12 @@
- [earth-mobile-drawer-ui-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
- [earth-compute-center-bgp-style-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)
- [earth-renderer-architecture-separation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-renderer-architecture-separation-plan.md)
- [earth-country-boundary-overlay-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-country-boundary-overlay-plan.md)
- [earth-predicted-orbit-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
- [earth-webgl-instancing-satellites-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
- [earth-real-terrain-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
- [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
- [frontend-public-docs-site-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
- [frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
- [ue5-mvp-fused-plan.md](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)

View File

@@ -0,0 +1,424 @@
# 自定义 API 数据源与 LLM 映射系统 — 实施计划
**状态**:规划中
**创建日期**2026-04-28
**核心原则**LLM 辅助生成映射配置;生产采集使用确定性转换引擎
## 已确认决策
| 项目 | 决策 |
|-----|------|
| 自定义 API 的定位 | 作为内置数据源的补充入口,不直接等同于 Earth 新功能 |
| LLM 的职责 | 探索未知 API、分析样本 JSON、生成 mapping 草案 |
| 采集时是否调用 LLM | 不调用;采集链路必须确定性、可审计、可复现 |
| 自定义数据如何进入 Earth | 必须映射到已支持的目标 schema或先进入通用数据沉淀 |
| 外部凭证放置位置 | Settings / 外部集成统一管理 provider tokenDataSources 引用 provider profile |
| TimescaleDB | 放入 TODO高频时序数据稳定后再评估迁移 |
---
## 一、背景与问题
当前系统已经有 `datasource_configs`,可以配置自定义数据源的 endpoint、auth、headers、config也已经有部分 collector 会读取这些配置。但这只能解决“怎么请求数据”,还没有解决以下问题:
- API 返回 JSON 后,如何转换成系统已有领域模型。
- 自定义数据源是补充已有能力,还是全新数据沉淀。
- 转换规则由谁生成、谁校验、谁执行。
- 未知数据是否能自动在 Earth 上展示。
- 外部 token 是放在全局配置中心,还是放在每个 datasource 下。
专业做法是把“请求配置”“外部凭证”“目标 schema”“字段映射”“采集执行”拆开
- Settings 管外部集成凭证,例如 AI Provider、BarentsWatch、未来付费 AIS API。
- DataSources 管具体数据源实例,例如 endpoint、调度频率、目标 schema、mapping 版本。
- LLM 只在配置阶段辅助生成 mapping不进入生产采集链路。
- Earth 只消费明确 schema 的数据,不消费任意未知 JSON。
---
## 二、目标架构
### 2.1 自定义 API 数据源生命周期
```mermaid
flowchart LR
A[配置 endpoint/auth/request] --> B[抓取 sample JSON]
B --> C[选择目标 schema]
C --> D[LLM 生成 mapping 草案]
D --> E[确定性 mapping engine 预览]
E --> F[schema validation]
F --> G[保存 mapping version]
G --> H[scheduler 执行 mapped collector]
H --> I[写入目标表或 generic_records]
```
### 2.2 目标 schema 分层
| schema | 用途 | Earth 可视化 |
|-------|------|-------------|
| `vessel_ais` | 船只 AIS 位置、航速、航向、MMSI 等 | 进入船舶图层 |
| `geo_points` | 通用点位数据,包含经纬度、名称、类型、时间 | 进入通用 geo layerTODO |
| `news_events` | 新闻/事件类数据,带时间、地点、摘要、来源 | 复用新闻/事件链路 |
| `compute_centers` | 算力中心、机房、数据中心数据 | 复用算力中心图层 |
| `generic_records` | 未知结构化数据沉淀 | 不直接展示 |
v1 建议优先实现:
- `vessel_ais`
- `geo_points`
- `generic_records`
其他 schema 可先在 registry 中预留名称,但不承诺完整落库与可视化。
### 2.3 LLM 的边界
LLM 可以做:
- 根据 API 文档或 sample JSON 解释字段含义。
- 推荐目标 schema。
- 生成 mapping JSON 草案。
- 给出字段置信度和需要人工确认的字段。
- 帮用户发现分页、数组路径、时间字段、坐标字段。
LLM 不应该做:
- 在正式采集时参与每批数据转换。
- 生成并执行 Python/JavaScript 代码。
- 接触 API key、bearer token、basic auth password。
- 自动创建新的 Earth 图层或数据库表。
---
## 三、后端实施计划
### Phase 1 — Target Schema Registry
新增代码级 registry统一描述系统支持的目标数据类型。
每个 target schema 至少包含:
- `key`:例如 `vessel_ais`
- `label`:前端展示名称。
- `description`:适用场景。
- `fields`:字段名、类型、是否必填、说明、示例。
- `validator`Pydantic 或等价校验器。
- `destination`:写入目标,例如 vessel 表、generic_records、future geo layer。
示例概念:
```json
{
"key": "vessel_ais",
"fields": [
{"name": "mmsi", "type": "integer", "required": true},
{"name": "lat", "type": "float", "required": true},
{"name": "lon", "type": "float", "required": true},
{"name": "sog", "type": "float", "required": false},
{"name": "cog", "type": "float", "required": false},
{"name": "received_at", "type": "datetime", "required": false}
]
}
```
### Phase 2 — Mapping Template Model
新增 mapping 配置持久化表,建议命名为 `datasource_mapping_templates`
关键字段:
- `id`
- `datasource_config_id`
- `target_schema`
- `mapping_json`
- `sample_payload_hash`
- `validation_status`
- `version`
- `is_active`
- `created_at`
- `updated_at`
`mapping_json` 是声明式 DSL不允许任意代码执行。
示例:
```json
{
"source": {
"items_path": "$.data.vessels[*]"
},
"fields": {
"mmsi": {"path": "$.mmsi", "type": "integer"},
"lat": {"path": "$.latitude", "type": "float"},
"lon": {"path": "$.longitude", "type": "float"},
"sog": {"path": "$.speedOverGround", "type": "float", "default": null},
"received_at": {"path": "$.timestamp", "type": "datetime"}
}
}
```
### Phase 3 — Deterministic Mapping Engine
实现独立 mapping engine输入 sample/raw payload 和 mapping JSON输出目标 schema 记录。
v1 支持能力:
- JSONPath/JMESPath 风格路径提取。
- 数组展开。
- 默认值。
- 基础类型转换string、integer、float、boolean、datetime。
- 坐标范围校验。
- 简单枚举映射。
- 错误收集:缺字段、类型转换失败、路径不存在。
明确不支持:
- 任意表达式执行。
- 用户提交脚本。
- LLM runtime 修复。
### Phase 4 — LLM Mapping Assistant API
新增配置阶段 API
- `POST /api/v1/datasources/custom/sample`
- 按 datasource 请求配置抓取 sample JSON。
- `GET /api/v1/datasources/target-schemas`
- 返回可选目标 schema 和字段说明。
- `POST /api/v1/datasources/mappings/propose`
- 输入 sample JSON + target schema调用 AI provider 生成 mapping 草案。
- `POST /api/v1/datasources/mappings/preview`
- 使用确定性 mapping engine 预览转换结果。
- `POST /api/v1/datasources/mappings`
- 保存 mapping 版本。
- `PUT /api/v1/datasources/mappings/{id}`
- 更新 mapping生成新版本或覆盖草稿。
- `POST /api/v1/datasources/{id}/run-mapped`
- 手动触发一次 mapped collector。
安全要求:
- `propose` 请求发送给 LLM 前必须脱敏 sample。
- auth headers、token、password 不进入 prompt。
- LLM 返回结果必须再经过 mapping schema 校验。
### Phase 5 — Generic Mapped HTTP Collector
新增通用 collector
- 读取 `DataSourceConfig` 请求配置。
- 读取 active mapping template。
- 拉取 API 数据。
- 使用 mapping engine 转换。
- 使用 target schema validator 校验。
- 调用 destination handler 写入目标表或 generic storage。
- 将失败记录写入错误日志或 dead-letter 结构。
对于 `generic_records`
- 保存 datasource id。
- 保存 target schema。
- 保存 normalized JSON。
- 保存 raw payload 摘要或 raw reference。
- 保存采集时间、source timestamp、mapping version。
---
## 四、前端实施计划
### Phase 1 — Settings 外部集成
Settings 中保留统一外部集成配置:
- AI Providerbase URL、model、API key。
- BarentsWatchclient id/client secret 或 bearer token。
- 未来付费接口AISHub、MarineTraffic、VesselFinder 等 provider profile。
DataSources 不直接管理全局 secret只引用 provider profile。
### Phase 2 — DataSources 自定义源向导
自定义数据源配置改成向导或右侧 drawer
1. Request
- endpoint
- method
- auth profile
- headers
- query/body config
- schedule
2. Sample
- 点击抓取 sample
- 展示 JSON tree
- 支持选择数组根路径
3. Target Schema
- 选择 `vessel_ais``geo_points``generic_records`
- 展示该 schema 必填字段
4. Mapping Proposal
- 调用 LLM 生成 mapping 草案
- 显示字段匹配置信度
- 标出需要人工确认的字段
5. Preview
- 用确定性 engine 预览前 N 条转换结果
- 展示校验错误
6. Save & Enable
- 保存 mapping version
- 启用调度或仅保存草稿
### Phase 3 — 运维视图
为 mapped datasource 展示:
- 上次运行时间。
- 成功记录数。
- 失败记录数。
- 当前 mapping version。
- 目标 schema。
- 最近错误。
- 手动运行按钮。
---
## 五、数据库与存储策略
### v1继续使用 PostgreSQL
PostgreSQL 可以承载当前规模的采集、关系查询、JSONB 沉淀和基础时序查询。v1 不必因为“时序数据”立刻引入 TimescaleDB。
适合继续用 PostgreSQL 的场景:
- 数据量可控。
- 最近状态查询为主。
- 历史保留窗口较短。
- 查询模式还没稳定。
- 需要快速迭代 schema 与 mapping。
### TODOTimescaleDB
以下条件满足后,再评估 TimescaleDB
- AIS、遥测、轨迹类数据达到高频持续写入。
- 需要按时间窗口做聚合、降采样、retention policy。
- 单表时间序列查询明显成为瓶颈。
- 历史轨迹保留从 24h 扩展到数周或数月。
候选迁移对象:
- `vessel_position`
- future telemetry tables
- future generic time-series records
备选方案:
- PostgreSQL 原生按天/月分区。
- TimescaleDB hypertable。
- 热数据 PostgreSQL冷数据对象存储。
---
## 六、安全与治理
### Secret 管理
- Settings 中保存 provider credentials。
- API 返回配置时必须 mask secret。
- LLM prompt 只能包含脱敏 sample 和 schema 说明。
- 后续 TODO引入字段级加密或 KMS。
### Mapping 治理
- 每次 mapping 变更保留版本。
- active mapping 只能有一个。
- 允许保存 draft mapping。
- 运行记录关联 mapping version。
- 校验失败不能自动启用。
### 错误处理
常见错误类型:
- API 401/403凭证错误或过期。
- API 429限流需要调整 schedule。
- JSON path 不存在:上游结构变化。
- 类型转换失败mapping 规则错误。
- schema validation failed转换结果不满足目标模型。
每次运行需要记录:
- datasource id。
- mapping version。
- started_at / finished_at。
- fetched count。
- mapped count。
- written count。
- failed count。
- error summary。
---
## 七、测试计划
### Backend Unit Tests
- mapping engine
- path 提取。
- 数组展开。
- 默认值。
- 类型转换。
- datetime parse。
- 枚举映射。
- 缺字段错误。
- target schema registry
- `vessel_ais` 必填字段校验。
- `geo_points` 经纬度范围校验。
- `generic_records` 接受未知结构。
- LLM assistant
- mock provider 返回 mapping。
- 验证 secret 不进入 prompt。
- 验证非法 mapping 被拒绝。
### Backend Integration Tests
- sample JSON -> propose mapping -> preview -> save mapping。
- mapped collector 使用保存的 mapping 写入 `generic_records`
- `vessel_ais` sample 写入船舶相关目标结构。
- 上游 JSON 结构变化时,运行失败并记录错误。
### Frontend Tests
- 自定义数据源向导完整流程。
- 未配置 AI Provider 时,提示去 Settings 配置,但允许手写 mapping。
- LLM 返回不完整 mapping 时Preview 阶段显示校验错误。
- 保存 mapping 后展示 active version 和运行状态。
---
## 八、分期工作量
| 阶段 | 内容 | 估算 |
|-----|------|------|
| Phase 0 | 完成本规划、确认 schema registry 设计 | 0.5 天 |
| Phase 1 | target schema registry + mapping template model | 12 天 |
| Phase 2 | deterministic mapping engine | 23 天 |
| Phase 3 | sample/propose/preview/save API | 23 天 |
| Phase 4 | DataSources 自定义源向导 | 35 天 |
| Phase 5 | generic mapped collector + run history | 24 天 |
| Phase 6 | vessel_ais / geo_points destination handler | 24 天 |
---
## 九、当前差距与下一步
当前差距:
- `datasource_configs` 只描述请求配置,不描述目标 schema 和 mapping。
- 自定义源没有 sample -> schema -> mapping -> preview -> save 的闭环。
- 生产采集还没有通用 mapped collector。
- Settings 与 DataSources 的职责边界需要在 UI 上进一步明确。
- Earth 还没有通用 `geo_points` 图层。
下一步建议:
1. 先实现 target schema registry 和 mapping engine不急着接 LLM。
2. 用固定 sample JSON 做 `vessel_ais``generic_records` 的单元测试。
3. 再接 LLM propose API让 LLM 产出的只是 mapping 草案。
4. 最后做前端向导,把人工确认和 preview 放到启用之前。

View File

@@ -0,0 +1,272 @@
# 实时船只监控系统 — 实施计划
**状态**:规划中
**创建日期**2026-04-27
**优先数据源**BarentsWatch免费→ AISHub / MarineTrafficTODO付费
## 已确认决策
| 项目 | 决策 |
|-----|------|
| 数据源 | BarentsWatch 先行AISHub / MarineTraffic TODO |
| 船只规模 | BarentsWatch 阶段全部显示;全球数据接入后按需加船型过滤(默认 Cargo + Tanker + Passenger |
| 更新频率 | 准实时:前端 5 分钟轮询,后端 Collector 每分钟拉取写库 |
| 历史轨迹 | 保留(`vessel_position` 表保留 24h后期按需扩展 |
| 推送方式 | HTTP 轮询(不用 WebSocket换实时数据源后再评估升级 |
---
## 一、技术背景
船只通过 AIS自动识别系统每 210 秒广播位置、航速、航向、目的地等信息。全球约 50 万艘持证船只在线,实时数据通过以下方式获取:
| 来源类型 | 典型服务 | 覆盖范围 | 成本 | 状态 |
|---------|---------|---------|------|------|
| **BarentsWatch Open API** | live.ais.barentswatch.no | 挪威海域实时 | 完全免费 | **当前使用** |
| **AISHub** | aishub.net | 全球实时 | 免费/小额 | TODO付费接入 |
| **MarineTraffic API** | marinetraffic.com | 全球实时 | $50$500/月 | TODO评估 tier |
| **VesselFinder API** | vesselfinder.com | 全球实时 | $50$300/月 | TODO备选 |
| **自建 SDR 接收** | RTL-SDR + AIS-catcher | 仅本地 3050km | 硬件 $30 | 不考虑 |
| **NOAA 历史数据** | Marine Cadastre | 美国近海历史 | 免费 | 可用于冷启动 |
### BarentsWatch API
- 端点:`https://live.ais.barentswatch.no/v1/latest/combined`
- 无需注册,直接 GET返回挪威近海 20005000 艘船只 JSON
- 字段mmsi, lat, lon, sog, cog, heading, nav_status, name, vessel_type, flag
- 刷新频率:数据约 3060s 更新一次,可随意轮询
### TODO付费数据源接入
- [ ] 评估 AISHub 订阅(全球覆盖,约 $30/月),接入全球实时流
- [ ] 评估 MarineTraffic API tier对比 AISHub 数据质量与成本
- [ ] 实现多数据源适配器,通过 `datasource_config` 切换
- [ ] 真实高频 AIS 稳定接入后,评估将 `vessel_position` 迁移为 TimescaleDB hypertable保留 Postgres 原生分区作为备选)
---
## 二、实施计划
### Phase 0 — 数据源验证与链路打通12 天)
- 接入 BarentsWatch Open API验证数据格式与字段
- 构建全球 mock 数据生成器(用于前端渲染压测,补充 BarentsWatch 的地域限制)
- 确认前端可渲染船只点,整条链路走通
### Phase 1 — 后端基础设施34 天)
#### 1.1 数据库 Schema
```sql
-- 船只静态信息(每 6h 刷新一次)
CREATE TABLE vessel_static (
mmsi BIGINT PRIMARY KEY,
name VARCHAR(128),
callsign VARCHAR(16),
vessel_type SMALLINT,
vessel_type_name VARCHAR(64),
flag VARCHAR(4), -- ISO 国家码
length FLOAT,
width FLOAT,
draught FLOAT,
imo BIGINT,
updated_at TIMESTAMPTZ
);
-- 船只实时位置(高频写入,保留 24h 轨迹)
CREATE TABLE vessel_position (
id BIGSERIAL PRIMARY KEY,
mmsi BIGINT NOT NULL,
lat FLOAT NOT NULL,
lon FLOAT NOT NULL,
sog FLOAT, -- Speed over ground
cog FLOAT, -- Course over ground
heading SMALLINT, -- 真北航向
nav_status SMALLINT, -- 0=航行 1=锚泊 5=停靠 ...
received_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
);
CREATE INDEX idx_vessel_pos_mmsi_time ON vessel_position(mmsi, received_at DESC);
CREATE INDEX idx_vessel_pos_time ON vessel_position(received_at DESC);
-- 最新位置物化视图(地图渲染主数据源,避免全表扫描)
CREATE MATERIALIZED VIEW vessel_latest AS
SELECT DISTINCT ON (mmsi)
vp.*, vs.name, vs.vessel_type_name, vs.flag, vs.length
FROM vessel_position vp
LEFT JOIN vessel_static vs USING (mmsi)
ORDER BY mmsi, received_at DESC;
CREATE UNIQUE INDEX ON vessel_latest(mmsi);
```
> 后期如需完整历史轨迹查询,迁移 `vessel_position` 到 TimescaleDB 或按天分区。
#### 1.2 CollectorVesselAISCollector
文件:`backend/app/services/collectors/vessel_ais.py`
- 继承 `BaseCollector`,注册到 `collector_registry`
- 轮询间隔3060s由数据源限速决定
- 支持多数据源切换,通过 `datasource_config` 配置 URL + API Key
- 写入逻辑upsert `vessel_latest`append `vessel_position`
- 接入现有调度系统(`scheduler.py`
#### 1.3 API 端点
```
GET /api/v1/visualization/geo/vessels
?bbox=lon_min,lat_min,lon_max,lat_max # 视口裁剪
?type=cargo,tanker,passenger # 船型过滤
?limit=5000
→ GeoJSON FeatureCollectionPoint
GET /api/v1/visualization/vessels/{mmsi} # 单船详情
GET /api/v1/visualization/vessels/{mmsi}/track # 历史轨迹(默认 6h
?hours=6
→ GeoJSON LineString
```
GeoJSON Feature 格式:
```json
{
"type": "Feature",
"geometry": { "type": "Point", "coordinates": [lon, lat] },
"properties": {
"mmsi": 123456789,
"name": "EVER GIVEN",
"vessel_type": 70,
"vessel_type_name": "Cargo",
"flag": "PA",
"sog": 12.4,
"cog": 247.0,
"heading": 245,
"nav_status": 0,
"length": 400,
"received_at": "2026-04-27T10:00:00Z"
}
}
```
#### 1.4 更新机制
**HTTP 轮询**(不使用 WebSocket
- 前端 `setInterval(fetchVessels, 5 * 60 * 1000)` 定期拉取最新快照
- 后端 Collector 每 60s 从 BarentsWatch 拉取并写库,`vessel_latest` 物化视图随时可查
- WebSocket 留给告警/事件驱动场景BGP、系统通知不混入周期性位置刷新
- 换用 AISHub / MarineTraffic 实时流后,届时再评估是否升级为 WebSocket delta push
---
### Phase 2 — 前端渲染34 天)
文件:`frontend/public/earth/js/vessels.js`
#### 2.1 渲染方案
参考现有卫星系统(`satellites.js`)的 InstancedMesh 模式:
- `THREE.InstancedMesh`:每个实例 = 一艘船,矩阵包含位置 + 旋转(朝向 COG
- 行进船:三角箭头图标,朝向 COG 方向
- 静止/锚泊船:圆点图标
- SVG 图标输出到 `frontend/public/earth/assets/icons/vessel-arrow.svg``vessel-dot.svg`
#### 2.2 船型颜色规范
| 船型 | 颜色 |
|-----|------|
| 货轮 Cargo | `#4A90D9` 蓝 |
| 油轮 Tanker | `#E85D04` 橙红 |
| 客船 Passenger | `#06D6A0` 绿 |
| 渔船 Fishing | `#FFD166` 黄 |
| 军舰 Military | `#73797E` 灰 |
| 其他 | `#9B9B9B` 浅灰 |
| 锚泊/停靠 | 降低饱和度 0.4x |
#### 2.3 LOD相机距离细节层次
| 相机距离 | 渲染策略 |
|---------|---------|
| > 400 | 仅渲染 top 1000 艘(按数据新鲜度 + 船型优先级) |
| 200400 | 渲染 top 5000 艘 |
| < 200 | 渲染当前视口 bbox 内全部船只 |
前端根据相机位置动态计算 bbox附加到 API 请求中。
#### 2.4 图层集成
接入现有图层系统,新增"船只"图层项,支持:
- 图层开/关,状态持久化
- 子过滤(按船型选择显示哪类,可在图例或设置面板中配置)
- 与海缆、BGP、卫星层级共存renderOrder 待定,参考现有层级文档)
#### 2.5 Info Card
复用 `showInfoCard` 机制,点击船只弹出:
```
EVER GIVEN 🚢
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
MMSI 123456789
IMO 9811000
旗帜 巴拿马 🇵🇦
船型 散货轮
当前航速 12.4 kn
航向 247°
状态 航行中
目的地 ROTTERDAM
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
[ 查看轨迹 ] [ MarineTraffic ↗ ]
```
#### 2.6 轨迹可视化
点击"查看轨迹" → 请求 `/vessels/{mmsi}/track` → 用 `THREE.CatmullRomCurve3` 渲染插值轨迹线,风格与海缆一致。
---
### Phase 3 — 功能完善23 天)
| 功能 | 说明 |
|-----|------|
| **船只搜索** | 接入现有搜索面板,按名称 / MMSI 搜索 |
| **统计 HUD** | 显示当前在线船只数、各类型分布 |
| **密度热图** | 超低 zoom 时切换为 hex-bin 热力图(避免点云爆炸) |
| **港口标注** | 加载 WorldPorts 数据集,显示主要港口标记 |
| **关键水道监控** | 马六甲、霍尔木兹、苏伊士等高亮 + 流量统计 |
---
### Phase 4 — 性能与生产化23 天)
- `vessel_position` 按天分区7 天自动清理
- TODO真实数据量达到百万级/日后,将 `vessel_position` 升级为 TimescaleDB hypertable配置 retention policy 与压缩策略
- GeoJSON endpoint 用 Redis 缓存 15s
- 若需 bbox 精确查询,引入 PostGIS `geography` + `ST_DWithin`
- InstancedMesh + frustum culling目标 5 万船只 60fps
---
## 三、工作量估算
| Phase | 内容 | 估计时间 |
|-------|-----|---------|
| Phase 0 | 数据源验证、mock | 12 天 |
| Phase 1 | 后端 Schema + Collector + API | 34 天 |
| Phase 2 | 前端渲染InstancedMesh + 图层 + Info Card | 34 天 |
| Phase 3 | 搜索 + 统计 + 轨迹 | 23 天 |
| Phase 4 | 性能优化 + 生产数据源接入 | 23 天 |
| **合计** | | **约 23 周** |
---
## 四、参考资料
- BarentsWatch AIS API 文档https://www.barentswatch.no/en/developer/ais-api/
- MarineTraffic APIhttps://www.marinetraffic.com/en/ais-api-services
- AISHubhttps://www.aishub.net/api
- AIS 导航状态码ITU-R M.1371-5
- 船型编码vessel_typeITU/IMO AIS Message 5 Type and Cargo
- WorldPorts 数据集https://msi.nga.mil/Publications/WPI

View File

@@ -0,0 +1,97 @@
# Markdown 渲染器完善计划
## 背景
Planet 控制台当前有三类主要 Markdown 使用场景:
- 文档中心:技术文档、计划文档、运行手册。
- AI Playground模型回复、分析结果、代码片段。
- BGP 简报:由系统生成并保存的态势报告。
这些场景都复用 `frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx`。因此 Markdown 能力应该集中在共享渲染器内完成,页面只负责传入内容、链接转换和布局约束,不能让每篇文档或每个页面手写复制按钮、表格样式、列表样式等交互细节。
## 目标
建设一个稳定、可复用、适合技术文档和 AI 输出的 Markdown 渲染器,优先覆盖常用语法、代码块操作和清晰的阅读样式,并为后续语法高亮、锚点导航、内容安全策略留出接口。
## 成功标准
- 代码块支持 fenced language、语言标签、复制按钮、复制成功状态和横向滚动。
- 常用块语法稳定渲染:标题 1-6、段落、引用、分割线、表格、无序列表、有序列表、任务列表。
- 常用行内语法稳定渲染:链接、自动链接、图片、行内代码、粗体、斜体、删除线。
- 文档中心、AI Playground、BGP 简报继续复用同一个组件,不出现页面级重复实现。
- 样式在普通业务面板和文档中心都有合理表现,文档中心可以通过 `.docs-markdown` 覆盖主题变量。
- 前端 TypeScript build 通过,`git diff --check` 无空白错误。
## 当前实施范围
### 第一阶段:共享渲染器补齐
-`MarkdownRenderer` 内解析 fenced code block 的语言信息。
- 引入 `MarkdownCodeBlock` 子组件,负责语言标签、复制按钮和复制状态。
- 保留现有 `Scrollbar` 横向滚动能力,避免长代码撑破页面。
- 扩展标题渲染到 h1-h6并保留 `getHeadingId` 对文档目录的支持。
- 扩展列表解析,支持 `-``*``+``1.``1)` 和 GitHub 风格任务列表。
- 扩展行内解析,支持图片、自动链接、删除线。
### 第二阶段:样式统一
- 全局 Markdown 样式覆盖业务场景,保持紧凑、清晰、可扫描。
- 文档中心用 `.docs-markdown` 适配主题变量,避免硬编码颜色破坏明暗主题。
- 代码块 toolbar 和 copy button 不依赖具体页面。
- 图片默认响应式展示,避免超出内容区域。
### 第三阶段:验证
- 使用前端 build 验证 TypeScript 和 Vite 构建。
- 使用 `git diff --check` 验证补丁格式。
- 手动检查至少一个文档页中代码块复制按钮、语言标签和表格滚动是否出现。
## 后续增强项
### 语法高亮
当前不新增高亮依赖,避免一次性引入过重运行时代码。后续可以在以下方案中二选一:
- `shiki`:适合文档中心,视觉质量高,但包体和初始化成本更高。
- `highlight.js`:接入简单,覆盖语言广,但样式控制需要额外约束。
建议当文档代码块数量稳定增加后再引入,并做按需加载或懒加载。
### 更完整 CommonMark 支持
当前渲染器覆盖 Planet 常见内容,不追求完整 CommonMark 兼容。后续如果需要完整规范,建议切换到成熟生态:
- `react-markdown`
- `remark-gfm`
- `rehype-sanitize`
- `rehype-slug`
切换前需要评估:链接转换、目录 ID、现有样式、AI 输出安全策略和包体影响。
### 安全策略
目前渲染器不解析原始 HTML这是正确默认值。后续如需支持 HTML必须先明确
- 是否允许用户输入 Markdown。
- 是否需要 HTML 白名单。
- 是否需要 `rehype-sanitize`
- 图片和链接是否需要域名策略。
### 文档页能力
可继续补齐:
- 标题锚点悬浮复制。
- Mermaid 图表。
- 代码块折叠。
- 文档内搜索结果定位到代码块。
- 复制按钮埋点,用于判断文档片段是否真正被使用。
## 维护约束
- Markdown 语法能力优先放在共享渲染器,不在具体文档页面散落实现。
- 文档内容只表达内容,不承载 UI 行为。
- 新增 Markdown 能力必须同时考虑文档中心、AI Playground、BGP 简报三个调用方。
- 不解析原始 HTML除非同步引入明确的 sanitize 策略。
- 与主题相关的样式优先走页面容器变量覆盖,不在组件内写死文档中心颜色。

View File

@@ -0,0 +1,486 @@
# Frontend Public Docs Site Plan
## 目标
新增一个公开访问的 `/docs` 页面,作为 Planet 的开发设计文档与使用手册入口。
这个页面应类似常见开源软件文档站:
- 不需要登录即可访问
-`/earth` 和 admin 后台平级,但视觉和信息架构独立
- 直接整理并展示仓库内 `docs/technical` 的 Markdown 文档
- 支持搜索、分类导航、文档目录和内部跳转
-`docs/technical` 继续作为文档真源,避免页面内容和仓库文档漂移
## 非目标
本阶段不做:
- 后端全文搜索服务
- 数据库驱动的 CMS
- 独立文档构建系统,例如 Docusaurus / VitePress
- 每篇文档单独手写 React 页面
- 用户权限、编辑器、在线保存或评论功能
-`docs/plans``docs/deprecated` 全量公开为正式手册
后续可以再决定是否把 plans / deprecated 做成独立的“路线图 / 历史归档”分区。
## 技术路线
### 推荐方案Markdown 直接渲染
使用 Vite 在前端构建阶段直接加载 `docs/technical/**/*.md`
```ts
const modules = import.meta.glob('../../../docs/technical/**/*.md', {
query: '?raw',
import: 'default',
})
```
这样每篇 Markdown 文件仍然留在仓库文档目录中,`/docs` 页面只是读取、索引和渲染这些文档。
当前项目已经满足主要前提:
- 前端使用 Vite + React
- `frontend/vite.config.ts` 已配置 `server.fs.allow: ['..']`
- 已有 `MarkdownRenderer` 可作为基础
- `docs/technical` 文档数量较少,前端本地搜索足够
### 不推荐方案:每篇文档单独写 React
不建议把每篇文档重写成 `.tsx` 页面,因为:
- 文档会出现两份真源
- 修改技术文档时还要同步 UI 页面
- 计划文档、技术上下文、变量表这类内容天然适合 Markdown
- 后续新增文档的成本会变高
只有当某篇文档需要强交互演示、实时图表或复杂 UI 时,才考虑给该文档补充一个 React 组件扩展。
## 信息架构
### 公开路由
新增:
- `/docs`
- `/docs/:slug`
路由行为:
- `/docs` 默认打开 `docs/technical/README.md`,或打开人工指定的首页文档
- `/docs/:slug` 打开对应技术文档
- 未找到文档时显示 docs 专属 404而不是跳回 admin
- `/docs` 加入 `App.tsx` 的公开路由白名单
### 文档分类
`docs/technical` 中的现有文档整理进以下分组:
#### Overview
- `README.md`
#### Earth
- `earth-frontend-context.md`
- `earth-layer-style-reference.md`
- `earth-render-layer-order.md`
- `earth-satellite-footprint-policy.md`
- `earth-bgp-context.md`
- `earth-news-live-streams-collector-format.md`
#### Frontend
- `frontend-admin-frontend-context.md`
- `frontend-layout-guidelines.md`
#### Backend
- `backend-collectors.md`
- `backend-system-service-control.md`
#### Agents
- `agents-aiprovider.md`
#### Ops
- `ops-docker-compose-buildx-upgrade.md`
### 页面布局
桌面端:
- 顶部:产品名、搜索框、当前文档标题
- 左侧:文档分组导航
- 中间Markdown 正文
- 右侧:当前文档目录,也就是 h2 / h3 anchors
移动端:
- 顶部固定搜索入口
- 导航折叠为抽屉或下拉
- 正文单列显示
- 当前文档目录折叠为“本文目录”
视觉风格:
- 像开源软件 docs 页面,清晰、安静、可长时间阅读
- 不复用 admin 后台的重操作感布局
- 不做 Earth 的沉浸式深色 HUD 风格
- 优先阅读性、扫描效率和代码/表格可读性
## 前端实现设计
### 文件结构
建议新增:
```text
frontend/src/pages/Docs/
Docs.tsx
docs-content.ts
docs-search.ts
docs-slugs.ts
Docs.css
```
可选拆分:
```text
frontend/src/pages/Docs/components/
DocsSidebar.tsx
DocsSearch.tsx
DocsToc.tsx
DocsMarkdown.tsx
```
如果初版代码量不大,可以先保持在 `Docs.tsx` + 少量 helper 文件中,避免过度拆分。
### 文档注册表
创建一个 registry负责将 Markdown 文件路径映射为文档元信息:
```ts
interface DocsEntry {
slug: string
path: string
title: string
group: string
order: number
loader: () => Promise<string>
}
```
slug 规则:
- `docs/technical/README.md` -> `overview`
- `docs/technical/earth-layer-style-reference.md` -> `earth-layer-style-reference`
- 只暴露稳定 slug不暴露本机绝对路径
标题规则:
- 优先读取 Markdown 第一个 `# heading`
- 没有 h1 时用人工 registry title
- 再 fallback 到文件名转换标题
### Markdown 渲染
初版可以复用现有:
- [frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx](/home/ray/dev/linkong/planet/frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx)
但建议增强或包装为 docs 专用渲染:
- heading 生成稳定 `id`
- 右侧 TOC 使用同一套 heading 解析结果
- 内部 Markdown 链接转换为 `/docs/:slug`
- 外部链接保留 `target="_blank" rel="noreferrer"`
- 表格横向滚动
- 代码块保留等宽字体和语言标记
- 支持 GitHub 风格的相对文档链接
内部链接转换示例:
- `earth-render-layer-order.md` -> `/docs/earth-render-layer-order`
- `./earth-layer-style-reference.md` -> `/docs/earth-layer-style-reference`
- `/home/ray/dev/linkong/planet/docs/technical/foo.md` -> `/docs/foo`
对非 `docs/technical` 的链接:
- 初版可保留原始链接文本
- 或显示为不可跳转的 repo path
- 后续再扩展为跨文档区导航
### 搜索
初版使用纯前端本地搜索。
索引字段:
- title
- slug
- group
- headings
- markdown 正文纯文本
搜索策略:
- 页面首次加载后异步加载所有 `docs/technical` Markdown
- 生成内存索引
- 用户输入时本地过滤
- 简单打分即可:
- 标题命中权重最高
- heading 命中其次
- 文件名 / slug 命中其次
- 正文命中最低
搜索结果展示:
- 文档标题
- 分组
- 命中的 heading 或正文摘要
- 点击跳转到文档
当前只有 13 篇文档,不需要 Lunr、Fuse 或后端搜索。后续文档数量显著增长时,再考虑引入轻量搜索库。
### 路由接入
修改:
- [frontend/src/App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
新增 lazy import
```ts
const Docs = lazy(() => import('./pages/Docs/Docs'))
```
公开路由:
```ts
const publicPaths = new Set(['/', '/earth', '/docs'])
```
注意:`/docs/:slug` 不能只用精确匹配 `Set`
建议改为:
```ts
const isPublicRoute =
window.location.pathname === '/' ||
window.location.pathname === '/earth' ||
window.location.pathname === '/docs' ||
window.location.pathname.startsWith('/docs/')
```
新增 routes
```tsx
<Route path="/docs" element={<Docs />} />
<Route path="/docs/:slug" element={<Docs />} />
```
### 样式
建议独立 `Docs.css`,不依赖 admin 页面布局。
核心样式要求:
- 文档正文最大宽度控制在适合阅读的范围
- 表格横向滚动,不撑破布局
- 代码块横向滚动
- 左侧导航固定或 sticky
- 右侧 TOC sticky
- 移动端隐藏右侧 TOC导航折叠
- 搜索结果浮层或独立面板不遮挡正文阅读
注意:
- 不做营销 hero
- 不做卡片堆叠式首页
- 首页第一屏应直接是文档入口和内容,而不是宣传页
## 实施阶段
### Phase 1基础文档站
目标:
- `/docs` 可公开访问
- 能看到 `docs/technical` 文档列表
- 能打开每篇 Markdown
- 能基本渲染标题、段落、列表、代码块、表格
任务:
- 新增 `Docs` 页面
- 新增 docs registry
- 接入 Vite raw Markdown loading
- 接入 `/docs``/docs/:slug`
- 加入公开路由白名单
- 初版 CSS 布局
验收:
- 未登录访问 `/docs` 不跳转登录
- `/docs/earth-layer-style-reference` 可打开样式参考文档
- `/docs/backend-collectors` 可打开后端采集器文档
- 构建通过:`source ~/.zshrc && bun run build`
### Phase 2搜索与 TOC
目标:
- 支持本地搜索所有 technical 文档
- 当前文档右侧显示目录
- 搜索结果可跳转
任务:
- 实现 heading parser
- 实现 TOC 组件
- 实现 search index
- 搜索结果显示文档标题、分组和摘要
- 当前文档标题与 active nav 高亮
验收:
- 搜索 `Fresnel` 能找到 Earth 图层样式文档
- 搜索 `collector` 能找到 backend collectors
- 点击搜索结果进入对应文档
- 右侧 TOC 点击后滚动到对应 heading
### Phase 3链接清理与文档体验
目标:
- Markdown 内部链接在 docs 站内自然跳转
- 长表格、代码块、绝对路径链接的显示更友好
任务:
- 转换 `docs/technical/*.md` 相对链接
- 转换 repo 内 technical 文档绝对路径
- 外链新窗口打开
- 文件路径链接以代码样式显示
- 增强空状态和 404
验收:
-`docs/technical/README.md` 点击 technical 文档链接进入 `/docs/:slug`
- 不支持的 repo 内路径不会导致前端崩溃
- 外部链接行为正常
### Phase 4文档内容整理
目标:
- `docs/technical` 的首页适合作为公开手册入口
- 每篇文档标题、摘要和分类清晰
任务:
- 检查每篇文档是否有唯一 h1
- 给 README 补公开手册导览
- 必要时补文档摘要
- 保持文档内容仍然服务开发维护,不改成营销语气
验收:
- `/docs` 首页能说明各技术文档用途
- 左侧分类和 README 内容一致
- 没有明显重复、过期或找不到的主入口
## 需要改动的文件
预计新增:
- `frontend/src/pages/Docs/Docs.tsx`
- `frontend/src/pages/Docs/Docs.css`
- `frontend/src/pages/Docs/docs-content.ts`
- `frontend/src/pages/Docs/docs-search.ts`
预计修改:
- `frontend/src/App.tsx`
- `frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx` 或新增 docs 专用 wrapper
- `docs/technical/README.md`
可选修改:
- `frontend/src/index.css`,只放全局极少量 docs shell reset 时才需要
- `docs/CHANGELOG.md`,实施完成后记录
- `docs/version-history.md`,若进入版本发布流程再更新
## 风险与注意事项
### 构建路径风险
Vite 从 `frontend/src` 读取 `../../../docs/technical/**/*.md` 时,需要确认开发和生产构建都可解析。
缓解:
- 使用相对路径 glob
- 构建验证必须跑 `source ~/.zshrc && bun run build`
- 不使用运行时 `fetch('/docs/...')` 读取仓库文件,避免生产环境缺文件
### Markdown 能力不足
现有 `MarkdownRenderer` 是轻量实现,可能不完整支持所有 GitHub Markdown。
缓解:
- 初版优先覆盖当前 `docs/technical` 实际用到的语法
- 若后续需要脚注、嵌套列表、复杂代码高亮,再考虑引入 `react-markdown` 等依赖
### Bundle 体积
把所有 Markdown 打进前端 bundle 会增加体积。
当前文档数量少,风险可接受。
缓解:
- 使用 lazy page chunk
- Markdown loader 保持异步
- 搜索索引在 `/docs` 页面内初始化,不影响 `/earth` 和 admin 首屏
### 公开内容边界
`docs/technical` 会被公开展示,需要避免包含密钥、内部机器地址、临时方案或不应公开的操作细节。
缓解:
- 实施前快速审阅 `docs/technical`
- 暂不公开 `docs/plans``docs/deprecated`
- 以后如需公开更多文档,先建立 allowlist
## 验收清单
- `/docs` 未登录可访问
- `/docs/:slug` 未登录可访问
- `/docs` 不影响 `/earth`
- 未登录访问 admin 仍然跳登录
- 左侧导航包含所有 `docs/technical` 文档
- 文档按 Overview / Earth / Frontend / Backend / Agents / Ops 分类
- Markdown 表格正常显示并可横向滚动
- 代码块正常显示并可横向滚动
- 搜索可搜索标题、heading 和正文
- 搜索结果点击可跳转
- 当前文档 TOC 可跳转
- 不存在的 slug 显示 docs 404
- `source ~/.zshrc && bun run build` 通过
## 后续增强
- 给文档页面增加复制 heading 链接按钮
- 给代码块增加复制按钮
- 增加“上一页 / 下一页”导航
- 增加最近更新信息
- 从 git metadata 读取文档更新时间
- 引入轻量全文搜索库
- 支持 plans / deprecated 独立分区
- 增加页面内反馈入口

View File

@@ -0,0 +1,35 @@
# Technical Docs
This directory holds "current implementation and current structure" documentation, focusing on:
- How the code is organized right now
- Where the current entry points are
- How state and components work
- Which implementation boundaries future changes should follow
What belongs here:
- Quickstart and user manual
- Frontend context
- Earth frontend structure
- Earth satellite footprint policy
- Earth render layer order
- Earth layer style property index
- Backend runtime control
- Collector status
- Collection format conventions
## Entry Points
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md): The shortest path to getting Planet running from scratch
- [manual.md](/home/ray/dev/linkong/planet/docs/technical/en/manual.md): Complete usage guide for the console, `planet.sh`, Earth, and Docs
What does not belong here:
- Incomplete roadmaps
- Future iteration plans
- Large-scale refactor proposals
Those belong in:
- [docs/plans/README.md](/home/ray/dev/linkong/planet/docs/plans/README.md)

View File

@@ -0,0 +1,264 @@
# Data Collectors
## I. System Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ Data Collection Architecture │
├─────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
│ │ TOP500 │ │ Epoch AI │ │ HuggingFace │ │
│ │ Collector │ │ Collector │ │ Collector │ │
│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
│ │ │ │ │
│ └───────────────────┼───────────────────┘ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ BaseCollector │◄── Base class (unified) │
│ │ run() method │ │
│ └─────────┬───────────┘ │
│ │ │
│ ┌─────────────────┼─────────────────┐ │
│ ▼ ▼ ▼ │
│ ┌───────────┐ ┌───────────┐ ┌───────────┐ │
│ │ fetch() │ │transform()│ │ _save_data│ │
│ │ raw data │ │ transform │ │ save to DB│ │
│ └───────────┘ └───────────┘ └───────────┘ │
│ │ │
│ ▼ │
│ ┌─────────────────────┐ │
│ │ CollectedData table│◄── Unified storage │
│ └─────────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Scheduler (APScheduler) │ │
│ │ Scheduled tasks: every 4h/6h/12h/1d auto-execute │ │
│ └─────────────────────────────────────────────────────────┘ │
│ │
└─────────────────────────────────────────────────────────────────┘
```
## II. Pipeline
```python
# 1. Scheduler triggers (scheduled or manual)
# ↓
# 2. run() executes the full pipeline
async def run(self, db):
# 2.1 Check if collector is enabled
if not collector_registry.is_active(self.name):
return {"status": "skipped"}
# 2.2 Record task start
task = CollectionTask(status="running")
db.add(task)
await db.commit()
# 2.3 FETCH — get raw data (implemented by subclass)
raw_data = await self.fetch()
# 2.4 TRANSFORM — convert to unified format
data = self.transform(raw_data)
# 2.5 SAVE — persist to database
records_count = await self._save_data(db, data)
# 2.6 Record task completion
task.status = "success"
task.records_processed = records_count
await db.commit()
```
**Core file**: `backend/app/services/collectors/base.py`
## III. Collector List
| Collector | Data type | Content | Frequency |
|-----------|-----------|---------|-----------|
| TOP500 | supercomputer | Global supercomputer rankings (compute, performance) | 4 hours |
| Epoch AI | gpu_cluster | GPU compute cluster info | 6 hours |
| HuggingFace Models | model | AI model information | 12 hours |
| HuggingFace Datasets | dataset | Dataset information | 12 hours |
| HuggingFace Spaces | space | Demo applications | 1 day |
| PeeringDB | ixp/network/facility | Internet exchange points / networks / facilities | 1-2 days |
| TeleGeography | submarine_cable | Submarine cable information | 7 days |
## IV. Data Format (stored in CollectedData table)
```python
# Each collector's parse_response() return format
{
"source_id": "top500_1", # Original system ID (required)
"name": "El Capitan", # Name (required)
"description": "System desc...", # Description
"country": "United States", # Country
"city": "Livermore, CA", # City
"latitude": "37.6819", # Latitude (string)
"longitude": "-121.7681", # Longitude (string)
"value": "1742.00", # Performance value (e.g. compute)
"unit": "PFlop/s", # Unit
"metadata": { # Extra data (JSON)
"rank": 1,
"r_peak": 2746.38,
"cores": 11039616
},
"reference_date": "2025-11-01" # Data reference date
}
```
## V. Database Schema
**CollectedData table** (`collected_data`)
| Field | Type | Description |
|-------|------|-------------|
| id | SERIAL | Primary key |
| source | VARCHAR(100) | Data source name (top500, huggingface, etc.) |
| source_id | VARCHAR(100) | Original data ID |
| data_type | VARCHAR(50) | Data type (supercomputer, model, etc.) |
| name | VARCHAR(500) | Name |
| title | VARCHAR(500) | Title |
| description | TEXT | Description |
| country | VARCHAR(100) | Country |
| city | VARCHAR(100) | City |
| latitude | VARCHAR(50) | Latitude |
| longitude | VARCHAR(50) | Longitude |
| value | VARCHAR(100) | Performance value |
| unit | VARCHAR(20) | Unit |
| metadata | JSONB | Extra metadata |
| collected_at | TIMESTAMP | Collection time |
| reference_date | TIMESTAMP | Data reference date |
| is_valid | INTEGER | Whether valid |
**Core file**: `backend/app/models/collected_data.py`
## VI. TOP500 Collector Example (full pipeline)
```python
# 1. fetch() — get HTML from the web
async def fetch(self):
url = "https://top500.org/lists/top500/list/2025/11/"
response = await client.get(url)
return response.text # returns HTML
# 2. parse_response() — parse HTML into unified format
def parse_response(self, html):
soup = BeautifulSoup(html, "html.parser")
table = soup.find("table")
for row in table.find_all("tr")[1:]: # skip header
cells = row.find_all("td")
entry = {
"source_id": f"top500_{cells[0].text}",
"name": cells[1].text.strip(),
"country": cells[2].text.strip(),
"city": "",
"latitude": "",
"longitude": "",
"value": "1742.00",
"unit": "PFlop/s",
"metadata": {
"rank": 1,
"cores": "11340000"
},
"reference_date": "2025-11-01"
}
data.append(entry)
return data
# 3. run() automatically calls _save_data() to save to database
```
**Core file**: `backend/app/services/collectors/top500.py`
## VII. Scheduler
```python
# Register all collectors into scheduled tasks at startup
def start_scheduler():
for name, collector in collectors.items():
if collector_registry.is_active(name):
scheduler.add_job(
run_collector_task,
trigger=IntervalTrigger(hours=collector.frequency_hours),
id=name,
name=name
)
```
| Collector | Frequency |
|-----------|-----------|
| TOP500 | Every 4 hours |
| Epoch AI | Every 6 hours |
| HuggingFace | Every 12 hours |
| PeeringDB | Every 1-2 days |
| TeleGeography | Every 7 days |
**Core file**: `backend/app/services/scheduler.py`
## VIII. Code Files
```
backend/app/services/collectors/
├── base.py # Base class: run() pipeline, _save_data() persistence
├── registry.py # Collector registry
├── scheduler.py # Scheduled task dispatch (APScheduler)
├── top500.py # TOP500 collector
├── epoch_ai.py # Epoch AI collector
├── huggingface.py # HuggingFace collector
├── peeringdb.py # PeeringDB collector
└── telegeraphy.py # TeleGeography submarine cable collector
backend/app/models/
└── collected_data.py # Unified data model
```
## IX. Data Usage
Collected data ultimately:
1. **Visualization** — displays supercomputers, GPU clusters, and submarine cables' geographic positions
2. **Situational analysis** — global compute distribution statistics and growth trends
3. **Alert system** — detects changes to important nodes
## X. Collector Registration
Collectors are automatically registered at application startup:
```python
# backend/app/services/collectors/__init__.py
collector_registry.register(TOP500Collector())
collector_registry.register(EpochAIGPUCollector())
collector_registry.register(HuggingFaceModelCollector())
collector_registry.register(HuggingFaceDatasetCollector())
collector_registry.register(HuggingFaceSpacesCollector())
collector_registry.register(PeeringDBIXPCollector())
collector_registry.register(PeeringDBNetworkCollector())
collector_registry.register(PeeringDBFacilityCollector())
collector_registry.register(TeleGeographyCableCollector())
collector_registry.register(TeleGeographyLandingPointCollector())
collector_registry.register(TeleGeographyCableSystemCollector())
```
**Core file**: `backend/app/services/collectors/registry.py`
## XI. Triggering Collection
### Method 1: Scheduled
At startup, APScheduler automatically creates scheduled tasks based on each collector's `frequency_hours` setting.
### Method 2: Manual API trigger
```bash
# Trigger TOP500 collection
curl -X POST http://localhost:8000/api/v1/datasources/1/trigger \
-H "Authorization: Bearer <token>"
```
**Core file**: `backend/app/api/v1/datasources.py`

View File

@@ -0,0 +1,252 @@
# Earth Frontend Context
This document describes the current real structure of the Earth display frontend. The focus is on helping future changes to the HUD, layers, media panel, real terrain, and BGP visualization avoid repeating past structural and state-sync pitfalls.
Related references:
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md)
## Current Goal
The Earth frontend is not an ordinary admin page — it is an independent large-screen display frontend. Current product goals:
- Maintain the spatial depth and readability of the globe view
- Keep HUD, layers, media panel, BGP, satellites, cables, and similar elements in a unified interaction model
- Clearly represent states like loading, enabled, hidden, and locked
## Current Entry Point
React route entry:
- [Earth.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Earth/Earth.tsx)
The current approach is simple:
- The React page only provides a full-screen `iframe`
- The actual Earth application runs at:
- [index.html](/home/ray/dev/linkong/planet/frontend/public/earth/index.html)
Earth frontend is essentially a standalone static application under `public/earth`.
## Current File Layers
### 1. Page Entry and Structure
- [index.html](/home/ray/dev/linkong/planet/frontend/public/earth/index.html)
Responsibilities:
- Base HUD DOM
- Layer panel
- Media panel
- Toolbar
- Settings dialog
- Legacy element ID compatibility
### 2. Main Runtime
- [main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js)
Responsibilities:
- Globe initialization
- Three.js scene assembly
- Data loading and refresh
- Layer module integration
- Earth-level state synchronization
### 3. Earth Control Layer
- [controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js)
Responsibilities:
- Toolbar interaction
- Layer panel interaction
- Rotation / zoom / layout
- HUD panel drag
- Layer toggle state machine
- Earth settings read, persist, and reset
This is currently the most critical UI control entry point for the Earth frontend.
### 4. UI and Status Messages
- [ui.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/ui.js)
Responsibilities:
- Loading panel
- Status message
- Tooltip / error / cleanup logic
### 5. Globe and Terrain
- [earth.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/earth.js)
- [terrain.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/terrain.js)
Responsibilities:
- Globe sphere, cloud layer, atmosphere
- Real terrain mesh
- Terrain tile fetch, decode, displacement, and shading
### 6. Layer Modules
- [satellites.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/satellites.js)
- [cables.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cables.js)
- [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js)
- [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
- [compute-centers.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/compute-centers.js)
- [country-boundaries.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/country-boundaries.js)
Each module is responsible for its own:
- Data fetching
- Three.js mesh creation and update
- State tracking (loaded, visible, hover, locked)
- Self-cleanup (dispose on scene destroy)
### 7. HUD Panels and Search
- [hud-panels.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/hud-panels.js)
- [info-card.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/info-card.js)
- [search.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/search.js)
- [legend.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/legend.js)
### 8. Cruise Mode
- [cruise-sequencer.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cruise-sequencer.js)
- [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js)
The cruise sequencer handles generic logic: current target, queue order, camera focus, and dwell / hide / switch. Business modules supply target queues and content — they should not contain camera control logic.
### 9. Constants
- [constants.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/constants.js)
All material, layer, satellite, BGP, cable, terrain, celestial, and other style parameters are maintained here. Do not scatter magic numbers in module files.
## Current Style Layers
CSS files in `frontend/public/earth/css/` each correspond to a specific component scope. Do not write global Earth styles into `base.css` unless they genuinely apply to everything.
## Current Layer Toggle State Semantics
### `data-status-target`
Layer toggle buttons use `data-status-target` attributes to link button state to layer state. The state machine in `controls.js` handles:
- `loading`: showing the loading indicator
- `enabled`: layer is active
- `hidden`: layer is hidden
- `error`: layer failed to load
This is the canonical way to synchronize button visual state with actual layer state. Do not maintain separate boolean flags for button display.
## Current Settings Persistence
Earth settings are stored in `localStorage`. The key is typically a namespaced string defined in `constants.js`. `controls.js` handles read, write, and reset.
Settings that affect visual layers (terrain opacity, day/night mode, satellite display style, etc.) are read during initialization and applied immediately.
## Current Terrain Pipeline
1. `terrain.js` creates a sphere geometry with enough segments
2. On load, fetches Terrarium-format elevation tiles from the backend
3. Decodes R/G/B into elevation values
4. Displaces vertex positions radially based on elevation
5. Applies a vertex alpha that fades terrain edges at coastlines
6. Terrain writes to the scene as a mesh above the HD texture layer
When HD texture is off, terrain is temporarily hidden and its state is remembered. When HD texture comes back on, terrain restores its prior visibility.
## Current High-Frequency Risk Points
### 1. Visual State and Business State Out of Sync
The most common class of Earth bugs:
- Button shows "loaded," but layer has no objects rendered
- Button shows "hidden," but objects are still visible
- Loading ended, but button still looks like it hasn't
All future changes must prioritize checking state sync.
### 2. HUD Layout: Check Structure First, Not CSS Patches
Earth HUD has repeatedly experienced:
- Panel compressed to a sliver
- Markdown content clipped
- Tabs/iframe content consumed by `overflow: hidden`
Inspection order:
1. Who is responsible for height
2. Who is responsible for scrolling
3. Which layer is doing the clipping
Do not immediately add `overflow: hidden` or extra wrapper layers.
### 3. Transitional Paths Must Be Closed Off
Earth has gone through multiple rounds of HUD, toolbar, and media panel refactoring, making it easy to accumulate:
- Old helpers
- Old classes
- Old fallback logic
- Deprecated variants
After each major feature is complete, do a cleanup pass.
### 4. Cruise Mode and Business Events Must Not Be Deeply Coupled
The correct boundary:
- The generic cruise layer only knows:
- Current target
- Queue order
- Camera focus
- Dwell / hide / switch
- Business modules only supply:
- Target queues
- Focus coordinates
- Card content
- Highlight / layer side effects
If future cable, satellite, or news cruise is added, do not copy a new set of `main.js` state variables. Instead reuse:
- [cruise-sequencer.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cruise-sequencer.js)
- [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js)
- The business adapter pattern from [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
## Recommended Change Approach
For future Earth changes:
1. First identify what you're changing:
- Three.js rendering layer
- HUD structure layer
- Layer state layer
- Panel content layer
2. If involving layer buttons, connect to the unified state machine
3. If involving visibility toggle, check whether tooltip / legend / info-card / lock all close together
4. If involving panel layout, check structure before touching CSS
## Current Boundary with the Console Frontend
The Earth frontend and the console frontend are not the same UI system:
- Console frontend: React + Ant Design workbench
- Earth frontend: native HUD + Three.js display under `public/earth`
Therefore:
- Earth should not directly reuse Ant Table / AppLayout semantics
- The console should not copy Earth HUD animations and glass-layer design language
For console structure, see:
- [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/en/frontend-admin-frontend-context.md)

View File

@@ -0,0 +1,224 @@
# Earth Layer Style Property Index
This document records the material, color, opacity, line width, radius offset, and `renderOrder` style properties of all Earth frontend layers. For layer ordering relationships, see [earth-render-layer-order.md](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md).
## Naming Conventions
| Category | Convention | Example |
| --- | --- | --- |
| Global config objects | `*_CONFIG` | `COUNTRY_BOUNDARY_CONFIG` |
| Layer radius offsets | `*AltitudeOffset` / `radiusOffset` | `lineAltitudeOffset`, `GRID_CONFIG.radiusOffset` |
| Opacity | `*Opacity` | `hoverLineOpacity` |
| Render order | `*RenderOrder` | `textureOverlayRenderOrder` |
| Color | `*Color`, hex number or CSS color value | `lineColor`, `colors.supercomputer` |
| Line width | `lineWidth` / `*LineWidth` | `GRID_CONFIG.lineWidth` |
## Earth Base and HD Texture
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Earth base radius | `CONFIG.earthRadius` | `100` | `earth.js:createEarth()` |
| Earth base color | `EARTH_MATERIAL_CONFIG.color` | `0x010609` | `MeshPhongMaterial.color` |
| Earth base emissive | `EARTH_MATERIAL_CONFIG.emissive` | `0x010609` | `MeshPhongMaterial.emissive` |
| Earth base specular | `EARTH_MATERIAL_CONFIG.specular` | `0x1a2d45` | `MeshPhongMaterial.specular` |
| Earth base shininess | `EARTH_MATERIAL_CONFIG.shininess` | `12` | `MeshPhongMaterial.shininess` |
| Earth base opacity | `EARTH_MATERIAL_CONFIG.opacity` | `1` | `MeshPhongMaterial.opacity` |
| HD texture radius offset | `EARTH_MATERIAL_CONFIG.textureOverlayAltitudeOffset` | `0.1` | Standalone HD texture sphere radius |
| HD texture opacity | `EARTH_MATERIAL_CONFIG.textureOverlayOpacity` | `0.88` | HD texture `MeshPhongMaterial.opacity` |
| HD texture renderOrder | `EARTH_MATERIAL_CONFIG.textureOverlayRenderOrder` | `0.96` | `_earthTextureOverlay.renderOrder` |
| HD texture specular | `EARTH_MATERIAL_CONFIG.textureOverlaySpecular` | `0x05080d` | Reduces specular highlight in direct-light areas to avoid blown-out texture |
| HD texture shininess | `EARTH_MATERIAL_CONFIG.textureOverlayShininess` | `4` | Reduces specular concentration |
| HD texture color multiplier | inline | `0xffffff` | `_earthTextureOverlayMaterial.color` |
## Earth Occluder and Day/Night
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Occluder radius factor | `EARTH_MATERIAL_CONFIG.occluderRadiusFactor` | `0.999` | Depth occluder sphere radius |
| Occluder segments | `EARTH_MATERIAL_CONFIG.occluderSegments` | `48` | Occluder geometry segments |
| Occluder renderOrder | inline | `-1` | `occluder.renderOrder` |
| Day/night sun direction | `EARTH_MATERIAL_CONFIG.dayNight.sunDirection` | `{ x: 1, y: 0.2, z: 0.4 }` | Custom day/night shader |
| Night-side minimum brightness | `EARTH_MATERIAL_CONFIG.dayNight.nightFloor` | `0.24` | Shader uniform |
| Day-side boost | `EARTH_MATERIAL_CONFIG.dayNight.dayBoost` | `1.12` | Shader uniform |
| Twilight width | `EARTH_MATERIAL_CONFIG.dayNight.twilightWidth` | `0.2` | Shader uniform |
| Twilight intensity | `EARTH_MATERIAL_CONFIG.dayNight.twilightIntensity` | `0.14` | Shader uniform |
| Twilight color | `EARTH_MATERIAL_CONFIG.dayNight.twilightColor` | `0x4ea0ff` | Shader uniform |
| Night tint color | `EARTH_MATERIAL_CONFIG.dayNight.nightTintColor` | `0x0b1830` | Shader uniform |
| Night tint intensity | `EARTH_MATERIAL_CONFIG.dayNight.nightTintIntensity` | `0.08` | Shader uniform |
## Atmospheric Glow and Clouds
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Inner atmosphere radius factor | `EARTH_MATERIAL_CONFIG.atmosInnerRadiusFactor` | `1.01` | `atmosInnerGeo` |
| Inner atmosphere segments | `EARTH_MATERIAL_CONFIG.atmosInnerSegments` | `64` | `atmosInnerGeo` |
| Inner atmosphere color | `EARTH_MATERIAL_CONFIG.atmosInnerColor` | `[0.25, 0.62, 1.0]` | Shader RGB |
| Inner atmosphere rim power | `EARTH_MATERIAL_CONFIG.atmosInnerRimPower` | `3.2` | Shader rim falloff |
| Inner atmosphere intensity | `EARTH_MATERIAL_CONFIG.atmosInnerIntensity` | `0.18` | Shader alpha multiplier |
| Outer atmosphere radius factor | `EARTH_MATERIAL_CONFIG.atmosOuterRadiusFactor` | `1.016` | `atmosOuterGeo` |
| Outer atmosphere segments | `EARTH_MATERIAL_CONFIG.atmosOuterSegments` | `48` | `atmosOuterGeo` |
| Outer atmosphere color | `EARTH_MATERIAL_CONFIG.atmosOuterColor` | `[0.18, 0.45, 0.9]` | Shader RGB |
| Outer atmosphere rim power | `EARTH_MATERIAL_CONFIG.atmosOuterRimPower` | `5.0` | Shader rim falloff |
| Outer atmosphere intensity | `EARTH_MATERIAL_CONFIG.atmosOuterIntensity` | `0.02` | Shader alpha multiplier |
| Atmosphere blending | inline | `THREE.AdditiveBlending` | `ShaderMaterial.blending` |
| Atmosphere renderOrder | inline | `1` | `atmosInner/Outer.renderOrder` |
| No-HD-texture rim glow color | `EARTH_MATERIAL_CONFIG.rimGlowColor` | `[0.35, 0.65, 1.0]` | Fresnel shell RGB when HD texture is hidden or unavailable |
| No-HD-texture rim glow power | `EARTH_MATERIAL_CONFIG.rimGlowPower` | `3.8` | Shader rim falloff; higher = narrower edge |
| No-HD-texture rim glow intensity | `EARTH_MATERIAL_CONFIG.rimGlowIntensity` | `0.28` | Shader alpha multiplier |
| No-HD-texture rim glow segments | `EARTH_MATERIAL_CONFIG.rimGlowSegments` | `64` | `earth-rim-glow` geometry segments |
| Cloud layer radius offset | `CLOUD_LAYER_CONFIG.radiusOffset` | `3` | Cloud sphere radius |
| Cloud layer segments | `CLOUD_LAYER_CONFIG.widthSegments / heightSegments` | `64 / 64` | Cloud sphere geometry |
| Cloud layer opacity | `CLOUD_LAYER_CONFIG.opacity` | `0.15` | `MeshPhongMaterial.opacity` |
| Cloud texture | `CLOUD_LAYER_CONFIG.textureUrl` | `"./assets/earth_clouds_1024.png"` | Cloud texture map |
| Cloud blending | inline | `THREE.AdditiveBlending` | `MeshPhongMaterial.blending` |
## Land/Ocean Base and Country Borders
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Country border data path | `COUNTRY_BOUNDARY_CONFIG.dataPath` | `"/earth/data/countries-admin0.min.geojson"` | GeoJSON input |
| Ocean fill color | local `OCEAN_HEX` | `0x010609` | Land/ocean base canvas background |
| Land fill color | `COUNTRY_BOUNDARY_CONFIG.landColor` | `0x080f1b` | Land/ocean base canvas land |
| Land/ocean base opacity | `COUNTRY_BOUNDARY_CONFIG.landOpacity` | `1.0` | `MeshBasicMaterial.opacity` |
| Land/ocean base radius offset | `COUNTRY_BOUNDARY_CONFIG.landAltitudeOffset` | `0.08` | `country-land-ocean` radius |
| Land/ocean base renderOrder | `COUNTRY_BOUNDARY_CONFIG.landRenderOrder` | `0.86` | `country-land-ocean.renderOrder` |
| Land/ocean mask size | `landMaskWidth / landMaskHeight` | `2048 / 1024` | Canvas / DataTexture size |
| Country tint color | `COUNTRY_BOUNDARY_CONFIG.tintColor` | `0x0b1830` | Tint when HD texture is off |
| Country tint radius offset | `COUNTRY_BOUNDARY_CONFIG.tintAltitudeOffset` | `0.04` | `country-tint` radius |
| Country tint renderOrder | `COUNTRY_BOUNDARY_CONFIG.tintRenderOrder` | `0.2` | `country-tint.renderOrder` |
| Border line color | `COUNTRY_BOUNDARY_CONFIG.lineColor` | `0x7fc7ff` | Normal border line |
| Border line opacity | `COUNTRY_BOUNDARY_CONFIG.lineOpacity` | `0.58` | Normal border line opacity |
| Border dimmed opacity on hover | `COUNTRY_BOUNDARY_CONFIG.dimmedLineOpacity` | `0.18` | Normal border opacity during hover |
| Border line radius offset | `COUNTRY_BOUNDARY_CONFIG.lineAltitudeOffset` | `0.24` | Normal border line radius |
| Border line renderOrder | `COUNTRY_BOUNDARY_CONFIG.lineRenderOrder` | `2.2` | Normal border line level |
| Border hover color | `COUNTRY_BOUNDARY_CONFIG.hoverLineColor` | `0xff3b1f` | Neon red-orange |
| Border hover opacity | `COUNTRY_BOUNDARY_CONFIG.hoverLineOpacity` | `1.0` | Hover line opacity |
| Border hover radius offset | `COUNTRY_BOUNDARY_CONFIG.hoverAltitudeOffset` | `0.32` | Hover line radius |
| Border hover renderOrder | `COUNTRY_BOUNDARY_CONFIG.hoverLineRenderOrder` | `2.3` | Hover line level |
| Border hover glow opacity | `COUNTRY_BOUNDARY_CONFIG.hoverGlowOpacity` | `0.38` | Glow line opacity |
| Border hover glow line width | `COUNTRY_BOUNDARY_CONFIG.hoverGlowLineWidth` | `3` | Glow `LineBasicMaterial.linewidth` |
| Border hover glow level offset | `COUNTRY_BOUNDARY_CONFIG.hoverGlowRenderOrderOffset` | `0.01` | Glow renderOrder = `2.29` |
| Border hover glow radius offset | `COUNTRY_BOUNDARY_CONFIG.hoverGlowRadiusOffset` | `0.04` | Glow radius = hover radius + 0.04 |
## Real Terrain
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Terrain tile size | `TERRAIN_CONFIG.tileSize` | `256` | Terrarium tile read size |
| Terrain base zoom | `TERRAIN_CONFIG.baseZoom` | `4` | Terrain sampling zoom |
| Terrain geometry segments | `geometryWidthSegments / geometryHeightSegments` | `320 / 320` | Terrain sphere geometry |
| Terrain base radius offset | `TERRAIN_CONFIG.baseRadiusOffset` | `0.16` | Terrain overlays HD texture |
| Terrain exaggeration | `TERRAIN_CONFIG.exaggeration` | `34` | Elevation to world units |
| Terrain land fade height | `TERRAIN_CONFIG.landRevealFadeMeters` | `220` | Vertex alpha for coastline fade |
| Terrain opacity | `TERRAIN_CONFIG.opacity` | `0.68` | `MeshPhongMaterial.opacity` |
| Terrain color | `TERRAIN_CONFIG.color` | `0x8aa884` | `MeshPhongMaterial.color` |
| Terrain emissive | `TERRAIN_CONFIG.emissive` | `0x030704` | Reduces self-emission to preserve terrain shading |
| Terrain specular | `TERRAIN_CONFIG.specular` | `0x344438` | Gives terrain local sheen without boosting HD texture brightness |
| Terrain shininess | `TERRAIN_CONFIG.shininess` | `16` | Tightens terrain highlight |
| Terrain renderOrder | inline | `1.2` | `terrain.renderOrder` |
| Terrain polygonOffset | inline | `factor -1`, `units -1` | Reduces z-fighting near sphere surface |
## Grid Lines
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Grid radius offset | `GRID_CONFIG.radiusOffset` | `0.14` | Grid sphere radius |
| Grid color | `GRID_CONFIG.color` | `0xc0e0ff` | `LineBasicMaterial.color` |
| Grid opacity | `GRID_CONFIG.opacity` | `0.08` | `LineBasicMaterial.opacity` |
| Grid line width | `GRID_CONFIG.lineWidth` | `1` | `LineBasicMaterial.linewidth` |
| Grid renderOrder | `GRID_CONFIG.renderOrder` | `2.05` | Grid level |
| Latitude step | `GRID_CONFIG.latitudeStep` | `15` | Latitude line generation step |
| Longitude step | `GRID_CONFIG.longitudeStep` | `30` | Longitude line generation step |
| Segment sample step | `GRID_CONFIG.segmentStep` | `5` | Grid line sample step |
## Submarine Cables and Landing Points
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Default cable color | `CABLE_COLORS.default` | `0xffff44` | Used when no data color available |
| Cable radius offset | `CABLE_CONFIG.line.altitudeOffset` | `0.2` | Cable line radius |
| Cable line width | `CABLE_CONFIG.line.lineWidth` | `1` | `LineBasicMaterial.linewidth` |
| Cable opacity | `CABLE_CONFIG.line.opacity` | `1.0` | Cable line opacity |
| Cable renderOrder | `CABLE_CONFIG.line.renderOrder` | `1` | Cable line level |
| Landing point radius offset | `CABLE_CONFIG.landingPoint.altitudeOffset` | `0.48` | Aligns with compute center marker height |
| Landing point icon texture size | `CABLE_CONFIG.landingPoint.textureSize` | `256` | Canvas size for solid map-pin icon |
| Landing point icon aspect ratio | `CABLE_CONFIG.landingPoint.iconAspectRatio` | `0.82` | `Sprite.scale.x = height * aspect` |
| Landing point icon anchor | `CABLE_CONFIG.landingPoint.anchorX / anchorY` | `0.52 / 0.276` | `Sprite.center`, aligns pin tip to landing point lat/lon |
| Landing point base scale | `CABLE_CONFIG.landingPoint.baseScale` | `12` | Matches compute center sprite height |
| Landing point color | `CABLE_CONFIG.landingPoint.color` | `0xffaa00` | `SpriteMaterial.color` |
| Landing point opacity | `CABLE_CONFIG.landingPoint.opacity` | `1.0` | `SpriteMaterial.opacity` |
| Landing point renderOrder | `CABLE_CONFIG.landingPoint.renderOrder` | `4.5` | Aligns with compute center surface level |
| Landing point dim brightness | `landingPointVisual.dimBrightness` | `0.62` | Dim state color multiplier |
| Dimmed landing point color | `landingPointVisual.dimmed.colorRGB` | `{ r: 180, g: 116, b: 28 }` | Dim state color; avoids dark base showing through as a dark hole |
| Dimmed landing point emissive | `landingPointVisual.dimmed.emissive` | `0x3a2200` | Dim state weak amber self-emission |
| Dimmed landing point opacity | `landingPointVisual.dimmed.opacity` | `0.78` | Dim state opacity; no longer uses low alpha blending with dark base |
## Satellites, Trails, and Footprints
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Satellite display radius offset | `SATELLITE_CONFIG.displayAltitudeOffset` | `8` | Satellite point position |
| Satellite dot base pixel size | `SATELLITE_CONFIG.dotBaseSize` | `2.8` | Point shader size |
| Satellite backdrop dot scale | `SATELLITE_CONFIG.dotBackdropScale` | `1.28` | Backdrop dot size |
| Satellite dot opacity range | `dotOpacityMin / dotOpacityMax` | `0.7 / 1.0` | Breathing animation |
| Satellite dot breathing speed | `SATELLITE_CONFIG.dotBreathingSpeed` | `0.12` | Dot opacity animation |
| Satellite backdrop renderOrder | inline | `5` | `satelliteBackdropPoints.renderOrder` |
| Satellite dot renderOrder | inline | `6` | `satellitePoints.renderOrder` |
| Satellite trail length | `SATELLITE_CONFIG.trailLength` | `10` | Trail buffer |
| Satellite trail line width | `SATELLITE_CONFIG.trailLineWidth` | `3` | Ribbon shader uniform |
| Selected ring size | `SATELLITE_CONFIG.ringSize` | `0.07` | Hover / locked ring sprite |
| Satellite overlay renderOrder | `SATELLITE_CONFIG.overlayRenderOrder` | `12` | Locked ring / halo / orbit |
| Footprint renderOrder | local `GROUND_FOOTPRINT_RENDER_ORDER` | `3` | Footprint fill |
## Compute Centers
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Compute center radius offset | `COMPUTE_CENTER_CONFIG.altitudeOffset` | `0.48` | Marker position |
| Compute center base opacity | `COMPUTE_CENTER_CONFIG.marker.baseOpacity` | `0.88` | `SpriteMaterial.opacity` |
| Supercomputer marker scale | `COMPUTE_CENTER_CONFIG.marker.supercomputerScale` | `12` | Supercomputer marker |
| GPU cluster marker scale | `COMPUTE_CENTER_CONFIG.marker.gpuClusterScale` | `12` | GPU marker |
| Hover scale | `COMPUTE_CENTER_CONFIG.marker.hoverScale` | `1.16` | Hover state |
| Locked scale | `COMPUTE_CENTER_CONFIG.marker.lockedScale` | `1.22` | Locked state |
| Dimmed scale / opacity | `dimmedScale / dimmedOpacity` | `0.82 / 0.34` | Dim state |
| Supercomputer color | `COMPUTE_CENTER_CONFIG.colors.supercomputer` | `"#38bdf8"` | Marker texture |
| GPU cluster color | `COMPUTE_CENTER_CONFIG.colors.gpu_cluster` | `"#2dd4bf"` | Marker texture |
| Linked color | `COMPUTE_CENTER_CONFIG.colors.linked` | `"#f8fafc"` | Linked state |
| Compute center renderOrder | local `COMPUTE_CENTER_RENDER_ORDER` | `4.5` | Surface facility below satellites |
## BGP Observation
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| BGP event radius offset | `BGP_CONFIG.altitudeOffset` | `2.1` | Anomaly marker |
| BGP collector radius offset | `BGP_CONFIG.collectorAltitudeOffset` | `1.6` | Collector marker |
| Event base scale | `BGP_CONFIG.marker.eventBaseScale` | `6.2` | Anomaly sprite |
| Collector base scale | `BGP_CONFIG.marker.collectorBaseScale` | `7.4` | Collector plane |
| Hover / dim scale | `hoverScale / dimmedScale` | `1.16 / 0.92` | Interaction states |
| Normal event opacity | `BGP_CONFIG.opacity.normal` | `0.78` | Anomaly sprite |
| Hover opacity | `BGP_CONFIG.opacity.hover` | `1.0` | Hover state |
| Dimmed opacity | `BGP_CONFIG.opacity.dimmed` | `0.24` | Dim state |
| Collector opacity | `BGP_CONFIG.opacity.collector` | `0.62` | Collector state |
| Critical color | `BGP_CONFIG.severityColors.critical` | `0xff4d4f` | Critical event |
| High color | `BGP_CONFIG.severityColors.high` | `0xff9f43` | High-severity event |
| Medium color | `BGP_CONFIG.severityColors.medium` | `0xffd166` | Medium-severity event |
| Low color | `BGP_CONFIG.severityColors.low` | `0x4dabf7` | Low-severity event |
| Collector base color | `BGP_CONFIG.collectorColor` | `0x6db7ff` | Default collector color |
| Region color | `BGP_CONFIG.regionColor` | `0x2dd4bf` | Region overlay |
## Celestial and Starfield
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Sky sphere radius | `CELESTIAL_CONFIG.skyRadius` | `2600` | Celestial background |
| Sky opacity | `CELESTIAL_CONFIG.skyOpacity` | `1` | Background material |
| Sun distance / scale | `sunDistance / sunScale` | `2150 / 78` | Sun sprite |
| Moon distance / scale | `moonDistance / moonScale` | `2050 / 38` | Moon sprite |
| Sun halo scale | `CELESTIAL_CONFIG.sunHaloScale` | `136` | Sun halo |
| Moon halo scale | `CELESTIAL_CONFIG.moonHaloScale` | `62` | Moon halo |
| Sun light color / intensity | `sunLightColor / sunLightIntensity` | `0xfff4df / 1.02` | Scene light |
| Back light color / intensity | `backLightColor / backLightIntensity` | `0x2b4c78 / 0.3` | Scene light |
| Star count | `STARFIELD_CONFIG.count` | `8000` | `createStars()` |
| Star radius range | `minRadius + radiusJitter` | `800 + 200` | Random distribution |
| Star color | `STARFIELD_CONFIG.color` | `0xffffff` | `PointsMaterial.color` |
| Star size | `STARFIELD_CONFIG.size` | `0.5` | `PointsMaterial.size` |

View File

@@ -0,0 +1,175 @@
# News Live Streams Collector Format
The `news_live_streams` collector accepts a "channel directory JSON" as input rather than scraping web pages directly.
Goals:
- Allow the backend to stably ingest live news streams from around the world
- Ensure the Earth page TV module always consumes a consistent structure
- Make it easy to integrate channel directories like `worldmonitor` that mix YouTube / HLS / iframe sources
## Recommended JSON Structure
```json
{
"sources": [
{
"id": "bbc-world-news",
"name": "BBC World News",
"provider": "BBC",
"region": "UK",
"language": "en",
"source_type": "youtube",
"youtube_video_id": "dQw4w9WgXcQ",
"youtube_channel": "https://www.youtube.com/@BBCNews",
"embed_url": "",
"stream_url": "",
"homepage_url": "https://www.youtube.com/@BBCNews/live",
"poster_url": "",
"sort_order": 220,
"is_enabled": true,
"notes": "Primary English global news channel"
},
{
"id": "france24-en",
"name": "France 24 English",
"provider": "France 24",
"region": "France",
"language": "en",
"source_type": "hls",
"stream_url": "https://example.com/live.m3u8",
"homepage_url": "https://www.france24.com/en/live",
"sort_order": 230,
"is_enabled": true
},
{
"id": "cctv4-page",
"name": "CCTV-4 Chinese International",
"provider": "CCTV",
"region": "China",
"language": "zh-CN",
"source_type": "iframe",
"embed_url": "https://tv.cctv.com/live/cctv4/",
"homepage_url": "https://tv.cctv.com/live/cctv4/",
"sort_order": 10,
"is_enabled": true
}
]
}
```
## Field Conventions
- `id`: unique identifier, should be stable
- `name`: channel display name
- `provider`: provider name
- `region`: country or region
- `language`: language code
- `source_type`: `iframe` / `hls` / `video` / `external` / `youtube`
- `embed_url`: page suitable for iframe embedding
- `stream_url`: direct video stream URL
- `homepage_url`: official website or channel page
- `youtube_video_id`: YouTube live video ID
- `youtube_channel`: YouTube channel handle or channel URL
- `poster_url`: cover image, optional
- `sort_order`: sort value, smaller = higher in the list
- `is_enabled`: whether enabled
- `notes`: brief notes
## Panel Behavior Conventions
- `youtube`
- Prefers `youtube_video_id`
- When embedding is not possible, at least keep `youtube_channel` or `homepage_url` for external opening
- `hls` / `video`
- Prefers `stream_url`
- `iframe`
- Prefers `embed_url`
- `external`
- No embedding attempt; only keeps external open link
## Current Implementation Status
- The backend settings page supports manually maintaining channel directories
- The Earth TV module merges:
- Manually configured sources
- Sources collected by the `news_live_streams` collector
- The current default fallback source is CCTV-4 Chinese International
- When no override is configured, `news_live_streams` defaults to `iptv-org`:
- `channels.json`
- `streams.json`
- `logos.json`
and automatically filters for news-category channel directories
## Collector Configuration
`news_live_streams` does not need a separate new page; it reuses the existing data source configuration:
- `endpoint`
- Channel directory JSON API URL
- `auth_type`
- `none` / `bearer` / `api_key` / `basic`
- `headers`
- Additional request headers
- `config`
- Collector request and parsing behavior
### Supported `config` Fields
```json
{
"timeout": 30,
"method": "GET",
"params": {
"region": "global"
},
"body_type": "json",
"body": {
"include_disabled": false
},
"response_path": "payload.channels"
}
```
- `timeout`: request timeout in seconds
- `method`: `GET` or `POST`
- `params`: query parameter object
- `body_type`: `json` or `form`
- `body`: request body for `POST`
- `json_body`: explicit JSON request body, takes priority over `body`
- `form_body`: explicit form request body, takes priority over `body`
- `response_path`: path to the channel array in the response JSON, supports dot notation, e.g.:
- `payload.channels`
- `data.items`
- `result.streams`
### Authentication Details
- `bearer`: uses `Authorization: Bearer <token>`
- `api_key`: sent as request header by default; if `auth_config.in = "query"`, sent as query param
- `basic`: uses HTTP Basic Authorization
## Compatible Response Structures
The collector first tries to read:
- Top-level array
- Or an array under these common fields:
- `sources`
- `streams`
- `channels`
- `items`
- `results`
- `data`
It also accepts these field aliases:
- `id` / `source_id` / `slug` / `channel_id` / `code`
- `name` / `title` / `channel` / `display_name`
- `provider` / `publisher` / `network`
- `stream_url` / `stream` / `playback_url` / `hls_url` / `m3u8_url`
- `embed_url` / `embed` / `page_url`
- `homepage_url` / `source_url` / `website`
- `language` / `lang` / `locale`
- `youtube_video_id` / `video_id`
- `youtube_channel` / `channel_handle`

View File

@@ -0,0 +1,56 @@
# Earth Render Layer Order
This document records the current Earth renderer's layer order and the intent of each layer. When adjusting `renderOrder`, radius offsets, depth strategy, or pointer interaction, update this document accordingly.
Note: the layer control panel order and the registration / startup load order are two separate semantics.
| Order type | Current sequence | Notes |
| --- | --- | --- |
| Control panel order | Cables → Trails → Satellites → Compute Centers → BGP → Terrain → HD Texture → Cloud Layer → Borders → Grid | Controlled by `displayOrder`, sorted by operational relevance. |
| Registration / startup load order | Grid → Borders → HD Texture → Cloud Layer → Cables → Compute Centers → BGP → Satellites | Controlled by registration order and `startupPriority`, sorted surface-to-sky; Trails and Terrain are dependency/optional display layers and do not participate in normal startup data loading. |
## Surface Layer Stack
| Order | Layer | Source | Render / Radius Strategy | Depth / Interaction Strategy | Notes |
| --- | --- | --- | --- | --- | --- |
| -1000 | Celestial background mesh | `celestial.js` | Background sphere | Not part of surface picking | Behind all Earth content. |
| -1 | Earth occluder sphere | `earth.js` | Invisible inner sphere | Writes depth buffer | Occludes objects behind the Earth. |
| 0 | Earth base sphere | `earth.js` | `CONFIG.earthRadius` | Surface picking fallback target | Dark base; still visible when all optional map layers are off. |
| 0.2 | Country dark tint | `country-boundaries.js` | `tintAltitudeOffset` | Raycast disabled | Used when HD texture is off. |
| 0.86 | Land/ocean base fill | `country-boundaries.js` | `landAltitudeOffset`; ocean `#010609`, land `#080f1b` | Raycast disabled | Base map remains usable even when country borders are off. |
| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset` | Surface picking target when visible | HD texture always overlays the land/ocean base fill. |
| 1 | Atmospheric glow and clouds | `earth.js` | Atmosphere / cloud spheres | Not in normal object selection path | Cloud layer controlled by the "Cloud Layer" toggle. |
| 1 | Submarine cables | `cables.js` | `CABLE_CONFIG.line.renderOrder` | Cable picking path | Preserves existing cable layer level. |
| 1.2 | Real terrain | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` plus terrain displacement | Raycast disabled | Terrain overlays HD texture; temporarily hidden when HD texture is off, restores to prior state when re-enabled. |
| 2.05 | Grid lines | `earth.js` | `CONFIG.earthRadius + 0.14` | Raycast disabled | Low-opacity lines over HD texture. |
| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset` | Raycast disabled | Only needs to stay above HD texture. |
| 2.29 | Country border hover glow | `country-boundaries.js` | Hover radius + glow offset | `depthTest: false`, raycast disabled | Additive glow to reinforce border edge and terrain hover visibility. |
| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset` | `depthTest: false`, raycast disabled | Neon red-orange hover line; China and Taiwan share the same highlight group. |
| 3 | Satellite footprint fill | `satellites.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-tested, Group renderOrder stays 0 | Footprint above country borders, below compute centers and satellites. |
| 3-5 | BGP markers and overlays | `bgp.js` | Each marker's own renderOrder | BGP picking path | Preserves existing BGP visual level. |
| 4.5 | Compute centers | `compute-centers.js` | `COMPUTE_CENTER_RENDER_ORDER` | Compute center picking path | Surface facilities, below satellites. |
| 5 | Satellite background dot | `satellites.js` | Fixed renderOrder | Screen-space satellite picking | Below satellite dots. |
| 6 | Satellite dots | `satellites.js` | Fixed renderOrder | Screen-space satellite picking | Satellite dots above footprints and compute centers. |
| 12+ | Satellite locked ring, halo, predicted orbit | `satellites.js` | `SATELLITE_CONFIG.overlayRenderOrder` and offsets | Satellite overlay path | Used for selected/locked satellite emphasis. |
| 98-100 | Sun / moon halo and sprite | `celestial.js` | Fixed renderOrder | Celestial picking disabled | Foreground celestial sprites. |
## Toggle Behavior
| Toggle | Behavior |
| --- | --- |
| HD texture off | Hides HD texture, enables country tint / base surface, disables terrain and day/night toggle interaction, and remembers terrain and day/night previous states. |
| HD texture on | Restores HD texture and the remembered terrain / day/night states. |
| Terrain on | Displayed above HD texture, but below country border hover, footprints, satellites, and other emphasis layers. |
| Cloud layer | Only controls cloud mesh visibility. |
| Country borders | Controls border line and hover line visibility; land/ocean base fill exists independently as the Earth base map. |
## Interaction Rules
| Interaction | Current Rule |
| --- | --- |
| Earth coordinate hover | When HD texture is visible, uses the HD texture overlay as the surface picking target; otherwise uses the Earth base sphere. |
| Country border hover | Converts surface pick coordinates to lat/lon, then uses GeoJSON point-in-polygon; the border hover line itself does not receive raycasts. |
| Country border hover visual | On hover, dims normal border lines and draws no-depth-test glow and solid lines. |
| China / Taiwan hover | `CHN` and `TWN` are grouped in the same hover highlight group; the tooltip still shows the actually-hit feature. |
| Terrain | Acts as a visual layer only; `terrain.raycast` is disabled. |
| Satellites | Uses screen-space satellite picking to prevent footprints or surface layers from blocking satellite clicks. |

View File

@@ -0,0 +1,198 @@
# Earth Satellite Footprint Policy
This document records the current product boundary, data rationale, and implemented behavior for `footprint` in the Earth satellite layer. The goal is to prevent the Starlink-specific ground coverage model from being misapplied to other constellations.
Related context:
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/en/earth-frontend-context.md)
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/en/backend-collectors.md)
- [backend/app/services/collectors/celestrak.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/celestrak.py)
- [frontend/public/earth/js/satellites.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/satellites.js)
## Current Goal
- Define which non-Starlink satellites should not show a ground footprint
- Define which constellations may have their own footprint in the future but cannot reuse the Starlink bowtie / GSO-gap model
- Solidify this policy as an executable implementation boundary, not leave it scattered across visual parameters
## Current Local Categories
Current CelesTrak satellite groups in [backend/app/services/collectors/celestrak.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/celestrak.py) include:
- `starlink`
- `gps-ops`
- `galileo`
- `glonass`
- `beidou`
- `leo`
- `geo`
- `iridium-next`
Non-Starlink categories:
- `gps-ops`
- `galileo`
- `glonass`
- `beidou`
- `leo`
- `geo`
- `iridium-next`
## Research Conclusions
### 1. GNSS / RNSS: `gps-ops`, `galileo`, `glonass`, `beidou`
Do not draw a localized ground footprint by default.
Reason:
- Public sources emphasize `Earth-pointing`, `Earth coverage`, `continuous global coverage`
- The public semantic of these systems is global navigation / timing coverage, not the localized spot footprint associated with Starlink's end-user service
More appropriate representation:
- Default: show only the satellite body and orbit
- If future needs require showing "service reachability," only a weak global coverage semantic is appropriate — do not draw a localized ground spot
References:
- [GPS III EC Antenna Patterns](https://www.navcen.uscg.gov/sites/default/files/pdf/gps/GPS_ZIP/GPS_III_EC_Antenna_Patterns_SVN_74_75_76_77_78.pdf)
- [ESA Galileo satellites](https://www.esa.int/Applications/Satellite_navigation/Galileo/Galileo_satellites)
- [Navipedia Galileo General Introduction](https://gssc.esa.int/navipedia/index.php/Galileo_General_Introduction)
- [BeiDou official overview](https://www.beidou.gov.cn/xt/gfxz/201812/P020190117356387956569.pdf)
- [GPS.gov GNSS overview](https://www.gps.gov/systems/gnss/)
### 2. `iridium-next`
Can have a footprint, but cannot reuse Starlink's single bowtie footprint.
Reason:
- Iridium NEXT public documentation emphasizes a fixed multi-spot beam system
- Public examples commonly show `48 fixed spot beams in 4 tiers`
- This is not the same problem as Starlink's "single satellite, single primary footprint, with GSO gap" business visualization
More appropriate representation:
- Default: still do not draw a Starlink-style ground footprint
- Future implementation: connect an independent Iridium multi-beam adapter layer
- Visually closer to multi-beam clusters / honeycomb / layered beams, not a single bowtie spot
Reference:
- [Iridium Satellite Spot Beam Coverage on the US](https://www.mathworks.com/help/phased/ug/iridium-satellite-spot-beam-coverage-on-the-us-1.html)
### 3. `geo`
Do not draw a unified footprint by default.
Reason:
- GEO communication satellites may use global beam, zone beam, spot beam, or steerable spot beam
- Without operator / payload / beam contour metadata, drawing a unified footprint is very likely incorrect
More appropriate representation:
- Default: show only the GEO belt and satellite parking position semantics
- Only allow footprint drawing when beam contour / operator metadata is available
Reference:
- [ITU Handbook on Satellite](https://www.itu.int/dms_pub/itu-r/opb/hdb/R-HDB-42-2002-PDF-E.pdf)
### 4. `leo` (generic)
Do not draw a footprint by default.
Reason:
- The `leo` group is too mixed — it may include communication, remote sensing, experimental, and observation satellites
- Without mission / payload / antenna pattern metadata, there is no basis for a service-coverage visualization
More appropriate representation:
- Default: show only the satellite and orbit
- Future: if subdivided by operator / mission subtype, decide then whether to introduce an independent coverage mode
## Product Policy
Current unified policy:
- `Starlink`
- Keep the current dedicated `ground_footprint` logic
- `Iridium NEXT`
- Reserve an independent adapter layer
- Do not reuse Starlink footprint currently
- `GPS / Galileo / GLONASS / BeiDou`
- No ground footprint
- `GEO`
- No footprint without beam metadata
- `Generic LEO`
- No footprint without mission metadata
## Implemented Behavior
This implementation only does the minimum executable version and does not change existing Starlink visual parameters:
1. Backend passes constellation group and footprint policy hint to the frontend
- CelesTrak collector stores `GROUP` in `metadata.constellation_group`
- Visualization API outputs:
- `properties.constellation_group`
- `properties.footprint_policy`
Current policy values:
- `starlink_ground_footprint`
- `iridium_coverage_ring`
- `none`
Relevant code:
- [backend/app/services/collectors/celestrak.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/celestrak.py)
- [backend/app/api/v1/visualization.py](/home/ray/dev/linkong/planet/backend/app/api/v1/visualization.py)
2. Frontend makes footprint a capability-gated renderer
- `ground_footprint` is only actually enabled when `footprint_policy === starlink_ground_footprint`
- `iridium-next` no longer falls back to a placeholder branch; it goes through an independent Iridium coverage ring adapter
- Other non-Starlink satellites automatically fall back to `self_glow` even if the user globally selects `ground_footprint`
Relevant code:
- [frontend/public/earth/js/satellites.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/satellites.js)
- [frontend/public/earth/js/iridium-footprint-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/iridium-footprint-adapter.js)
3. Satellite info card shows capability, not just orbital parameters
- Satellite details now clearly display:
- `Constellation / Group`
- `Coverage Capability`
- `Current Display`
- `Coverage Model`
- Users can directly see:
- Whether the current satellite supports footprint
- Whether the current display has been fallen back due to capability gating
- That Iridium and Starlink use different models
Relevant code:
- [frontend/public/earth/js/main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js)
- [frontend/public/earth/js/info-card.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/info-card.js)
## Current Implementation Boundary
This boundary must be maintained:
- Starlink's footprint parameters and shader logic serve Starlink only
- Non-Starlink capability decisions belong to the "policy layer / adapter layer"
- Do not re-mix different constellations' coverage models into the same parameter set
- `iridium-next` has been separated into an independent adapter and should continue along this boundary rather than adding more if/else to the existing Starlink bowtie
## Recommended Next Steps
If continuing forward, the recommended order is:
1. Create a dedicated footprint adapter for `iridium-next`
2. Add a read-only indicator in the UI to tell users whether the current satellite supports footprint
3. If GEO beam contour / operator metadata becomes available, enable operator-specific footprint for GEO

View File

@@ -0,0 +1,293 @@
# Admin Frontend Context
This document describes the current real structure of the console frontend. The goal is to help future page development, table refactoring, layout governance, and state consolidation quickly find the right entry points.
Related references:
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md)
## Current Goal
The console frontend is a backend workbench, not a display-style dashboard. Current constraints:
- Pages default to a single-screen work area
- Primary interaction happens through in-module scrolling, not relying on the whole page growing infinitely
- Lists, tables, and analysis pages prioritize keeping the main work area visible
- Common layout, scrollbar, and table scroll behavior should be reused across pages
## Current Route Entry Points
Main entry point:
- [App.tsx](/home/ray/dev/linkong/planet/frontend/src/App.tsx)
Current admin-related routes:
- `/admin`
- `/users`
- `/datasources`
- `/data`
- `/alerts/system`
- `/alerts/bgp`
- `/alerts/situational`
- `/bgp`
- `/playground`
- `/settings`
`/earth` is a standalone display page and is not part of the console shell.
## Current Page Shell
The console shared shell is at:
- [AppLayout.tsx](/home/ray/dev/linkong/planet/frontend/src/components/AppLayout/AppLayout.tsx)
Responsibilities:
- Left-side navigation
- Collapse and expand
- Current account / version information
- Content area height closure
- Site-wide unified sidebar scrollbar
Current structure:
```tsx
<Layout className="dashboard-layout">
<Sider className="dashboard-sider">...</Sider>
<Layout>
<Content className="dashboard-content">
<div className="dashboard-content-inner">{children}</div>
</Content>
</Layout>
</Layout>
```
Future console pages should adapt to this shell rather than redefining full-page height semantics.
## Current Shared Components
### 1. `Scrollbar`
File:
- [Scrollbar.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/Scrollbar.tsx)
Purpose:
- Ordinary content containers like the console sidebar
- Internally manages visibility, thumb size, drag, and dual-axis overflow detection
Current constraint:
- The scrollbar must be a floating overlay that does not participate in layout
- Should leave no visible trace when there is no overflow
- Real scrolling is still handled by the native container; only the visible layer and interaction layer are replaced
### 2. `ScrollbarOverlay`
File:
- [ScrollbarOverlay.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/ScrollbarOverlay.tsx)
Purpose:
- Areas like Ant Table that already have an internal scroll container
- Does not take over scroll semantics; only adds a new scrollbar visible layer
Current usage:
- Data sources
- Collected data
- User management
- Settings page
- Alerts page
- BGP page
### 3. `TableScrollRegion`
File:
- [TableScrollRegion.tsx](/home/ray/dev/linkong/planet/frontend/src/components/Scrollbar/TableScrollRegion.tsx)
Purpose:
- Provides a unified wrapper for table scroll areas
- New table pages should reuse this rather than repeating the "table area + overlay scrollbar" boilerplate
### 4. `SegmentedControl`
Files:
- [SegmentedControl.tsx](/home/ray/dev/linkong/planet/frontend/src/components/SegmentedControl/SegmentedControl.tsx)
- [SegmentedControl.css](/home/ray/dev/linkong/planet/frontend/src/components/SegmentedControl/SegmentedControl.css)
Purpose:
- Segmented controls for language, theme, mode, or other 2 to 3 option settings
- Settings that need the shared animated slider, active state, and compact button layout
- The `/docs` footer language switcher and theme switcher already reuse it
Interface semantics:
- `options`: each option contains `value` and `label`, with optional `icon` and `title`
- `value`: current active value
- `onChange`: called when the selected option changes
- `ariaLabel`: accessible name for the control
- `className`: page-level hook for size or local style overrides
Current constraints:
- The component owns slider count, position, and spring-like transition
- Feature pages should only pass options and state, not recreate private slider DOM
- Prefer CSS variable overrides for colors instead of hard-coding theme colors in feature components
- Best for a small set of mutually exclusive choices; do not use it as a long list, navigation menu, or select replacement
### 5. `MarkdownRenderer`
File:
- [MarkdownRenderer.tsx](/home/ray/dev/linkong/planet/frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx)
Purpose:
- Renders Markdown content for `/docs`
- Supports headings, lists, blockquotes, code blocks, tables, and basic inline formatting
- Code blocks and tables reuse `Scrollbar` so horizontal content does not blow out the docs page
Current constraints:
- It is not a full GitHub Markdown engine; it only covers the syntax currently needed by project docs
- Internal document links should be converted to `/docs/:slug` through `transformLink`
- Heading anchors are injected through `getHeadingId`, keeping route state outside the renderer
### 6. `TableActions`
File:
- [TableActions.tsx](/home/ray/dev/linkong/planet/frontend/src/components/TableActions/TableActions.tsx)
Purpose:
- Shared action entry for table operation columns
- Shows inline actions when expanded
- Uses a more-actions dropdown when collapsed
Companion export:
- `actionCellProps`: for action-column `onCell`, preventing action buttons from being ellipsized or wrapped
## Current State Sources
### 1. Auth State
File:
- [auth.ts](/home/ray/dev/linkong/planet/frontend/src/stores/auth.ts)
Responsibilities:
- Token
- Current user
- Login / logout
`App.tsx` uses it to decide whether to redirect to the login page.
### 2. Business Data Gateway
AI / situational awareness related services are currently in:
- [http-gateway.ts](/home/ray/dev/linkong/planet/frontend/src/services/situational-awareness/http-gateway.ts)
- [port.ts](/home/ray/dev/linkong/planet/frontend/src/services/situational-awareness/port.ts)
- [types.ts](/home/ray/dev/linkong/planet/frontend/src/services/situational-awareness/types.ts)
Constraints:
- Pages must not scatter URL construction directly
- Define boundaries through port/types first
- Then implement via http/mock gateway
## Current Page Layer Recommendations
### 1. Dashboard and Summary Pages
Example:
- [Dashboard.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Dashboard/Dashboard.tsx)
Priority goals:
- Stable header
- Summary cards compact first
- Main work area occupies primary height
### 2. Table Pages
Examples:
- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx)
- [DataList.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataList/DataList.tsx)
- [Users.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Users/Users.tsx)
- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx)
Constraints:
- Prefer internal scrolling
- Do not let tables blow out the full page
- New table areas should reuse `TableScrollRegion` / `ScrollbarOverlay`
### 3. Complex Workspace Pages
Examples:
- [BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
- [Playground.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Playground/Playground.tsx)
Constraints:
- Tab content must not share the same height logic
- Table tabs, Markdown tabs, and config tabs each need their own scroll responsibility
- AI result areas and long text areas should maintain a minimum readable height
## Current Layout Constraints
These principles have been repeatedly validated in the project:
1. Parent container height chain must close
2. `min-height: 0` must not be omitted
3. Overflow responsibility must be explicit
4. Do not use `overflow: hidden` to mask structural issues
5. Do not compress the main work area to make summary cards show completely
6. Custom scrollbars must be floating overlays; they must not squeeze content width
For detailed experience, see:
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/en/frontend-layout-guidelines.md)
## Recommended Change Approach
For future console page changes:
1. Confirm whether the page is a summary page, table page, or complex workspace
2. Integrate into the existing shell and scroll semantics first
3. Reuse shared scroll components
4. Handle visual and detail interactions last
Do not write local CSS patches first, then retrofit the structure.
## Current Clear Boundary
The console frontend and the Earth frontend are not the same system:
- Console frontend: React + Ant Design workbench
- Earth frontend: independent native HUD system under `public/earth`
Therefore:
- Do not move Earth's HUD / animations / state machine directly into the console
- Do not force the console's table / scroll strategy onto the Earth HUD
For Earth-related structure, see:
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/en/earth-frontend-context.md)

View File

@@ -0,0 +1,309 @@
# Frontend Layout Guidelines
Admin pages in this project default to a "single-screen workspace" layout standard. The goal is not to prevent all overflow, but to ensure that under common desktop viewports:
- The main page structure is visible within one screen
- The user can simultaneously see the page header, summary area, and main workspace
- Overflow content scrolls within its module, rather than stretching the entire page vertically
Current recommended reference implementations:
- [frontend/src/pages/BGP/BGP.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/BGP/BGP.tsx)
- [frontend/src/index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
## Core Principles
### 1. Pages Should Prioritize a Single-Screen Workspace
Admin pages default to:
- Header: title, description, main actions
- Main workspace: stats cards, tables, charts, lists, tabs
Recommended structure:
```tsx
<AppLayout>
<div className="page-shell">
<div className="page-shell__header">...</div>
<div className="page-shell__body">...</div>
</div>
</AppLayout>
```
Total page height should be bounded within the `AppLayout` content area, not allowed to grow naturally downward without limit.
### 2. Scrolling Should Happen Inside Modules
If tables, logs, long lists, or chart details overflow their space:
- Let the card scroll internally
- Let the table scroll internally
- Let the tab content area scroll internally
Do not rely on full-page scrolling to "solve" the space problem.
### 3. The Main Workspace Must Get the Most Space
The most important module on a page must be the visual and spatial lead. Typically ensure:
- Header always visible
- Summary area height controlled
- Main table / chart / analysis area occupies more than 50% of visible height
If a page has multiple large modules, priority order is:
1. First compress the description and summary areas
2. Then move secondary modules into tabs or switch views
3. Only then consider adding more full-page scrolling
### 4. Small Screens and High Zoom Must Enter Compact Mode
When window height is low, width is narrow, or system zoom is high, actively switch to a compact layout:
- Reduce card padding
- Reduce header and cell spacing
- Convert summary area to a more compact single-row or horizontal-scroll layout
- Move secondary modules into tabs, drawers, or collapsed areas
Compact mode goal: maintain usability, not just shrink all text and controls.
### 5. Overflow Responsibility Must Be Explicit
Large content blocks on the page must explicitly define:
- Who is responsible for filling remaining height
- Who is responsible for clipping
- Who is responsible for scrolling
Common requirements:
- Parent container chain needs `min-height: 0`
- Workspace containers typically need `display: flex`
- The real scroll node must explicitly use `overflow: auto`
### 6. Cards Must Not Be Compressed to Unreadable
Historical problems have not been "missing scrollbars," but:
- Cards compressed by `flex` to only a tiny visible area
- Text can render but cannot be read completely
- Content exists but is cut off by `overflow: hidden`
Future constraints:
- First ensure cards have a readable minimum height
- If further compression affects readability, switch to internal scrolling
- Do not compress body text, tables, or description areas into unreadable strips just to "maintain one screen"
### 7. Tabs Are Not Inherently Safe Layout Containers
Historical regressions with Tabs include:
- Hidden tab panes reappearing due to custom `display: flex`
- All tabs having the same height/overflow rules forced on them
- Table tabs work, but markdown / help / diagnostics tabs get crushed
Constraints:
- Each type of content inside `Tabs` must define its own layout strategy
- Table tab: "fixed height + internal scrolling"
- Docs/Markdown tab: better as "tab pane self-scrolls + content normal document flow"
- If overriding component library styles, verify the hidden state still holds
### 8. Summary Areas Should Enter Compact Mode First, Not Compress Body
Historical experience shows the top summary cards are most often mishandled:
- They frequently get forcibly narrowed to "fit everything"
- Then the body, tables, and AI result areas all lose their main space
Unified constraint:
- On small screens or high zoom, summary cards should first:
- Reduce padding
- Switch to horizontal scrolling
- Switch to a more compact grid
- Do not sacrifice the main workspace's visible area first
### 9. Long-Document Content Should Prioritize Reading Experience
Content like the following cannot directly apply "table workspace" logic:
- AI briefs
- Runtime logs
- Raw JSON
- Help text
- Multi-paragraph descriptive text
These areas should prioritize:
- Stable title and meta information visibility
- Body has a clear minimum readable height
- Body scroll strategy defined separately
- Support for Markdown tables, dividers, quotes, code blocks
### 10. Height Critical Paths Should Use Fewer Wrapper Layers
Many scroll problems historically were not in the component itself, but came from an extra wrapper layer:
- Height chain broken
- `min-height: 0` not passed down
- `overflow` responsibility absorbed
Therefore:
- For height-critical areas, prefer the most direct DOM structure
- When using `Space`, extra wrapper `div`, or third-party layout containers, verify they don't change scroll and height semantics
- If an area shows "content is there but only a sliver is visible," first suspect an intermediate wrapper layer
## Historical Pitfalls
From Earth, Playground, BGP, DataSources page bugfixes, several high-frequency pitfall types:
### 1. Using `overflow: hidden` to Mask Layout Problems
Superficially the page looks "clean," but actually causes:
- Content getting clipped
- Tab content reduced to a sliver
- Panel renders successfully but users can't see it
Correct approach:
- Let the real content node scroll
- Don't let upper containers unconditionally clip all child content
### 2. Treating All Tabs as the Same Content Type
Tables, Markdown, help cards, and log streams have completely different space requirements.
Correct approach:
- Table: fixed workspace + internal scrolling
- Document: normal flow content + pane-level scrolling
- Side description: content-driven height, not forced to fill
### 3. Only Doing Visual Shrinking, Not Space Reallocation
This causes:
- Card text truncated
- Table shows only 1-2 rows
- Buttons and filters crammed together
Correct approach:
- Compact mode prioritizes re-layout
- Summary area horizontal scrolling
- Collapse / hide secondary modules
### 4. Incomplete Parent Container Height Chain
This is the most common cause of internal scrolling failing.
Inspection order:
1. Does the outer layer actually have a determined height?
2. Does the flex parent have `min-height: 0`?
3. Does the real scroll node explicitly use `overflow: auto`?
4. Have intermediate wrapper layers silently changed layout semantics?
### 5. UI State and Display State Out of Sync
Repeated in Earth-related changes:
- Layer hidden, but hover/lock still active
- Tooltip still showing stale object
- Legend not switching with the state
These constraints also apply to admin pages:
- Hidden, unmounted, or switched-out content should not retain active interaction state
## Recommended Implementation Patterns
### Page Shell
Reuse existing common structures in the project:
- `.dashboard-content-inner`
- `.page-shell`
- `.page-shell__header`
- `.page-shell__body`
- `.table-scroll-region`
Do not invent a completely different height and scroll semantics for each page.
### Table Workspace
Recommended pattern:
```tsx
<Card>
<div className="table-scroll-region" ref={tableRegionRef}>
<Table
pagination={false}
scroll={{ x: 1200, y: tableHeight }}
/>
</div>
</Card>
```
Requirements:
- Tables should scroll inside their card
- `scroll.y` should come from actual available height calculation, not a completely static magic number
- Parent container chain must ensure header, body, content overflow all close inside the table
### Multi-Module Pages
If a page has:
- Summary cards
- Table
- Anomaly details
- Recent events
Do not simply stack all modules vertically. Prefer:
- Top summary + single main workspace at bottom
- Tab-switch multiple secondary data views
- Left-right split with each column scrolling independently
## Discouraged Patterns
The following patterns are considered non-compliant with this project's page standard:
- Relying on full-page vertical scrolling to display the main workspace
- Stacking 3-4 large cards vertically on one page, each wanting to display fully
- Table without internal scrolling, causing only 1-2 rows visible after zoom
- Parent container missing `min-height: 0`, causing internal scrolling to fail
- Only doing visual shrinking without addressing real space allocation
## Page Acceptance Checklist
Before submitting, check at minimum:
- Can page header, summary area, and main workspace appear simultaneously?
- Does the main workspace get the most height on the page?
- When table or detail overflows, does the scrollbar appear inside the module?
- Is the card compressed to the point where text doesn't display completely? If so, has it switched to internal scrolling?
- Is it still usable at browser zoom `125%` / `150%`?
- In a low-height window, is there still a reasonable number of visible content rows?
- Are Tabs, Card, Table still operable when overflowing?
- Do non-table tabs (Markdown, help text, logs) have their own independent and reasonable scroll strategy?
## Implementation Order
When adding or refactoring admin pages, design in this order:
1. Define the main workspace first
2. Determine which modules must always be visible
3. Then handle styling and visual hierarchy
Simply put:
- First ensure correct space allocation
- Then handle scroll boundaries
- Finally handle aesthetics

489
docs/technical/en/manual.md Normal file
View File

@@ -0,0 +1,489 @@
# Planet Manual
This manual is for daily use, demos, development integration, and local operations. It covers four core entry points:
- `planet.sh`: local start, stop, restart, health check, and log access
- Earth: public 3D situational awareness page
- Console: admin backend (login required)
- Docs: public developer documentation and manual
For the shortest path to getting started, see [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md).
## Entry Overview
After a default startup, the common URLs are:
| Name | URL | Login Required | Description |
| --- | --- | --- | --- |
| Earth | `http://localhost:3000/earth` | No | 3D globe, layers, BGP, satellites, cables, news situational awareness |
| Docs | `http://localhost:3000/docs` | No | Developer docs, technical reference, usage manual |
| Console | `http://localhost:3000/admin` | Yes | Data, config, alerts, logs, and situational observation |
| AI Playground | `http://localhost:3000/playground` | Yes | AI Provider status and debugging |
| Backend API Docs | `http://localhost:8000/docs` | Depends on endpoint | FastAPI / OpenAPI documentation |
## planet.sh
`planet.sh` is the main control script for local development and demos. Use it to manage services rather than manually starting frontend, backend, database, and AI Provider separately.
### Start
```bash
./planet.sh start
```
Default behavior:
- Starts PostgreSQL and Redis
- Starts AI Provider
- Starts the backend API
- Starts the frontend Vite dev server
- Outputs Earth, console, Playground, and backend API doc URLs
Specify custom ports:
```bash
./planet.sh start -b 8001 -f 3001 -a 8101
```
Parameters:
| Flag | Meaning |
| --- | --- |
| `-b <port>` | Backend port |
| `-f <port>` | Frontend port |
| `-a <port>` | AI Provider port |
| `--allow-lan` | Enable LAN access |
| `--verbose` | Show more command output during execution |
### Stop
```bash
./planet.sh stop
```
Stops:
- Backend
- AI Provider
- Frontend
- PostgreSQL
- Redis
### Restart
Full restart:
```bash
./planet.sh restart
```
Per-module restart:
```bash
./planet.sh restart -b
./planet.sh restart -f
./planet.sh restart -a
./planet.sh restart -d
```
| Flag | Effect |
| --- | --- |
| `-b` | Backend only |
| `-f` | Frontend only |
| `-a` | AI Provider only |
| `-d` | Database only |
Per-module restarts are preferred during development — they avoid interrupting unrelated services.
### Create User
```bash
./planet.sh createuser
```
Used to create a console login account before first use. The script interactively prompts for username, password, and role.
### Health Check
```bash
./planet.sh health
```
Checks:
- `planet_*` container status
- Backend `/health`
- AI Provider `/health`
- Frontend reachability
If something shows offline, check the corresponding logs first.
### Logs
Recent logs:
```bash
./planet.sh log
```
Follow logs:
```bash
./planet.sh log -f
./planet.sh log -b
./planet.sh log -a
```
| Flag | Log source |
| --- | --- |
| `-f` / `--frontend` | `/tmp/planet_frontend.log` |
| `-b` / `--backend` | `/tmp/planet_backend.log` |
| `-a` / `--ai-provider` | `planet_aiprovider` container logs |
### LAN Access
```bash
./planet.sh start --allow-lan
```
Useful for:
- Starting in WSL, accessing from Windows browser
- Demos on phone or tablet
- Another machine on the same LAN accessing the same dev instance
After starting, check your firewall and WSL network forwarding if access fails.
## Earth
Earth is the public 3D situational awareness page, accessed at:
```text
http://localhost:3000/earth
```
It is a standalone frontend. The actual page lives at:
- `frontend/public/earth/index.html`
- `frontend/public/earth/js/`
- `frontend/public/earth/css/`
The React route `/earth` simply hosts it in an iframe.
### Main Uses
Earth is used to observe in a single globe view:
- BGP events, anomalies, and situational posture
- Satellites and orbital trails
- Submarine cables and landing points
- Compute centers
- Country borders, grid lines, HD texture, cloud layer, terrain
- Live news streams and situational news
- Search and focused object details
### Layer Control
The right-side layer panel toggles visualization layers on or off.
Common layers include:
- Grid lines
- Country borders
- HD texture
- Atmospheric cloud layer
- Submarine cables
- Compute centers
- BGP observation
- Satellites
- Orbital trails
- Terrain
Some layers have dependencies:
- Terrain requires HD texture
- Trails require Satellites
- When HD texture is off, the globe shows the base map and edge glow effect
### Search
Earth search finds current globe objects, such as:
- Submarine cables
- Landing points
- Satellites
- Compute centers
- BGP events
- BGP collectors
Search results can be used to quickly locate objects and open their details.
### Settings
The settings panel contains:
- Rotation mode / cruise mode
- Cruise modules: BGP, News
- Satellite display style: self-glow, real ground footprint
- Day/night mode
- Panel visibility toggles
- Globe default size
- Terrain opacity
- Reset settings
These settings are stored in browser local storage. They revert to defaults if you switch browsers or clear site data.
### Cruise Mode
Cruise mode makes Earth automatically cycle through focus targets.
Current cruise modules:
- BGP
- News
Suitable for demos, monitoring displays, or unattended presentations.
### Mobile
Earth has a mobile drawer layout. On small screens:
- Layer controls open in a mobile drawer
- Search, settings, and details use mobile panels
- Main interactions remain centered on globe object clicks, search, and layer toggles
### Common Issues
#### Earth Won't Open
Check whether the frontend is online:
```bash
./planet.sh health
./planet.sh log -f
```
If the frontend port is not `3000`, use the actual port shown at startup.
#### Layer Has No Data
Check the backend and data sources:
```bash
./planet.sh health
./planet.sh log -b
```
Then open the console and check:
- `/datasources`
- `/data`
- `/bgp`
#### Satellites, BGP, or Cables Load Slowly
These layers may depend on backend APIs, external data sources, or first-run collection tasks. Wait for startup tasks to finish before checking logs and console data source status.
## Console
Console entry point:
```text
http://localhost:3000/admin
```
The console requires login. Create a user first if this is your first time:
```bash
./planet.sh createuser
```
### Page Structure
The console uses React + Ant Design, with a left-side menu organized by work domain.
Common pages:
| Page | Route | Purpose |
| --- | --- | --- |
| Dashboard | `/admin` | System overview |
| Earth | `/earth` | Opens the public Earth page |
| Data Sources | `/datasources` | Manage data sources and trigger collection |
| Collected Data | `/data` | View collected data |
| BGP Observation | `/bgp` | BGP situational data |
| System Alerts | `/alerts/system` | System-level alerts |
| BGP Alerts | `/alerts/bgp` | BGP-related alerts |
| Situational Alerts | `/alerts/situational` | Situational assessment alerts |
| AI Playground | `/playground` | AI Provider debugging |
| System Logs | `/logs` | View system logs (typically super admin only) |
| Users | `/users` | User management |
| Settings | `/settings` | System config and TV live stream sources |
### Data Sources
`/datasources` shows and manages collection sources.
Common operations:
- View data source status
- Trigger collection
- View recent collection tasks
- Adjust configuration
If a category of objects is missing on Earth, start here to confirm the data source is available.
### Collected Data
`/data` shows the collected data table.
Useful for diagnosing:
- Whether data has entered the system
- Whether data update times match expectations
- Whether a data source produced valid records
### BGP Observation
`/bgp` is the BGP-focused page.
It complements the BGP layer on Earth:
- Earth emphasizes spatial posture and visual focus
- The console BGP page emphasizes lists, status, details, and assessment
### Alerts
Alert entry points:
- `/alerts/system`
- `/alerts/bgp`
- `/alerts/situational`
Used to view system, network, and situational alerts.
### System Settings
`/settings` manages system-level configuration.
Current common uses:
- System settings
- TV live stream source configuration
- Data source configuration entry points
Available configuration depends on the current user's role.
### System Logs
`/logs` views system logs. If the menu item is not visible, the current user likely lacks the required role.
Common troubleshooting sequence:
```bash
./planet.sh health
./planet.sh log
```
Then open `/logs` for more structured runtime information.
## Docs
Public documentation site:
```text
http://localhost:3000/docs
```
Current public content comes from:
```text
docs/technical/zh/ (Chinese)
docs/technical/en/ (English)
```
Docs supports:
- Category navigation
- Markdown rendering
- Tables and code blocks
- In-document table of contents
- Local search
- Internal links between technical documents
When adding a new technical document, check:
- Does it have a clear top-level heading
- Does it need to be added to the `/docs` manual category and ordering
- Does it contain information that should not be publicly displayed
## Development Command Conventions
Frontend commands must use Bun:
```bash
cd frontend
bun install
bun run dev
bun run build
```
Do not use `npm run ...`. The project uses Bun in WSL / Windows mixed environments to avoid Node/npm path compatibility issues.
Verify the frontend build:
```bash
source ~/.zshrc && bun run build
```
## Troubleshooting Order
When something goes wrong, follow this sequence:
1. Check service status:
```bash
./planet.sh health
```
2. Check recent logs:
```bash
./planet.sh log
```
3. Check per-module logs:
```bash
./planet.sh log -f
./planet.sh log -b
./planet.sh log -a
```
4. Restart only the affected module:
```bash
./planet.sh restart -f
./planet.sh restart -b
./planet.sh restart -a
```
5. If database or cache is abnormal, restart the database:
```bash
./planet.sh restart -d
```
6. If still unrecovered, do a full restart:
```bash
./planet.sh restart
```
## Related Docs
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md)
- [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/en/frontend-admin-frontend-context.md)
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/en/earth-frontend-context.md)
- [earth-layer-style-reference.md](/home/ray/dev/linkong/planet/docs/technical/en/earth-layer-style-reference.md)
- [backend-system-service-control.md](/home/ray/dev/linkong/planet/docs/technical/en/backend-system-service-control.md)
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/en/backend-collectors.md)

View File

@@ -0,0 +1,105 @@
# Docker + Compose + Buildx Upgrade Guide
Process: remove old version → install new version → verify
---
# 1. Remove Old Version
## Remove apt-installed packages
```bash
sudo apt remove -y docker.io docker-compose docker-compose-v2 docker-doc podman-docker containerd runc
```
---
## Remove system `docker-compose` (V1)
```bash
sudo rm -f "$(which docker-compose 2>/dev/null)"
```
---
## Find and remove manually installed Buildx plugin
```bash
docker info | sed -n '/Plugins:/,/^ Server:/p' | grep -A2 buildx
```
Get the `Path` from the output, then run:
```bash
rm -f <path to docker-buildx file>
```
---
## Clean up unused dependencies
```bash
sudo apt autoremove -y
```
---
# 2. Install Official Docker
Includes Docker Engine, Docker Compose plugin, and Docker Buildx plugin.
## Install dependencies
```bash
sudo apt update
sudo apt install -y ca-certificates curl gnupg
```
---
## Add Docker GPG key
```bash
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
```
---
## Add official repository
```bash
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
```
---
## Install Docker + Compose + Buildx
```bash
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
```
---
# 3. Verify Installation
```bash
docker --version
docker compose version
docker buildx version
```
---
# 4. Common Commands
```bash
docker compose up -d
docker compose down
docker buildx build .
```

View File

@@ -0,0 +1,193 @@
# Quickstart
This guide is for developers or demo operators starting Planet for the first time. The goal is to get services running via the shortest path and know which URLs to open.
## Prerequisites
Recommended: run in a WSL / Linux shell.
You need:
- Docker / Docker Compose available
- `uv` and `bun` accessible in the current shell
- Repository cloned locally
On a new machine, run the bootstrap script first:
```bash
./scripts/bootstrap-dev.sh
```
This script checks and syncs common dependencies, and generates if missing:
- `backend/.env`
- `aiprovider/.env`
- `frontend/.env.local`
## 1. Start Services
From the repository root:
```bash
./planet.sh start
```
After startup, the key URLs are:
| Entry | Default URL | Purpose |
| --- | --- | --- |
| Earth | `http://localhost:3000/earth` | Public 3D Earth visualization |
| Console | `http://localhost:3000/admin` | Admin console (login required) |
| Docs | `http://localhost:3000/docs` | Public developer docs and manual |
| AI Playground | `http://localhost:3000/playground` | AI debugging (login required) |
| Backend API Docs | `http://localhost:8000/docs` | FastAPI / OpenAPI interface docs |
If the default ports are taken, specify custom ports:
```bash
./planet.sh start -f 3001 -b 8001 -a 8101
```
## 2. Create a Login User
The console requires login. For first-time use:
```bash
./planet.sh createuser
```
Follow the prompts to enter username, password, and role.
## 3. Open Earth
Visit:
```text
http://localhost:3000/earth
```
Earth is a public page — no login required.
Once in, verify:
- The globe renders correctly
- The right-side layer panel can toggle layers on/off
- Search can find cables, satellites, compute centers, BGP events
- Settings panel can switch cruise mode, day/night mode, satellite display style
## 4. Open the Console
Visit:
```text
http://localhost:3000/admin
```
The console manages data sources, collected data, situational observation, alerts, system logs, and configuration.
First-time inspection checklist:
- `/datasources`: data source configuration and collection status
- `/data`: collected data
- `/bgp`: BGP situational view
- `/alerts/system`: system alerts
- `/settings`: system configuration
## 5. Check Service Health
```bash
./planet.sh health
```
This shows container status and checks:
- Backend
- AI Provider
- Frontend
## 6. View Logs
Recent logs:
```bash
./planet.sh log
```
Follow a specific service:
```bash
./planet.sh log -f
./planet.sh log -b
./planet.sh log -a
```
Flags:
- `-f`: frontend logs
- `-b`: backend logs
- `-a`: AI Provider logs
## 7. Common Restarts
Frontend only:
```bash
./planet.sh restart -f
```
Backend only:
```bash
./planet.sh restart -b
```
AI Provider only:
```bash
./planet.sh restart -a
```
Database only:
```bash
./planet.sh restart -d
```
Full restart:
```bash
./planet.sh restart
```
## 8. LAN Access
To allow a Windows browser, phone, or another device on the same network:
```bash
./planet.sh start --allow-lan
```
This makes the frontend and backend listen on a LAN-accessible address.
If access fails, check from the shell running Planet:
```bash
curl http://localhost:3000
curl http://localhost:8000/health
ss -ltnp | grep -E ':3000|:8000'
```
## 9. Stop Services
```bash
./planet.sh stop
```
This shuts down the frontend, backend, AI Provider, PostgreSQL, and Redis.
## Next Steps
- Full usage guide: [manual.md](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
- Console structure: [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/en/frontend-admin-frontend-context.md)
- Earth structure: [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/en/earth-frontend-context.md)
- Backend collectors: [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/en/backend-collectors.md)

View File

@@ -1,4 +1,4 @@
# Technical Docs
# 技术文档
这里放“当前实现和当前结构”的文档,重点回答:
@@ -9,16 +9,24 @@
适合放入这里的内容:
- 快速开始和使用手册
- 前端上下文
- Earth 前端结构
- Earth 卫星 footprint 策略
- Earth 卫星覆盖策略
- Earth 渲染图层顺序
- Earth 图层样式属性索引
- 后端运行控制
- collector 现状
- 采集器现状
- 采集格式约定
## 使用入口
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md):从零启动 Planet 的最短路径
- [manual.md](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md):控制台、`planet.sh`、Earth 和 Docs 的完整使用手册
不适合放入这里的内容:
- 尚未完成的 roadmap
- 尚未完成的路线图
- 未来迭代方案
- 大范围重构计划

View File

@@ -0,0 +1,333 @@
# AI Provider 指南
## 概览
`aiprovider` 是 Planet 的模型适配服务。
它把模型厂商差异隔离在主后端之外,让系统其它部分可以调用稳定的业务 API
- 调用方服务 -> `planet backend`
- `planet backend` -> `aiprovider`
- `aiprovider` -> 具体模型提供方
推荐默认方式:
- 外部调用方和跨服务调用方统一调用 `planet backend`
- 只有基础设施级内部任务才直接调用 `aiprovider`
## 职责边界
`backend` 负责:
- 身份认证和权限控制
- 业务层请求整理
- 稳定的 `/api/v1/ai/...` 接口
- 面向 `aiprovider` 的内部服务认证
`aiprovider` 负责:
- 模型协议适配
- 基于 `.env` 选择 provider
- 超时和轻量重试
- 通过 `X-Request-ID` 串联请求追踪
当前配置采用类似 OpenClaw 的拆分方式:
- `AI_PROVIDER` 标识厂商或逻辑 provider
- `AI_PROVIDER_API` 标识实际请求协议适配器
这个拆分能更清楚地表达 MiniMax、Claude 兼容网关、自托管 OpenAI 兼容服务等情况,避免把所有含义塞进一个配置项。
## 支持的 Provider
`aiprovider` 当前支持以下 provider 标识:
- `openai`
- `anthropic`
- `minimax`
- `ollama`
支持的请求适配器:
- `openai-completions`
- `anthropic-messages`
- `ollama-generate`
仍然兼容的历史别名:
- `openai_compatible`
- `anthropic_compatible`
- `claude_compatible`
推荐映射关系:
- `vLLM``LM Studio``One API``AI_PROVIDER=openai``AI_PROVIDER_API=openai-completions`
- `MiniMax``AI_PROVIDER=minimax``AI_PROVIDER_API=anthropic-messages`
- Claude 兼容网关:`AI_PROVIDER=anthropic``AI_PROVIDER_API=anthropic-messages`
- `Ollama``AI_PROVIDER=ollama``AI_PROVIDER_API=ollama-generate`
## API 面
### 主后端 API
推荐使用的稳定入口:
- `GET /api/v1/ai/provider/status`
- `POST /api/v1/ai/situational-awareness/analyze`
认证方式:
- `Authorization: Bearer <jwt>`
可选追踪头:
- `X-Request-ID: <caller-generated-id>`
后端会把 `X-Request-ID` 透传给 `aiprovider`,并在响应中返回同一个 header。
### AI Provider 内部 API
仅供内部调用的接口:
- `GET /v1/provider/status`
- `POST /v1/analyze`
认证方式:
- `X-Provider-Token: <shared-secret>`
可选追踪头:
- `X-Request-ID: <caller-generated-id>`
## 请求示例
### 通过后端调用
```bash
curl -X POST http://localhost:8000/api/v1/ai/situational-awareness/analyze \
-H "Authorization: Bearer <access_token>" \
-H "X-Request-ID: bgp-incident-20260407-001" \
-H "Content-Type: application/json" \
-d '{
"title": "BGP异常研判",
"objective": "总结当前风险并给出处置建议",
"observations": [
"collector A 在 5 分钟内出现多次 origin 变更",
"异常集中在同一地区前缀"
],
"constraints": [
"不要编造不存在的数据",
"区分事实和推断"
],
"context": {
"source": "bgp-monitor",
"severity": "high"
}
}'
```
### 直接调用 `aiprovider`
```bash
curl -X POST http://localhost:8010/v1/analyze \
-H "X-Provider-Token: change_me" \
-H "X-Request-ID: ai-batch-job-001" \
-H "Content-Type: application/json" \
-d '{
"title": "链路波动分析",
"objective": "给出简要态势摘要和下一步建议",
"observations": [
"多个节点出现延迟上升"
],
"constraints": [
"不要假设根因已经确认"
],
"context": {
"region": "APAC"
}
}'
```
## 响应结构
后端和 `aiprovider` 返回相同的 payload 结构:
```json
{
"provider": "minimax",
"api": "anthropic-messages",
"model": "MiniMax-M2.7",
"content": "1) 态势摘要 ...",
"content_blocks": [],
"text_blocks": [],
"thinking_blocks": [],
"raw_response": {}
}
```
两个服务都会返回:
- `X-Request-ID: <id>`
## 配置
### 后端
推荐的后端 `.env`
```env
AI_PROVIDER_SERVICE_URL=http://localhost:8010
AI_PROVIDER_SERVICE_TOKEN=change_me
AI_PROVIDER_TIMEOUT_SECONDS=60
AI_PROVIDER_RETRY_ATTEMPTS=2
```
参考文件:
- [backend/.env.example](/home/ray/dev/linkong/planet/backend/.env.example)
### AI Provider
参考文件:
- [aiprovider/.env.example](/home/ray/dev/linkong/planet/aiprovider/.env.example)
前端本地参考:
- [frontend/.env.example](/home/ray/dev/linkong/planet/frontend/.env.example)
通用配置:
```env
SERVICE_NAME=planet-ai-provider
SERVICE_VERSION=0.1.0
AI_PROVIDER_SERVICE_TOKEN=change_me
AI_TIMEOUT_SECONDS=60
AI_HTTP_RETRY_ATTEMPTS=2
AI_ANALYSIS_SYSTEM_PROMPT=你是态势感知分析助手。请基于输入的上下文、观测与约束,输出结构化、克制、可执行的分析。
```
### OpenAI 兼容示例
```env
AI_PROVIDER=openai
AI_PROVIDER_API=openai-completions
AI_BASE_URL=http://127.0.0.1:8001/v1
AI_API_KEY=local-key
AI_MODEL=your-local-model
```
### MiniMax 中国区示例
```env
AI_PROVIDER=minimax
AI_PROVIDER_API=anthropic-messages
AI_BASE_URL=https://api.minimaxi.com/anthropic
AI_API_KEY=sk-cp-xxxxx
AI_MODEL=MiniMax-M2.7
AI_MAX_TOKENS=1200
AI_ANTHROPIC_VERSION=2023-06-01
```
MiniMax 说明:
- 这里使用官方 MiniMax 示例中的 Anthropic Messages 请求结构。
- 对 MiniMax`aiprovider` 默认不会开启 `thinking`,除非调用方显式传入 `thinking` 对象。
- 这个行为和 OpenClaw 对 MiniMax Anthropic 兼容接口的谨慎处理保持一致。
### Anthropic 兼容示例
```env
AI_PROVIDER=anthropic
AI_PROVIDER_API=anthropic-messages
AI_BASE_URL=https://your-claude-compatible-endpoint.example.com/anthropic
AI_API_KEY=your_api_key
AI_MODEL=your-model
AI_MAX_TOKENS=1200
AI_ANTHROPIC_VERSION=2023-06-01
```
### Ollama 示例
```env
AI_PROVIDER=ollama
AI_PROVIDER_API=ollama-generate
AI_BASE_URL=http://127.0.0.1:11434
AI_API_KEY=
AI_MODEL=qwen2.5:7b
```
## 部署模式
### 单机部署
推荐的本地流程:
- `backend` 运行在 `localhost:8000`
- `aiprovider` 运行在 `localhost:8010`
- 本地模型网关运行在 `localhost:11434` 或其它本地端口
仓库内已包含辅助入口:
- [planet.sh](/home/ray/dev/linkong/planet/planet.sh)
- [docker-compose.local-model.yml](/home/ray/dev/linkong/planet/docker-compose.local-model.yml)
### 多机部署
示例拓扑:
- 应用机器:`backend`
- AI 网关机器:`aiprovider`
- 模型机器:本地模型服务或云代理
此时链路变成服务间 HTTP RPC
- caller -> backend
- backend -> `http://10.0.0.12:8010`
- `aiprovider` -> 模型端点
推荐的跨机器后端配置:
```env
AI_PROVIDER_SERVICE_URL=http://10.0.0.12:8010
AI_PROVIDER_SERVICE_TOKEN=change_me
AI_PROVIDER_TIMEOUT_SECONDS=60
AI_PROVIDER_RETRY_ATTEMPTS=2
```
推荐运行规则:
-`aiprovider` 放在私有网络内
- 至少用 `X-Provider-Token` 保护它
- 始终发送 `X-Request-ID`
- 除基础设施任务外,调用方优先走后端 API
## 重试和失败行为
`backend -> aiprovider`
- 对轻量网络错误和 5xx 失败进行重试
- provider 服务不可用时返回 `502`
`aiprovider -> model provider`
- 对轻量网络错误和 5xx 失败进行重试
- 模型提供方不可用时返回 `502`
这个策略故意保持保守:它能吸收短暂抖动,但不会掩盖持续性错误。
## 运维说明
- `./planet.sh start` 会自动启动 `aiprovider`
- `./planet.sh restart -a` 只重启 `aiprovider`
- `./planet.sh log -a` 跟随查看 `aiprovider` 日志
- `./planet.sh health` 会报告 `aiprovider` 健康状态
## 推荐调用策略
- 前端和应用服务:调用 `backend`
- 定时基础设施任务和诊断任务:可选直接调用 `aiprovider`
- 不要让多个业务服务分别接入模型厂商
这样可以集中管理 provider 切换,避免模型相关差异在系统里四处扩散。

View File

@@ -0,0 +1,100 @@
# DataSources 列表接口性能优化
## 背景
`GET /api/v1/datasources` 是数据源管理页面的核心接口,响应慢会直接阻塞页面渲染。
## 优化前的查询链路
`_load_datasource_list_context` 按顺序执行以下查询:
| 序号 | 函数 | 查询内容 | 瓶颈 |
|------|------|---------|------|
| 1 | `_load_latest_running_tasks` | collection_tasks 窗口函数stale check 依赖此结果 | 必须串行 |
| 2 | `_load_latest_completed_tasks` | collection_tasks 窗口函数(最近完成任务) | 串行等待 |
| 3 | `_load_datasource_data_counts` | `COUNT(*) GROUP BY source` on collected_data | **最慢,全表扫描** |
| 4 | `_load_datasource_endpoint_overrides` | datasource_configs 简单 SELECT | 串行等待 |
## 第一阶段:并行化
将 2/3/4 三个互不依赖的查询改为 `asyncio.gather` + 独立 session 并行执行:
```python
async def _fetch_completed():
async with async_session_factory() as s:
return await _load_latest_completed_tasks(s, datasource_ids)
async def _fetch_counts():
async with async_session_factory() as s:
return await _load_datasource_data_counts(s, sources)
async def _fetch_overrides():
async with async_session_factory() as s:
return await _load_datasource_endpoint_overrides(s, sources)
completed_tasks, data_counts, endpoint_overrides = await asyncio.gather(
_fetch_completed(), _fetch_counts(), _fetch_overrides(),
)
```
> **注意**SQLAlchemy `AsyncSession` 不支持在同一 session 上并发,每个协程必须独立开 session。
## 第二阶段:删除重量级查询
### 删除 `_load_datasource_data_counts`
`data_count` 字段仅用于前端在"最近采集"列显示 `(0条)` 的边缘提示,不值得为此维持一次 `COUNT(*) GROUP BY` 全表扫描。
- 前端同步移除 `(0条)` 显示逻辑
- 移除 `BuiltInDataSource` 接口中的 `data_count` 字段
### 删除 `_load_latest_completed_tasks`
`last_status``last_run_at` 已由 collector 在任务完成时直接更新到 `DataSource` 模型字段,不需要再 JOIN collection_tasks 获取:
```python
# 优化前:需要查 completed_tasks
last_run_at = datasource.last_run_at or (last_task.completed_at if last_task else None)
last_status = datasource.last_status or (last_task.status if last_task else None)
# 优化后:直接读模型字段
last_run_at = datasource.last_run_at
last_status = datasource.last_status
```
同步移除 `last_records_processed` 字段(来源是 completed_tasks列表不显示此字段
## 优化后的查询链路
```
datasources SELECT → 主数据,必须
_load_latest_running_tasks → 必须(进行中状态 + stale check
_load_datasource_endpoint_overrides → 必须endpoint 覆盖,编辑内置 collector 时需要默认值)
```
3 个查询(原来 5 个后两个顺序执行running tasks 先完成用于 stale checkendpoint overrides 轻量)。
## 前端 triggerDatasource 双调修复
`triggerDatasource` 中存在双重 `fetchData()` 调用:
```typescript
// 修复前
} else {
window.setTimeout(() => { fetchData() }, 800) // 无 task_id 时延迟刷
}
fetchData() // 总是立即刷 → 与上面的延迟刷重叠
// 修复后(二者互斥)
if (res.data.task_id) {
setTaskProgress(...)
fetchData() // 有 task_id立即刷一次
} else {
window.setTimeout(fetchData, 800) // 无 task_id等 800ms 再刷一次
}
```
## 相关文件
- `backend/app/api/v1/datasources.py``_load_datasource_list_context``list_datasources`
- `frontend/src/pages/DataSources/DataSources.tsx``BuiltInDataSource` interface、`triggerDatasource`

View File

@@ -0,0 +1,333 @@
# 系统服务控制
本文定义后台控制面动作与现有 `planet.sh` 服务管理命令之间的固定映射。
目标是在复用当前运维脚本语义的同时,不向前端或 API 调用方暴露任意 shell 执行能力。
## 范围
- 这套映射只用于管理端运维控制。
- 控制面必须提交固定 action 名称,而不是原始 shell 命令。
- 后端负责把允许的 action 翻译成固定的 `planet.sh` 调用。
## 设计规则
- 只允许执行白名单 action。
- 前端绝不能发送任意 shell 字符串。
- 后端必须从固定映射表构造命令参数。
- 高风险 action 应限制为 `super_admin`
- 在 UI 连续性重要时,优先局部重启,而不是全栈重启。
## Action 映射
| Action 名称 | 用途 | `planet.sh` 命令 | 备注 |
| --- | --- | --- | --- |
| `restart-backend` | 只重启后端 API | `./planet.sh restart -b` | 推荐作为 UI 触发重启流程的第一阶段实现。 |
| `restart-database` | 重启 PostgreSQL 和 Redis 容器 | `./planet.sh restart -d` | 适合数据库/缓存需要受控重启但不希望重启 UI 的场景。 |
| `restart-system` | 重启整个应用栈 | `./planet.sh restart` | 前端会短暂中断UI 应进入引导恢复模式。 |
| `restart-frontend` | 只重启前端开发服务器 | `./planet.sh restart -f` | 谨慎使用UI 连续性弱于只重启后端。 |
| `restart-backend-port` | 在指定端口重启后端 | `./planet.sh restart -b <port>` | 执行前必须由后端校验端口。 |
| `restart-frontend-port` | 在指定端口重启前端 | `./planet.sh restart -f <port>` | 执行前必须由后端校验端口。 |
| `health-check` | 读取当前服务健康状态 | `./planet.sh health` | 安全的只读运维动作。 |
| `show-logs-backend` | 查看后端日志 | `./planet.sh log -b` | 更适合 CLI/运维工具,不建议作为普通 Web UI 日志流。 |
| `show-logs-frontend` | 查看前端日志 | `./planet.sh log -f` | 更适合 CLI/运维工具,不建议作为普通 Web UI 日志流。 |
## 默认不暴露到 UI 的能力
除非有明确产品需求并经过额外安全评审,否则以下脚本能力不应直接暴露到 Web UI
- `./planet.sh restart`
- `./planet.sh start`
- `./planet.sh stop`
- `./planet.sh createuser`
- 任何未来的原始 shell 透传能力
原因:
- 全量重启可能打断当前控制会话;
- stop/start 影响面更大;
- 用户创建不是服务控制操作;
- 原始 shell 透传会引入不必要的权限风险。
## 第一阶段推荐 UI 契约
### 前端 action payload
```json
{
"action": "restart-backend"
}
```
### 后端命令解析
```text
restart-backend -> ["./planet.sh", "restart", "-b"]
restart-database -> ["./planet.sh", "restart", "-d"]
restart-system -> ["./planet.sh", "restart"]
restart-frontend -> ["./planet.sh", "restart", "-f"]
health-check -> ["./planet.sh", "health"]
```
## API 草案
### 主接口
- `POST /api/v1/system/restart-tasks`
用途:
- 创建受控重启任务;
- 将白名单 action 解析成固定 `planet.sh` 命令;
- 把执行交给外部 runner 或 detached subprocess。
### 请求体
```json
{
"action": "restart-backend"
}
```
未来可选形态:
```json
{
"action": "restart-backend-port",
"port": 8000
}
```
### 响应
```json
{
"task_id": "restart_20260331_153000_ab12cd",
"action": "restart-backend",
"status": "queued",
"stage": "accepted",
"message": "Restart task accepted"
}
```
### 任务查询接口
- `GET /api/v1/system/restart-tasks/{task_id}`
响应结构:
```json
{
"task_id": "restart_20260331_153000_ab12cd",
"action": "restart-backend",
"status": "queued",
"stage": "accepted",
"message": "Waiting for execution",
"requested_by": {
"id": 1,
"username": "admin"
},
"created_at": "2026-03-31T15:30:00+08:00",
"updated_at": "2026-03-31T15:30:02+08:00"
}
```
### 可选日志接口
- `GET /api/v1/system/restart-tasks/{task_id}/logs`
建议响应:
```json
{
"task_id": "restart_20260331_153000_ab12cd",
"lines": [
"accepted restart-backend request",
"spawning restart command",
"waiting for backend shutdown",
"waiting for backend health recovery"
]
}
```
日志接口在第一阶段不是必需项。首版可以只依赖任务状态加 `/health` 轮询。
## 任务状态模型
### Status
- `queued`
- `running`
- `succeeded`
- `failed`
- `timeout`
### Stage
- `accepted`
- `spawning`
- `stopping`
- `starting`
- `waiting_for_health`
- `healthy`
- `failed`
### 含义
- `status` 是高层终态/非终态状态。
- `stage` 是面向运维人员和 UI 的执行阶段。
- `message` 是 modal 或全屏遮罩中展示的短文本。
## 权限模型
- `restart-backend` 应要求 `super_admin`
- 权限检查应沿用 [users.py](/home/ray/dev/linkong/planet/backend/app/api/v1/users.py) 中已有的角色模式。
- 前端可以对非 `super_admin` 隐藏控件,但后端必须继续强制鉴权。
## 存储模型
推荐第一阶段实现:
- 将重启任务状态存入 Redis
- 任务生命周期保持较短;
- 最近日志用有界列表保存。
建议 key
- `system:restart_task:{task_id}`
- `system:restart_task:{task_id}:logs`
建议字段:
- `task_id`
- `action`
- `status`
- `stage`
- `message`
- `requested_by_id`
- `requested_by_username`
- `created_at`
- `updated_at`
## 执行模型
处理请求的 API 进程不应依赖自身持续存活来流式输出完整重启日志。
推荐执行流程:
1. 校验调用方和 action
2. 在 Redis 中创建任务状态
3. 将 action 解析为固定 `planet.sh` argv
4. 启动 detached executor
5. 返回 `task_id`
6. executor 在重启过程中更新任务状态
7. 前端轮询健康状态和/或任务状态,直到服务恢复
推荐命令解析示例:
```text
restart-backend -> ["./planet.sh", "restart", "-b"]
restart-frontend -> ["./planet.sh", "restart", "-f"]
restart-backend-port -> ["./planet.sh", "restart", "-b", "<port>"]
health-check -> ["./planet.sh", "health"]
```
## 前端轮询流程
推荐第一阶段 UX
1. 用户点击 `重启后端`
2. 确认 modal 说明服务会短暂不可用
3. 前端调用 `POST /api/v1/system/restart-tasks`
4. UI 进入阻塞式重启状态
5. 前端每 `1-2s` 轮询 `/health`
6. 临时请求失败视为预期现象
7. 连续 `2-3` 次健康检查成功后,前端刷新页面
可选增强轮询:
1. 后端仍可达时轮询任务状态接口
2. 断连开始后切换为 `/health` 恢复轮询
3. 健康恢复后刷新页面
## 前端状态机
- `idle`
- `confirming`
- `submitting`
- `waiting_for_shutdown`
- `waiting_for_recovery`
- `recovered`
- `failed`
- `timeout`
建议 UI 文案:
- `已发送重启指令`
- `正在停止后端服务`
- `正在等待服务恢复`
- `服务已恢复,正在刷新页面`
- `恢复超时,请手动检查服务状态`
## 第一阶段建议
第一阶段只实现:
- `restart-backend`
- `super_admin` 权限门禁
- 任务创建接口
- Redis 任务状态
- 前端确认 modal
- 前端 `/health` 轮询
- 恢复后自动刷新页面
第一阶段不要实现:
- 完整 `./planet.sh restart`
- 原始 shell 命令透传
- 任意服务控制
- 完整终端 stdout 流式输出
- 多 action 并发重启队列
## 实现清单
### 后端
1.`backend/app/api/v1/` 下新增专用系统控制 API 模块
2. 增加基于白名单的 `planet.sh` action 解析器
3. 将重启任务状态存入 Redis
4. 增加 detached restart-runner 脚本执行
5. 暴露:
- `POST /api/v1/system/restart-tasks`
- `GET /api/v1/system/restart-tasks/{task_id}`
- 可选任务日志接口
6. 对所有 restart-task 接口强制 `super_admin` 权限
### 前端
1. 在 dashboard 为 `super_admin` 增加 `重启后端` 控件
2. 发送前展示确认 modal
3. 提交后将 modal 切换为阻塞式重启状态
4. 轮询 `/health` 直到确认后端恢复
5. 连续健康检查成功后自动刷新页面
6. 展示简短阶段日志,而不是原始终端流
### 运维说明
1. 第一阶段目标应限定为只重启后端
2. 前端重启初期保持在范围外
3. 命令执行必须始终从仓库根目录发起
4. API 边界只能传递固定 action 名称
## 校验要求
- 拒绝任何不在白名单中的 action。
- 如果增加带端口 action端口必须校验为 `1..65535` 的整数。
- 从仓库根目录解析命令,确保 `planet.sh` 的工作目录稳定。
- 记录请求 action、操作者身份、执行开始时间和结果。
## 实现建议
- UI 触发重启流程时,优先实现 `restart-backend`
- 不要依赖当前 API 请求进程在触发自身重启后继续输出完整日志。
- 主 UX 使用任务记录加轮询/健康检查恢复流程,而不是原始终端流。

View File

@@ -0,0 +1,355 @@
# BGP 态势上下文
## 当前目标
BGP 模块正在从一个只展示异常的演示功能,演进为分层观测管线:
`raw observations -> enrichment -> detectors -> incidents -> console/Earth visualization`
实际产品目标已经不只是“在地球上显示事件”。当前目标是:
1. 即使 incident 密度很低,也让 BGP 在 Earth 上保持可见存在感
2. 让 incident 明显比 anomaly 更像高置信度事件层
3. 即使没有活跃 incident也能表达观测网络仍在运行
换句话说Earth 应该表现为观测面,而不只是事件地图:
- `collectors` 表达观测正在发生
- `activity` 表达哪里的路由状态近期活跃或噪声较高
- `incidents` 成为最高置信度的聚焦层
## 当前后端架构
### 数据层
1. `BGPObservation`
- 文件:`backend/app/models/bgp_observation.py`
- 用途:存储从实时/历史来源归一化后的原始路由观测。
- 典型字段:
- `source`
- `collector`
- `peer_asn`
- `peer_ip`
- `prefix`
- `event_type`
- `as_path`
- `origin_asn`
- `next_hop`
- `communities`
- `observed_at`
- `raw_payload`
- `collector_geo`
- `ingest_batch_id`
2. `BGPAnomaly`
- 文件:`backend/app/models/bgp_anomaly.py`
- 用途:保存原子级 detector 输出。
- 当前 detector 输出类型包括:
- `origin_change`
- `more_specific_burst`
- `mass_withdrawal`
3. `BGPIncident`
- 文件:`backend/app/models/bgp_incident.py`
- 用途:把原子 anomaly 聚合成人类和 UI 可消费的 incident 对象。
### 管线
主流程目前集中在:
- `backend/app/services/collectors/bgp_common.py`
- `backend/app/services/bgp_enrichment.py`
- `backend/app/services/bgp_detectors.py`
- `backend/app/services/bgp_incidents.py`
运行流程:
1. 采集器抓取原始 BGP 数据
2. `normalize_bgp_event()` 规范化 payload
3. observation 写入 `bgp_observations`
4. enrichment 为事件补充分析上下文
5. detector 创建 `bgp_anomalies`
6. incident 聚合把 anomaly 汇总为 `bgp_incidents`
### 当前接入来源
1. `RIPE RIS Live`
- 采集器文件:`backend/app/services/collectors/ris_live.py`
- 用于实时观测流。
2. `CAIDA BGPStream Backfill`
- 采集器文件:`backend/app/services/collectors/bgpstream.py`
- 用作历史/回填入口。
## 当前 enrichment 状态
已在以下文件实现 enrichment 骨架:
- `backend/app/services/bgp_enrichment.py`
当前 enrichment 内容:
- prefix family / prefix length
- supernet / more-specific 推导
- 去重 AS path
- path prepending 提示
- collector 区域信息
- prefix baseline 提示
- new-origin 检测
- 可用时从 PeeringDB 获取 ASN 组织画像
- prefix scope / 受影响区域提示
- prefix 地理来源优先级:
- `OpenGeoFeed`override高置信
- `IPtoASN`(国家范围 baseline
- `NRO delegated stats`registry allocation fallback
当前限制:
- `RPKI` 仍只是占位,返回 `unknown`
- 尚未集成真实 ROA 校验来源
- `inetnum` / `inet6num` whois fallback 仍待实现
## 当前 API 面
主 API 文件:
- `backend/app/api/v1/bgp.py`
可用接口:
- `/api/v1/bgp/events`
- `/api/v1/bgp/events/summary`
- `/api/v1/bgp/events/{id}`
- `/api/v1/bgp/anomalies`
- `/api/v1/bgp/anomalies/summary`
- `/api/v1/bgp/anomalies/{id}`
- `/api/v1/bgp/incidents`
- `/api/v1/bgp/incidents/summary`
- `/api/v1/bgp/incidents/{id}`
可视化 GeoJSON 接口:
- `backend/app/api/v1/visualization.py`
- `/api/v1/visualization/geo/bgp-collectors`
- `/api/v1/visualization/geo/bgp-anomalies`
- `/api/v1/visualization/geo/bgp-incidents`
## 当前 Earth 行为
相关文件:
- `frontend/public/earth/js/bgp.js`
- `frontend/public/earth/js/main.js`
- `frontend/public/earth/js/info-card.js`
- `frontend/public/earth/js/constants.js`
- `frontend/public/earth/index.html`
当前设计:
1. BGP 启用时始终显示 collectors。
2. Incident marker 现在是 Earth BGP 的主 marker。
3. 如果没有 incidentEarth 回退显示 anomaly marker。
4. 如果也没有 anomalycollector 仍然提供存在感。
5. 专用 `activity layer` 现在增加:
- 每个 collector 最近 15 分钟活动 halo
- 基于活跃 collector 推导的区域聚合活动提示
6. Incident marker 现在使用:
- 由符号驱动的事件核心
- 向外扩散的环形脉冲
- 相比旧版 Earth 更少的弥散 glow
7. 右侧统计现在显示:
- BGP events
- collector count
- BGP status summary
这个方向是对的,但在低事件密度时期仍不完整。当前 Earth 在 incident 稀疏时仍可能显得过于安静,因为系统还缺少位于原始观测和 incident 聚焦之间的专用 `activity layer`
当前 BGP 状态策略:
- 有 incident显示活跃 incident 数量
- 无 incident 但有 anomaly显示活跃 anomaly 数量,并在可用时显示活跃观测区域
- 无 incident/anomaly 但有 activity显示 `观测网络运行中`
- 无 incident/anomaly 但有 collectors显示 `观测网络运行中 · 当前未发现聚合级事件`
- 完全无 BGP 数据:显示 `暂无观测数据`
Earth info-card 策略:
- `bgp` 卡片文案以 incident 为中心
- `bgp_collector` 卡片显示 collector 位置和当前事件数
## 当前产品缺口
主要缺口不是架构正确性,而是低密度可视化策略。
当前事实:
- incident 数量天然远低于 anomaly 数量
- 这是预期行为,因为 incident 是聚合和去噪后的结果
- 但 incident-first 渲染会让 Earth 显得过于安静,除非有另一层始终可用的 activity layer
推荐 `activity layer` 的实现细节在 [bgp-region-aggregation-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-bgp-region-aggregation-plan.md) 中展开。
因此最近的里程碑是:
`event map -> observability map`
这意味着 Earth 需要三层同时可读:
1. `observation layer`
- collectors
- recent collector activity
- baseline coverage
2. `activity layer`
- recent event density
- anomaly/noise hotspots
- regional activity scoring
- incident presence bonus
3. `incident layer`
- 稀疏但高度清晰的高置信事件对象
- 符号化 marker
- 向外环形脉冲,而不是大面积弥散 glow
## Incident 视觉方向
Earth 的 `incident` 层不应该像一大片发光区域,而应该像紧凑、高置信度的事件焦点。
设计原则:
1. `incident` marker 应使用强主符号
- 符号形状尽量承载类型含义
- 示例:
- `origin_change`:类似三角警告 marker
- `mass_withdrawal`:告警/感叹号风格 marker
- `more_specific_burst`:分裂/放射 marker
2. 强调应来自向外扩散的环形脉冲,而不是区域泛光
- 使用紧凑高亮核心
- 使用一个或多个扩张环形脉冲
- 避免让事件中心变得模糊的大面积亮斑
3. `collector``incident` 必须保持视觉区别
- collector 是观测基础设施
- incident 是抽取后的事件焦点
- collector activity 应比 incident pulse 更安静
4. 平静期仍需要观测存在感
- collectors 和 activity layer 应让地图保持活跃
- 一旦出现 incident它们应明确压过附近 BGP 视觉元素
5. incident 地理位置应转向 `prefix-centric`
- collector 应保持证据来源身份,而不是主要事件位置
- 推荐地理优先级:
- `prefix_geography`
- `prefix_scope`
- `ASN organization region`
- `collector centroid` 作为最终 fallback
- `prefix_scope` 应保持为由观测推导出的范围提示
- 应新增真正面向 prefix 位置的 `prefix_geography`
参考灵感:
- `World Monitor`
- 稀疏事件符号
- 紧凑中心
- 类似环形的向外脉冲
- 比弥散 glow 更强的 incident 可读性
## 当前控制台行为
相关页面:
- `frontend/src/pages/BGP/BGP.tsx`
当前 BGP 控制台页面有三层:
1. 观测摘要
- 总事件数
- collector 数量
- prefix 数量
2. incident 摘要和 incident 表格
3. anomaly 详情表和最近 observation events
这意味着即使 anomaly 为零BGP 页面仍有可用信号。
## 已知产品/工程边界
1. 当前系统仍更接近事件看板,而不是完整 BGP sensing platform。
2. RIS 覆盖范围仍需从较窄订阅范围继续扩展。
3. BGPStream 历史数据仍不是完整 MRT-to-prefix 解码分析。
4. Collector 地理位置仍高度依赖静态 RIPE RIS 映射。
5. Incident 与海缆、IXP、区域之间的关联仍较弱且处于早期阶段。
6. Earth 当前可视化的是逻辑观测/影响结构,而不是真实物理流量路径。
## 测试状态
BGP 专项测试位于:
- `backend/tests/test_bgp.py`
当前已验证状态:
- `backend/tests/test_bgp.py``25 passed`
- `backend/tests``62 passed`
覆盖范围包括:
- normalization
- observation serialization
- enrichment
- detectors包括 route leak candidate 和 path flap
- incident aggregation
- batch anomaly creation
- BGP events/incidents API
- summary endpoints
## 最相关文件
后端:
- `backend/app/models/bgp_observation.py`
- `backend/app/models/bgp_anomaly.py`
- `backend/app/models/bgp_incident.py`
- `backend/app/services/collectors/bgp_common.py`
- `backend/app/services/bgp_enrichment.py`
- `backend/app/services/bgp_detectors.py`
- `backend/app/services/bgp_incidents.py`
- `backend/app/api/v1/bgp.py`
- `backend/app/api/v1/visualization.py`
前端:
- `frontend/src/pages/BGP/BGP.tsx`
- `frontend/public/earth/js/bgp.js`
- `frontend/public/earth/js/main.js`
- `frontend/public/earth/js/info-card.js`
- `frontend/public/earth/js/constants.js`
- `frontend/public/earth/index.html`
## 推荐下一步
### 后端 / 检测优先级
1. 集成真实 RPKI 校验数据。
2. 扩展实时 collector 覆盖范围,并更广泛纳入 withdrawals。
3. 用更强启发式继续完善 route leak 和 path instability detector。
### 关联 / 叙事优先级
4. 强化 incident 聚合语义和标题。
5. 增加 incident 与以下对象的弱关联:
- 海缆走廊
- 登陆点
- IXPs
- 其它流量异常来源
6. 优化 Earth 中 collector 和 incident 之间的 hover/click 交接。
### 可视化优先级
7. 调整区域 activity scoring让 activity layer 有信息量但不嘈杂。
8. 随着新 detector 落地,增加更多 incident 符号类型。
9. 增加真实 prefix geography 来源:
- `IPtoASN / IPtoCountry` 作为第一阶段可用数据集
- `OpenGeoFeed` 作为更高质量 override 层
- registry/whois 只作为 fallback

View File

@@ -1,11 +1,11 @@
# Earth Frontend Context
# Earth 前端结构
本文件描述当前 Earth 大屏前端的真实结构,重点是帮助后续继续改 HUD、图层、媒体面板、真实地形、BGP 可视化时,不再重复踩结构和状态同步上的坑。
相关规则建议一起参考:
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)
## 当前目标
@@ -378,4 +378,4 @@ Earth 前端和控制台前端不是同一套 UI 系统:
控制台相关结构见:
- [admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/frontend-admin-frontend-context.md)
- [admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)

View File

@@ -0,0 +1,244 @@
# Earth 图层样式属性索引
本文记录当前 Earth 前端各图层的材质、颜色、透明度、线宽、半径偏移和
`renderOrder` 等样式属性。层级关系请配合
[earth-render-layer-order.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-render-layer-order.md)
查看。
## 命名约定
| 类别 | 约定 | 示例 |
| --- | --- | --- |
| 全局配置对象 | `*_CONFIG` | `COUNTRY_BOUNDARY_CONFIG` |
| 图层半径偏移 | `*AltitudeOffset` / `radiusOffset` | `lineAltitudeOffset`, `GRID_CONFIG.radiusOffset` |
| 透明度 | `*Opacity` | `hoverLineOpacity` |
| 渲染顺序 | `*RenderOrder` | `textureOverlayRenderOrder` |
| 颜色 | `*Color`,十六进制数字或 CSS 色值 | `lineColor`, `colors.supercomputer` |
| 线宽 | `lineWidth` / `*LineWidth` | `GRID_CONFIG.lineWidth` |
## Earth 基座与高清材质
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| Earth 基座半径 | `CONFIG.earthRadius` | `100` | `earth.js:createEarth()` |
| Earth 基座颜色 | `EARTH_MATERIAL_CONFIG.color` | `0x010609` | `MeshPhongMaterial.color` |
| Earth 基座 emissive | `EARTH_MATERIAL_CONFIG.emissive` | `0x010609` | `MeshPhongMaterial.emissive` |
| Earth 基座 specular | `EARTH_MATERIAL_CONFIG.specular` | `0x1a2d45` | `MeshPhongMaterial.specular` |
| Earth 基座 shininess | `EARTH_MATERIAL_CONFIG.shininess` | `12` | `MeshPhongMaterial.shininess` |
| Earth 基座 opacity | `EARTH_MATERIAL_CONFIG.opacity` | `1` | `MeshPhongMaterial.opacity` |
| 高清材质半径偏移 | `EARTH_MATERIAL_CONFIG.textureOverlayAltitudeOffset` | `0.1` | 独立高清材质球半径 |
| 高清材质透明度 | `EARTH_MATERIAL_CONFIG.textureOverlayOpacity` | `0.88` | 高清材质 `MeshPhongMaterial.opacity` |
| 高清材质 renderOrder | `EARTH_MATERIAL_CONFIG.textureOverlayRenderOrder` | `0.96` | `_earthTextureOverlay.renderOrder` |
| 高清材质 specular | `EARTH_MATERIAL_CONFIG.textureOverlaySpecular` | `0x05080d` | 降低直射区域镜面高光,避免贴图死白 |
| 高清材质 shininess | `EARTH_MATERIAL_CONFIG.textureOverlayShininess` | `4` | 降低高光集中度 |
| 高清材质颜色乘色 | inline | `0xffffff` | `_earthTextureOverlayMaterial.color` |
## Earth 遮挡与昼夜
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 遮挡球半径系数 | `EARTH_MATERIAL_CONFIG.occluderRadiusFactor` | `0.999` | 深度遮挡球半径 |
| 遮挡球分段 | `EARTH_MATERIAL_CONFIG.occluderSegments` | `48` | 遮挡球几何分段 |
| 遮挡球 renderOrder | inline | `-1` | `occluder.renderOrder` |
| 昼夜太阳方向 | `EARTH_MATERIAL_CONFIG.dayNight.sunDirection` | `{ x: 1, y: 0.2, z: 0.4 }` | 自定义 day/night shader |
| 夜侧最低亮度 | `EARTH_MATERIAL_CONFIG.dayNight.nightFloor` | `0.24` | shader uniform |
| 日侧增强 | `EARTH_MATERIAL_CONFIG.dayNight.dayBoost` | `1.12` | shader uniform |
| 暮光宽度 | `EARTH_MATERIAL_CONFIG.dayNight.twilightWidth` | `0.2` | shader uniform |
| 暮光强度 | `EARTH_MATERIAL_CONFIG.dayNight.twilightIntensity` | `0.14` | shader uniform |
| 暮光颜色 | `EARTH_MATERIAL_CONFIG.dayNight.twilightColor` | `0x4ea0ff` | shader uniform |
| 夜侧 tint 颜色 | `EARTH_MATERIAL_CONFIG.dayNight.nightTintColor` | `0x0b1830` | shader uniform |
| 夜侧 tint 强度 | `EARTH_MATERIAL_CONFIG.dayNight.nightTintIntensity` | `0.08` | shader uniform |
## 大气辉光与云图
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 内层大气半径系数 | `EARTH_MATERIAL_CONFIG.atmosInnerRadiusFactor` | `1.01` | `atmosInnerGeo` |
| 内层大气分段 | `EARTH_MATERIAL_CONFIG.atmosInnerSegments` | `64` | `atmosInnerGeo` |
| 内层大气颜色 | `EARTH_MATERIAL_CONFIG.atmosInnerColor` | `[0.25, 0.62, 1.0]` | shader RGB |
| 内层大气 rim power | `EARTH_MATERIAL_CONFIG.atmosInnerRimPower` | `3.2` | shader rim |
| 内层大气强度 | `EARTH_MATERIAL_CONFIG.atmosInnerIntensity` | `0.18` | shader alpha multiplier |
| 外层大气半径系数 | `EARTH_MATERIAL_CONFIG.atmosOuterRadiusFactor` | `1.0025` | `atmosOuterGeo` |
| 外层大气分段 | `EARTH_MATERIAL_CONFIG.atmosOuterSegments` | `48` | `atmosOuterGeo` |
| 外层大气颜色 | `EARTH_MATERIAL_CONFIG.atmosOuterColor` | `[0.18, 0.45, 0.9]` | shader RGB |
| 外层大气 rim power | `EARTH_MATERIAL_CONFIG.atmosOuterRimPower` | `9.0` | shader rim |
| 外层大气强度 | `EARTH_MATERIAL_CONFIG.atmosOuterIntensity` | `0.0025` | shader alpha multiplier |
| 大气辉光 blending | inline | `THREE.AdditiveBlending` | `ShaderMaterial.blending` |
| 大气辉光 renderOrder | inline | `1` | `atmosInner/Outer.renderOrder` |
| 无高清材质边缘光颜色 | `EARTH_MATERIAL_CONFIG.rimGlowColor` | `[0.42, 0.72, 1.0]` | 高清材质隐藏或不可用时的 Fresnel shell RGB |
| 无高清材质边缘光半径系数 | `EARTH_MATERIAL_CONFIG.rimGlowRadiusFactor` | `1.0035` | `earth-rim-glow` 外扩球壳半径 |
| 无高清材质边缘光 rim power | `EARTH_MATERIAL_CONFIG.rimGlowPower` | `3.4` | shader rim 衰减;值越大边缘越窄 |
| 无高清材质边缘光强度 | `EARTH_MATERIAL_CONFIG.rimGlowIntensity` | `0.24` | shader alpha multiplier |
| 无高清材质边缘光分段 | `EARTH_MATERIAL_CONFIG.rimGlowSegments` | `96` | `earth-rim-glow` 几何分段 |
| 无高清材质边缘光 renderOrder | `EARTH_MATERIAL_CONFIG.rimGlowRenderOrder` | `1.08` | `_earthRimGlow.renderOrder` |
| 无高清材质边缘光 depthTest | inline | `false` | 避免被海陆基座或地表填充遮住 |
| 云图半径偏移 | `CLOUD_LAYER_CONFIG.radiusOffset` | `3` | 云层球半径 |
| 云图分段 | `CLOUD_LAYER_CONFIG.widthSegments / heightSegments` | `64 / 64` | 云层球几何分段 |
| 云图透明度 | `CLOUD_LAYER_CONFIG.opacity` | `0.15` | `MeshPhongMaterial.opacity` |
| 云图贴图 | `CLOUD_LAYER_CONFIG.textureUrl` | `"./assets/earth_clouds_1024.png"` | 云层贴图 |
| 云图 blending | inline | `THREE.AdditiveBlending` | `MeshPhongMaterial.blending` |
## 海陆基座与国界
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 国界数据路径 | `COUNTRY_BOUNDARY_CONFIG.dataPath` | `"/earth/data/countries-admin0.min.geojson"` | GeoJSON 输入 |
| 海洋填充色 | local `OCEAN_HEX` | `0x010609` | 海陆基座 canvas 背景 |
| 陆地填充色 | `COUNTRY_BOUNDARY_CONFIG.landColor` | `0x080f1b` | 海陆基座 canvas 陆地 |
| 海陆基座透明度 | `COUNTRY_BOUNDARY_CONFIG.landOpacity` | `1.0` | `MeshBasicMaterial.opacity` |
| 海陆基座半径偏移 | `COUNTRY_BOUNDARY_CONFIG.landAltitudeOffset` | `0.08` | `country-land-ocean` 半径 |
| 海陆基座 renderOrder | `COUNTRY_BOUNDARY_CONFIG.landRenderOrder` | `0.86` | `country-land-ocean.renderOrder` |
| 海陆 mask 尺寸 | `landMaskWidth / landMaskHeight` | `2048 / 1024` | canvas / DataTexture 尺寸 |
| 国界 tint 颜色 | `COUNTRY_BOUNDARY_CONFIG.tintColor` | `0x0b1830` | 高清材质关闭时 tint |
| 国界 tint 半径偏移 | `COUNTRY_BOUNDARY_CONFIG.tintAltitudeOffset` | `0.04` | `country-tint` 半径 |
| 国界 tint renderOrder | `COUNTRY_BOUNDARY_CONFIG.tintRenderOrder` | `0.2` | `country-tint.renderOrder` |
| 国界线颜色 | `COUNTRY_BOUNDARY_CONFIG.lineColor` | `0x7fc7ff` | 普通国界线 |
| 国界线透明度 | `COUNTRY_BOUNDARY_CONFIG.lineOpacity` | `0.58` | 普通国界线 opacity |
| 国界线 hover 时压暗透明度 | `COUNTRY_BOUNDARY_CONFIG.dimmedLineOpacity` | `0.18` | hover 时普通国界线 opacity |
| 国界线半径偏移 | `COUNTRY_BOUNDARY_CONFIG.lineAltitudeOffset` | `0.24` | 普通国界线半径 |
| 国界线 renderOrder | `COUNTRY_BOUNDARY_CONFIG.lineRenderOrder` | `2.2` | 普通国界线层级 |
| 国界 hover 颜色 | `COUNTRY_BOUNDARY_CONFIG.hoverLineColor` | `0xff3b1f` | 霓虹红橘 |
| 国界 hover 透明度 | `COUNTRY_BOUNDARY_CONFIG.hoverLineOpacity` | `1.0` | hover 实线 opacity |
| 国界 hover 半径偏移 | `COUNTRY_BOUNDARY_CONFIG.hoverAltitudeOffset` | `0.32` | hover 实线半径 |
| 国界 hover renderOrder | `COUNTRY_BOUNDARY_CONFIG.hoverLineRenderOrder` | `2.3` | hover 实线层级 |
| 国界 hover glow 透明度 | `COUNTRY_BOUNDARY_CONFIG.hoverGlowOpacity` | `0.38` | glow 线 opacity |
| 国界 hover glow 线宽 | `COUNTRY_BOUNDARY_CONFIG.hoverGlowLineWidth` | `3` | glow `LineBasicMaterial.linewidth` |
| 国界 hover glow 层级偏移 | `COUNTRY_BOUNDARY_CONFIG.hoverGlowRenderOrderOffset` | `0.01` | glow renderOrder = `2.29` |
| 国界 hover glow 半径偏移 | `COUNTRY_BOUNDARY_CONFIG.hoverGlowRadiusOffset` | `0.04` | glow 半径 = hover 半径 + 0.04 |
## 真实地形
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 地形 tile size | `TERRAIN_CONFIG.tileSize` | `256` | Terrarium tile 读取 |
| 地形 base zoom | `TERRAIN_CONFIG.baseZoom` | `4` | 地形采样 zoom |
| 地形几何分段 | `geometryWidthSegments / geometryHeightSegments` | `320 / 320` | 地形球几何 |
| 地形基准半径偏移 | `TERRAIN_CONFIG.baseRadiusOffset` | `0.16` | 地形压过高清材质 |
| 地形夸张系数 | `TERRAIN_CONFIG.exaggeration` | `34` | 海拔转世界单位 |
| 地形陆地淡入高度 | `TERRAIN_CONFIG.landRevealFadeMeters` | `220` | 顶点 alpha |
| 地形透明度 | `TERRAIN_CONFIG.opacity` | `0.68` | `MeshPhongMaterial.opacity` |
| 地形颜色 | `TERRAIN_CONFIG.color` | `0x8aa884` | `MeshPhongMaterial.color` |
| 地形 emissive | `TERRAIN_CONFIG.emissive` | `0x030704` | 降低自发光,恢复地形明暗层次 |
| 地形 specular | `TERRAIN_CONFIG.specular` | `0x344438` | 给地形局部光泽,不抬高高清贴图直射亮度 |
| 地形 shininess | `TERRAIN_CONFIG.shininess` | `16` | 收紧地形高光,增强起伏辨识 |
| 地形 renderOrder | inline | `1.2` | `terrain.renderOrder` |
| 地形 polygonOffset | inline | `factor -1`, `units -1` | 降低贴近球面时的闪烁 |
## 经纬线
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 经纬线半径偏移 | `GRID_CONFIG.radiusOffset` | `0.14` | 经纬线球面半径 |
| 经纬线颜色 | `GRID_CONFIG.color` | `0xc0e0ff` | `LineBasicMaterial.color` |
| 经纬线透明度 | `GRID_CONFIG.opacity` | `0.08` | `LineBasicMaterial.opacity` |
| 经纬线线宽 | `GRID_CONFIG.lineWidth` | `1` | `LineBasicMaterial.linewidth` |
| 经纬线 renderOrder | `GRID_CONFIG.renderOrder` | `2.05` | 经纬线层级 |
| 纬线间隔 | `GRID_CONFIG.latitudeStep` | `15` | 纬线生成步长 |
| 经线间隔 | `GRID_CONFIG.longitudeStep` | `30` | 经线生成步长 |
| 线段采样步长 | `GRID_CONFIG.segmentStep` | `5` | 经纬线采样步长 |
## 海缆与登陆点
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 默认海缆颜色 | `CABLE_COLORS.default` | `0xffff44` | 无数据颜色时使用 |
| 海缆半径偏移 | `CABLE_CONFIG.line.altitudeOffset` | `0.2` | 海缆线半径 |
| 海缆线宽 | `CABLE_CONFIG.line.lineWidth` | `1` | `LineBasicMaterial.linewidth` |
| 海缆透明度 | `CABLE_CONFIG.line.opacity` | `1.0` | 海缆线 opacity |
| 海缆 renderOrder | `CABLE_CONFIG.line.renderOrder` | `1` | 海缆线层级 |
| 登陆点半径偏移 | `CABLE_CONFIG.landingPoint.altitudeOffset` | `0.48` | 对齐算力中心贴地表 marker 高度 |
| 登陆点 icon 贴图尺寸 | `CABLE_CONFIG.landingPoint.textureSize` | `256` | canvas 渲染 EPS 参考图的实心 map-pin中间圆孔透明镂空 |
| 登陆点 icon 宽高比 | `CABLE_CONFIG.landingPoint.iconAspectRatio` | `0.82` | `Sprite.scale.x = height * aspect` |
| 登陆点 icon 锚点 | `CABLE_CONFIG.landingPoint.anchorX / anchorY` | `0.52 / 0.276` | `Sprite.center`,将 pin 下端点对齐登陆点经纬度 |
| 登陆点基础缩放 | `CABLE_CONFIG.landingPoint.baseScale` | `12` | 对齐算力中心等地表 icon 的 sprite 高度 |
| 登陆点颜色 | `CABLE_CONFIG.landingPoint.color` | `0xffaa00` | `SpriteMaterial.color` |
| 登陆点 emissive | `CABLE_CONFIG.landingPoint.emissive` | `0x442200` | 兼容旧球体材质sprite 不使用 |
| 登陆点 emissive 强度 | `CABLE_CONFIG.landingPoint.emissiveIntensity` | `0.5` | 兼容旧球体材质sprite 不使用 |
| 登陆点透明度 | `CABLE_CONFIG.landingPoint.opacity` | `1.0` | `SpriteMaterial.opacity` |
| 登陆点 renderOrder | `CABLE_CONFIG.landingPoint.renderOrder` | `4.5` | 对齐算力中心地表设施层级 |
| 登陆点 dim 亮度系数 | `landingPointVisual.dimBrightness` | `0.62` | dim 状态颜色乘数 |
| 相关登陆点高亮 opacity | `landingPointVisual.related.opacityBase / opacityPulse` | `0.8 / 0.2` | 高亮脉冲 |
| 非相关登陆点颜色 | `landingPointVisual.dimmed.colorRGB` | `{ r: 180, g: 116, b: 28 }` | dim 状态颜色,避免黑色基座透出成暗洞 |
| 非相关登陆点 emissive | `landingPointVisual.dimmed.emissive` | `0x3a2200` | dim 状态弱琥珀自发光 |
| 非相关登陆点 emissive 强度 | `landingPointVisual.dimmed.emissiveIntensity` | `0.18` | dim 状态弱发光强度 |
| 非相关登陆点 opacity | `landingPointVisual.dimmed.opacity` | `0.78` | dim 状态透明度,不再用低 alpha 混黑底 |
## 卫星、轨迹和 footprint
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 卫星显示半径偏移 | `SATELLITE_CONFIG.displayAltitudeOffset` | `8` | 卫星点位置 |
| 卫星点基础像素大小 | `SATELLITE_CONFIG.dotBaseSize` | `2.8` | 点 shader size |
| 卫星背景点缩放 | `SATELLITE_CONFIG.dotBackdropScale` | `1.28` | 背景点大小 |
| 卫星点透明度范围 | `dotOpacityMin / dotOpacityMax` | `0.7 / 1.0` | 呼吸动画 |
| 卫星点呼吸速度 | `SATELLITE_CONFIG.dotBreathingSpeed` | `0.12` | 点 opacity 动画 |
| 卫星背景点颜色 | inline | `0x0b1626` | backdrop point baseColor |
| 卫星背景点透明度 | inline | `0.42` | backdrop point opacity |
| 卫星点透明度 | inline | `0.9` | point material opacity |
| 卫星背景点 renderOrder | inline | `5` | `satelliteBackdropPoints.renderOrder` |
| 卫星点 renderOrder | inline | `6` | `satellitePoints.renderOrder` |
| 卫星轨迹长度 | `SATELLITE_CONFIG.trailLength` | `10` | trail buffer |
| 卫星轨迹线宽 | `SATELLITE_CONFIG.trailLineWidth` | `3` | ribbon shader uniform |
| 选中 ring 大小 | `SATELLITE_CONFIG.ringSize` | `0.07` | hover / locked ring sprite |
| 卫星覆盖层 renderOrder | `SATELLITE_CONFIG.overlayRenderOrder` | `12` | locked ring / halo / orbit |
| 自发光选中点颜色 | inline default | `"#ffd25a"` | `showSelfGlowStyle()` |
| 自发光选中点透明度 | inline | `0.96` | locked dot material |
| footprint renderOrder | local `GROUND_FOOTPRINT_RENDER_ORDER` | `3` | footprint fill |
| footprint group renderOrder | inline | `0` | 避免 Group 排序盖过卫星点 |
## 算力中心
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 算力中心半径偏移 | `COMPUTE_CENTER_CONFIG.altitudeOffset` | `0.48` | marker 位置 |
| 算力中心基础透明度 | `COMPUTE_CENTER_CONFIG.marker.baseOpacity` | `0.88` | `SpriteMaterial.opacity` |
| 超算 marker 缩放 | `COMPUTE_CENTER_CONFIG.marker.supercomputerScale` | `12` | 超算 marker |
| GPU 集群 marker 缩放 | `COMPUTE_CENTER_CONFIG.marker.gpuClusterScale` | `12` | GPU marker |
| hover 缩放 | `COMPUTE_CENTER_CONFIG.marker.hoverScale` | `1.16` | hover 状态 |
| locked 缩放 | `COMPUTE_CENTER_CONFIG.marker.lockedScale` | `1.22` | locked 状态 |
| dimmed 缩放 / 透明度 | `dimmedScale / dimmedOpacity` | `0.82 / 0.34` | dim 状态 |
| 超算颜色 | `COMPUTE_CENTER_CONFIG.colors.supercomputer` | `"#38bdf8"` | marker texture |
| GPU 集群颜色 | `COMPUTE_CENTER_CONFIG.colors.gpu_cluster` | `"#2dd4bf"` | marker texture |
| 关联颜色 | `COMPUTE_CENTER_CONFIG.colors.linked` | `"#f8fafc"` | 关联态 |
| 算力中心 renderOrder | local `COMPUTE_CENTER_RENDER_ORDER` | `4.5` | 地表设施低于卫星点 |
## BGP 观测
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| BGP 事件半径偏移 | `BGP_CONFIG.altitudeOffset` | `2.1` | anomaly marker |
| BGP collector 半径偏移 | `BGP_CONFIG.collectorAltitudeOffset` | `1.6` | collector marker |
| 事件基础缩放 | `BGP_CONFIG.marker.eventBaseScale` | `6.2` | anomaly sprite |
| collector 基础缩放 | `BGP_CONFIG.marker.collectorBaseScale` | `7.4` | collector plane |
| hover / dim 缩放 | `hoverScale / dimmedScale` | `1.16 / 0.92` | 交互状态 |
| 普通事件透明度 | `BGP_CONFIG.opacity.normal` | `0.78` | anomaly sprite |
| hover 透明度 | `BGP_CONFIG.opacity.hover` | `1.0` | hover 状态 |
| dimmed 透明度 | `BGP_CONFIG.opacity.dimmed` | `0.24` | dim 状态 |
| collector 透明度 | `BGP_CONFIG.opacity.collector` | `0.62` | collector 状态 |
| critical 颜色 | `BGP_CONFIG.severityColors.critical` | `0xff4d4f` | 严重事件 |
| high 颜色 | `BGP_CONFIG.severityColors.high` | `0xff9f43` | 高危事件 |
| medium 颜色 | `BGP_CONFIG.severityColors.medium` | `0xffd166` | 中危事件 |
| low 颜色 | `BGP_CONFIG.severityColors.low` | `0x4dabf7` | 低危事件 |
| collector 基础色 | `BGP_CONFIG.collectorColor` | `0x6db7ff` | collector 默认色 |
| region 色 | `BGP_CONFIG.regionColor` | `0x2dd4bf` | 区域覆盖 |
| BGP ring 缩放 | `BGP_CONFIG.ring.scaleA / scaleB` | `2.5 / 3.4` | anomaly ring |
| BGP ring 透明度 | `BGP_CONFIG.ring.opacity` | `0.5` | anomaly ring |
| collector marker renderOrder | inline | `3` | `marker.renderOrder` |
| anomaly marker renderOrder | inline | `5` normal, `7` active | `marker.renderOrder` |
## 天体与星空
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 天球半径 | `CELESTIAL_CONFIG.skyRadius` | `2600` | 天体背景 |
| 天球透明度 | `CELESTIAL_CONFIG.skyOpacity` | `1` | 背景材质 |
| 太阳距离 / 缩放 | `sunDistance / sunScale` | `2150 / 78` | 太阳 sprite |
| 月亮距离 / 缩放 | `moonDistance / moonScale` | `2050 / 38` | 月亮 sprite |
| 太阳 halo 缩放 | `CELESTIAL_CONFIG.sunHaloScale` | `136` | 太阳 halo |
| 月亮 halo 缩放 | `CELESTIAL_CONFIG.moonHaloScale` | `62` | 月亮 halo |
| 太阳光颜色 / 强度 | `sunLightColor / sunLightIntensity` | `0xfff4df / 1.02` | scene light |
| 背光颜色 / 强度 | `backLightColor / backLightIntensity` | `0x2b4c78 / 0.3` | scene light |
| 星空点数量 | `STARFIELD_CONFIG.count` | `8000` | `createStars()` |
| 星空半径范围 | `minRadius + radiusJitter` | `800 + 200` | 随机分布 |
| 星空点颜色 | `STARFIELD_CONFIG.color` | `0xffffff` | `PointsMaterial.color` |
| 星空点大小 | `STARFIELD_CONFIG.size` | `0.5` | `PointsMaterial.size` |

View File

@@ -1,4 +1,4 @@
# News Live Streams Collector Format
# 新闻直播采集格式
`news_live_streams` 采集器面向“频道目录 JSON”输入而不是直接抓网页。

View File

@@ -0,0 +1,57 @@
# Earth 渲染图层顺序
本文记录当前 Earth 渲染器的图层顺序和每层意图。后续调整
`renderOrder`、半径偏移、深度策略或指针交互时,需要同步更新这里。
注意:图层控制面板顺序和注册 / 启动加载顺序是两套语义。
| 顺序类型 | 当前顺序 | 说明 |
| --- | --- | --- |
| 控制面板顺序 | 海缆 → 轨迹 → 卫星 → 算力中心 → BGP → 地形 → 高清材质 → 大气云图 → 国界 → 经纬线 | 由 `displayOrder` 控制,按操作关注度排列。 |
| 注册 / 启动加载顺序 | 经纬线 → 国界 → 高清材质 → 大气云图 → 海缆 → 算力中心 → BGP → 卫星 | 由注册顺序和 `startupPriority` 控制,按地表到天空排列;轨迹和地形是依赖/可选显示层,不参与常规启动数据加载。 |
## 地表图层栈
| 顺序 | 图层 | 来源 | 渲染 / 半径策略 | 深度 / 交互策略 | 备注 |
| --- | --- | --- | --- | --- | --- |
| -1000 | 天体背景 mesh | `celestial.js` | 背景球 | 不参与地表拾取 | 位于所有 Earth 内容之后。 |
| -1 | Earth 遮挡球 | `earth.js` | 地球内侧不可见球 | 写入深度 | 遮挡地球背面的对象。 |
| 0 | Earth 基座球 | `earth.js` | `CONFIG.earthRadius` | 地表拾取兜底目标 | 深色基座,所有可选地图层关闭时仍可见。 |
| 0.2 | 国界暗色 tint | `country-boundaries.js` | `tintAltitudeOffset` | 禁用 raycast | 高清材质关闭时使用。 |
| 0.86 | 海陆基座填充 | `country-boundaries.js` | `landAltitudeOffset`; 海洋 `#010609`,陆地 `#080f1b` | 禁用 raycast | 即使国界线关闭,基座地图仍保持可用。 |
| 0.96 | 高清 Earth 材质 | `earth.js` | `textureOverlayAltitudeOffset` | 可见时作为地表拾取目标 | 高清材质始终压过海陆基座填充。 |
| 1 | 大气辉光和云图 | `earth.js` | 大气 / 云层球 | 不走普通对象选择路径 | 云图由“大气云图”图层开关控制。 |
| 1 | 海缆 | `cables.js` | `CABLE_CONFIG.line.renderOrder` | 海缆拾取路径 | 保持现有海缆层级。 |
| 1.2 | 真实地形 | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` 加地形位移 | 禁用 raycast | 地形压过高清材质;高清材质关闭时临时隐藏,重新开启后恢复原状态。 |
| 2.05 | 经纬线 | `earth.js` | `CONFIG.earthRadius + 0.14` | 禁用 raycast | 低透明度显示在高清材质上。 |
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset` | 禁用 raycast | 只保证压过高清材质。 |
| 2.29 | 国界 hover 光晕 | `country-boundaries.js` | hover 半径加 glow 偏移 | `depthTest: false`,禁用 raycast | 用 additive 光晕增强交界边和地形开启时的 hover 可见性。 |
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;中国和中国(台湾)共享高亮组。 |
| 3 | 卫星 footprint 填充 | `satellites.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-testedGroup renderOrder 保持 0 | Footprint 在国界线之上,但在算力中心和卫星之下。 |
| 3-5 | BGP 标记和覆盖层 | `bgp.js` | 各 marker 自身 renderOrder | BGP 拾取路径 | 保持现有 BGP 视觉层级。 |
| 4.5 | 算力中心 | `compute-centers.js` | `COMPUTE_CENTER_RENDER_ORDER` | 算力中心拾取路径 | 地表设施,保持在卫星下方。 |
| 5 | 卫星背景点 | `satellites.js` | 固定 renderOrder | 屏幕空间卫星拾取 | 位于卫星点下方。 |
| 6 | 卫星点 | `satellites.js` | 固定 renderOrder | 屏幕空间卫星拾取 | 卫星点压过 footprint 和算力中心。 |
| 12+ | 卫星锁定 ring、halo、预测轨道 | `satellites.js` | `SATELLITE_CONFIG.overlayRenderOrder` 及偏移 | 卫星覆盖层路径 | 用于选中 / 锁定卫星强调。 |
| 98-100 | 太阳 / 月亮 halo 和 sprite | `celestial.js` | 固定 renderOrder | 天体拾取禁用 | 前景天体 sprite。 |
## 开关联动
| 开关 | 行为 |
| --- | --- |
| 高清材质 off | 隐藏高清材质,启用国界 tint / 基座表面,禁用地形和昼夜开关交互,并记住地形和昼夜之前状态。 |
| 高清材质 on | 恢复高清材质,并恢复记住的地形 / 昼夜状态。 |
| 地形 on | 显示在高清材质之上,但低于国界 hover、footprint、卫星等强调层。 |
| 大气云图 | 只控制云图 mesh 显隐。 |
| 国界 | 控制国界线和 hover 线显隐;海陆基座填充独立存在,作为 Earth 基座地图使用。 |
## 交互规则
| 交互 | 当前规则 |
| --- | --- |
| Earth 坐标 hover | 高清材质可见时使用高清材质 overlay 作为地表拾取目标,否则使用 Earth 基座球。 |
| 国界 hover | 先把地表拾取坐标转成经纬度,再用 GeoJSON 点面判断;国界 hover 线本身不接收 raycast。 |
| 国界 hover 视觉 | hover 时压暗普通国界线,并绘制无深度测试的光晕和实线。 |
| 中国 / 台湾 hover | `CHN``TWN` 被归到同一个 hover 高亮组tooltip 仍显示鼠标实际命中的 feature。 |
| 地形 | 只作为视觉层参与,`terrain.raycast` 已禁用。 |
| 卫星 | 使用屏幕空间卫星拾取,避免 footprint 或地表层挡住卫星点击。 |

View File

@@ -1,11 +1,11 @@
# Earth Satellite Footprint Policy
# Earth 卫星覆盖策略
本文件记录 Earth 卫星图层当前关于 `footprint` 的产品边界、资料依据和已落地实现,目标是避免把 Starlink 这套专用地表覆盖模型误用到其它星座上。
相关上下文:
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/backend-collectors.md)
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)
- [backend/app/services/collectors/celestrak.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/celestrak.py)
- [frontend/public/earth/js/satellites.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/satellites.js)

View File

@@ -1,11 +1,11 @@
# Admin Frontend Context
# 控制台前端结构
本文件描述当前控制台前端的真实结构,目标是帮助后续页面开发、表格改造、布局治理和状态收口时快速找到正确入口。
相关规则建议一起参考:
- [rules.md](/home/ray/dev/linkong/planet/rules.md)
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)
## 当前目标
@@ -116,11 +116,68 @@
- 为表格滚动区提供统一包裹层
- 后续新表格页优先复用,不要重复写“表格区域 + overlay scrollbar”样板
### 4. 其他共享组件
### 4. `SegmentedControl`
文件:
- [SegmentedControl.tsx](/home/ray/dev/linkong/planet/frontend/src/components/SegmentedControl/SegmentedControl.tsx)
- [SegmentedControl.css](/home/ray/dev/linkong/planet/frontend/src/components/SegmentedControl/SegmentedControl.css)
用途:
- 语言切换、主题切换、模式切换这类 2 到 3 项的分段控制器
- 需要保留滑块动画、激活态和紧凑按钮布局的设置项
- 当前 `/docs` 页底部语言切换与主题切换已经复用它
接口语义:
- `options`:每个选项包含 `value``label`,可选 `icon``title`
- `value`:当前激活值
- `onChange`:切换选项时回调
- `ariaLabel`:控制器可访问名称
- `className`:业务页面用于覆盖尺寸或局部样式
当前约束:
- 组件自身负责滑块数量、位置和弹性动画
- 业务页面只传选项和状态,不要重复写私有 slider DOM
- 颜色优先通过 CSS 变量覆盖,避免在业务组件里硬编码主题色
- 适合少量互斥选项,不适合用作长列表、导航菜单或表单下拉
### 5. `MarkdownRenderer`
文件:
- [MarkdownRenderer.tsx](/home/ray/dev/linkong/planet/frontend/src/components/MarkdownRenderer/MarkdownRenderer.tsx)
用途:
- 渲染 `/docs` 的 Markdown 正文
- 支持标题、列表、引用、代码块、表格和基础行内格式
- 代码块和表格内部复用 `Scrollbar`,避免横向内容撑爆文档页
当前约束:
- 它不是完整 GitHub Markdown 引擎,只覆盖项目文档当前需要的语法
- 文档内部链接应通过 `transformLink` 转成 `/docs/:slug`
- 标题锚点由 `getHeadingId` 注入,避免渲染器自己理解路由状态
### 6. `TableActions`
文件:
- [TableActions.tsx](/home/ray/dev/linkong/planet/frontend/src/components/TableActions/TableActions.tsx)
用途:
- 表格操作列的统一操作入口
- 展开状态下直接展示按钮
- 收起状态下用更多菜单承载操作
配套导出:
- `actionCellProps`:用于操作列 `onCell`,防止操作按钮被省略号截断或换行
## 当前状态来源
### 1. 认证状态
@@ -206,7 +263,7 @@
详细经验见:
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/frontend-layout-guidelines.md)
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)
## 当前推荐改动方式
@@ -233,4 +290,4 @@
Earth 相关结构见:
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md)
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)

View File

@@ -1,4 +1,4 @@
# Frontend Layout Guidelines
# 前端布局指南
本项目后台页面默认遵循“单屏工作区”布局规范。目标不是让页面永远不溢出,而是确保在常见桌面视口下:

488
docs/technical/zh/manual.md Normal file
View File

@@ -0,0 +1,488 @@
# Planet 使用手册
这份手册面向日常使用、演示、开发联调和本地运维。它覆盖四个核心入口:
- `planet.sh`:本地启动、停止、重启、健康检查和日志入口
- Earth公开 3D 地球态势页面
- 控制台:登录后的管理后台
- Docs公开开发文档与使用手册
快速启动路径见 [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md)。
## 入口总览
默认启动后,常用地址如下:
| 名称 | 地址 | 是否需要登录 | 说明 |
| --- | --- | --- | --- |
| Earth | `http://localhost:3000/earth` | 否 | 3D 地球、图层、BGP、卫星、海缆、新闻态势 |
| Docs | `http://localhost:3000/docs` | 否 | 开发文档、技术说明、使用手册 |
| 控制台 | `http://localhost:3000/admin` | 是 | 数据、配置、告警、日志和专题观测 |
| AI Playground | `http://localhost:3000/playground` | 是 | AI Provider 状态和调试 |
| 后端 API 文档 | `http://localhost:8000/docs` | 视接口而定 | FastAPI / OpenAPI 文档 |
## planet.sh
`planet.sh` 是本地开发和演示的主控脚本。优先使用它管理服务,而不是手动分别启动前端、后端、数据库和 AI Provider。
### 启动
```bash
./planet.sh start
```
默认行为:
- 启动 PostgreSQL 和 Redis
- 启动 AI Provider
- 启动后端 API
- 启动前端 Vite dev server
- 输出 Earth、控制台、Playground 和后端 API 文档入口
可指定端口:
```bash
./planet.sh start -b 8001 -f 3001 -a 8101
```
参数含义:
| 参数 | 含义 |
| --- | --- |
| `-b <port>` | 后端端口 |
| `-f <port>` | 前端端口 |
| `-a <port>` | AI Provider 端口 |
| `--allow-lan` | 允许局域网访问 |
| `--verbose` | 在执行过程中显示更多命令输出 |
### 停止
```bash
./planet.sh stop
```
会停止:
- 后端
- AI Provider
- 前端
- PostgreSQL
- Redis
### 重启
全量重启:
```bash
./planet.sh restart
```
按模块重启:
```bash
./planet.sh restart -b
./planet.sh restart -f
./planet.sh restart -a
./planet.sh restart -d
```
| 参数 | 作用 |
| --- | --- |
| `-b` | 只重启后端 |
| `-f` | 只重启前端 |
| `-a` | 只重启 AI Provider |
| `-d` | 只重启数据库 |
按模块重启适合日常开发,能避免无关服务被打断。
### 创建用户
```bash
./planet.sh createuser
```
用于首次进入控制台前创建登录账号。脚本会交互式提示用户名、密码和角色。
### 健康检查
```bash
./planet.sh health
```
会检查:
- `planet_*` 容器状态
- 后端 `/health`
- AI Provider `/health`
- 前端页面可达性
如果某项显示 offline优先查看对应日志。
### 日志
最近日志:
```bash
./planet.sh log
```
持续跟随日志:
```bash
./planet.sh log -f
./planet.sh log -b
./planet.sh log -a
```
| 参数 | 日志来源 |
| --- | --- |
| `-f` / `--frontend` | `/tmp/planet_frontend.log` |
| `-b` / `--backend` | `/tmp/planet_backend.log` |
| `-a` / `--ai-provider` | `planet_aiprovider` 容器日志 |
### 局域网访问
```bash
./planet.sh start --allow-lan
```
适合:
- WSL 中启动Windows 浏览器访问
- 手机或平板演示 Earth
- 局域网其他机器访问同一个开发实例
启动后注意检查防火墙和 WSL 网络转发。
## Earth
Earth 是公开的 3D 态势页面,入口:
```text
http://localhost:3000/earth
```
它是独立前端,实际页面位于:
- `frontend/public/earth/index.html`
- `frontend/public/earth/js/`
- `frontend/public/earth/css/`
React 路由中的 `/earth` 只是用 iframe 承载它。
### 主要用途
Earth 用于在一个地球视图中观察:
- BGP 事件、异常和观测态势
- 卫星和轨迹
- 海缆与登陆点
- 算力中心
- 国界、经纬线、高清材质、云图、地形
- 新闻直播和态势新闻
- 搜索和聚焦对象详情
### 图层控制
右侧图层面板用于打开或关闭可视图层。
常见图层包括:
- 经纬线
- 国界
- 高清材质
- 大气云图
- 海缆
- 算力中心
- BGP 观测
- 卫星
- 轨迹
- 地形
部分图层存在依赖关系:
- 地形依赖高清材质
- 轨迹依赖卫星
- 高清材质关闭时,地球会显示基座地图和边缘识别效果
### 搜索
Earth 搜索支持查找当前地球对象,例如:
- 海缆
- 登陆点
- 卫星
- 算力中心
- BGP 事件
- BGP 观测站
搜索结果可以用于快速定位对象,并打开对应详情。
### 设置
设置面板包含:
- 旋转模式 / 巡航模式
- 巡航模块BGP、新闻
- 卫星显示风格:自身发光、真实地表覆盖
- 日夜模式
- 面板显示开关
- 地球默认大小
- 地形透明度
- 重置设置
这些设置会保存在浏览器本地存储中。换浏览器或清理站点数据后会恢复默认值。
### 巡航模式
巡航模式会让 Earth 自动轮播聚焦目标。
当前巡航模块包括:
- BGP
- 新闻
适合演示、监控大屏或无人值守展示。
### 移动端
Earth 有移动端抽屉布局。小屏下:
- 图层控制进入移动抽屉
- 搜索、设置、详情会使用移动端面板
- 主要交互仍围绕地球对象点击、搜索和图层开关
### 常见问题
#### Earth 打不开
先检查前端是否在线:
```bash
./planet.sh health
./planet.sh log -f
```
如果前端端口不是 `3000`,使用启动时输出的实际端口。
#### 图层没有数据
检查后端和数据源:
```bash
./planet.sh health
./planet.sh log -b
```
然后进入控制台查看:
- `/datasources`
- `/data`
- `/bgp`
#### 卫星、BGP 或海缆加载慢
这些图层可能依赖后端接口、外部数据源或首次加载任务。先等待启动任务完成,再查看日志和控制台数据源状态。
## 控制台
控制台入口:
```text
http://localhost:3000/admin
```
控制台需要登录。首次使用先创建用户:
```bash
./planet.sh createuser
```
### 页面结构
控制台使用 React + Ant Design左侧菜单按工作域组织。
常见入口:
| 页面 | 路由 | 用途 |
| --- | --- | --- |
| 仪表盘 | `/admin` | 系统概览 |
| Earth | `/earth` | 打开公开 Earth 页面 |
| 数据源 | `/datasources` | 管理数据源和触发采集 |
| 采集数据 | `/data` | 查看采集后的数据 |
| BGP 观测 | `/bgp` | 查看 BGP 专题数据 |
| 系统告警 | `/alerts/system` | 系统级告警 |
| BGP 告警 | `/alerts/bgp` | BGP 相关告警 |
| 态势告警 | `/alerts/situational` | 态势研判告警 |
| AI Playground | `/playground` | AI Provider 调试 |
| 系统日志 | `/logs` | 查看系统日志,通常仅 super admin 可见 |
| 用户管理 | `/users` | 管理用户 |
| 系统配置 | `/settings` | 系统配置和电视直播源等设置 |
### 数据源
`/datasources` 用于查看和管理采集来源。
常见操作:
- 查看数据源状态
- 触发采集
- 查看最近采集任务
- 调整配置项
如果 Earth 上某类对象缺失,通常先到这里确认数据源是否可用。
### 采集数据
`/data` 用于查看采集后的数据表。
适合排查:
- 数据是否已经进入系统
- 数据更新时间是否符合预期
- 某个数据源是否产出了有效记录
### BGP 观测
`/bgp` 是 BGP 专题页面。
它和 Earth 的 BGP 图层互补:
- Earth 强调空间态势和可视聚焦
- 控制台 BGP 页面强调列表、状态、详情和研判
### 告警
告警入口包括:
- `/alerts/system`
- `/alerts/bgp`
- `/alerts/situational`
用于查看系统、网络和态势相关告警。
### 系统配置
`/settings` 用于管理系统级配置。
当前常见用途包括:
- 系统设置
- 电视直播源配置
- 数据源相关配置入口
具体可用配置取决于当前登录用户权限。
### 系统日志
`/logs` 用于查看系统日志。若菜单中不可见,通常是当前用户角色没有权限。
排查问题时常用组合:
```bash
./planet.sh health
./planet.sh log
```
再进入 `/logs` 查看更结构化的运行信息。
## Docs
公开文档站入口:
```text
http://localhost:3000/docs
```
当前公开内容来自:
```text
docs/technical/*.md
```
Docs 支持:
- 分类导航
- Markdown 渲染
- 表格和代码块
- 文档内目录
- 本地搜索
- technical 文档之间的内部链接跳转
如果新增 technical 文档,应同步检查:
- 是否有清晰的一级标题
- 是否需要加入 `/docs` 的人工分类和排序
- 是否包含不适合公开展示的信息
## 开发命令约定
前端命令必须使用 Bun
```bash
cd frontend
bun install
bun run dev
bun run build
```
不要使用 `npm run ...`。项目在 WSL / Windows 混合环境中优先依赖 Bun避免 Node/npm 路径差异带来的兼容问题。
验证前端构建:
```bash
source ~/.zshrc && bun run build
```
## 故障排查顺序
遇到问题时,建议按这个顺序排查:
1. 看服务状态:
```bash
./planet.sh health
```
2. 看最近日志:
```bash
./planet.sh log
```
3. 按模块查看日志:
```bash
./planet.sh log -f
./planet.sh log -b
./planet.sh log -a
```
4. 只重启有问题的模块:
```bash
./planet.sh restart -f
./planet.sh restart -b
./planet.sh restart -a
```
5. 如果数据库或缓存异常,再重启数据库:
```bash
./planet.sh restart -d
```
6. 仍无法恢复时,执行全量重启:
```bash
./planet.sh restart
```
## 相关文档
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md)
- [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
- [earth-layer-style-reference.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-layer-style-reference.md)
- [backend-system-service-control.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-system-service-control.md)
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)

View File

@@ -0,0 +1,154 @@
# planet.sh 启动性能优化
## 背景
`planet.sh` 管理所有服务的启动/停止/重启。原有实现存在以下问题:
1. AI Provider 每次都重新构建(即使代码未变)
2. 杀端口速度极慢(最长等 45 秒)
3. 端口绑定检测用 Python 子进程(每次 ~300ms
4. 无参 `restart``restart -b` 行为不一致
## 问题一AI Provider 每次重建
### 根因
构建戳文件存放在 `/tmp/`WSL/Linux 重启后 `/tmp` 被清空,导致三个条件中的"戳文件非空"这一条始终不满足,进而判定需要重建:
```bash
# 三个条件必须同时成立才跳过重建
image_exists AND stamp_non_empty AND fingerprint_match
```
### 修复
将戳文件路径从 `/tmp/` 改到持久路径:
```bash
AI_PROVIDER_BUILD_STAMP_FILE="$HOME/.cache/planet/aiprovider_build.sha256"
```
写入时确保目录存在:
```bash
write_ai_provider_build_stamp() {
mkdir -p "$(dirname "$AI_PROVIDER_BUILD_STAMP_FILE")"
compute_ai_provider_build_fingerprint > "$AI_PROVIDER_BUILD_STAMP_FILE"
}
```
### fingerprint 计算提速
原实现对整个 `aiprovider/` 打 tar 包再算 SHA大目录下耗时可达数秒。改为 `find + stat`(只读文件元信息,不读内容):
```bash
compute_ai_provider_build_fingerprint() {
find aiprovider \
-type f \
! -path '*/__pycache__/*' \
! -name '*.pyc' \
! -name '*.pyo' \
| LC_ALL=C sort \
| xargs -r stat --format="%Y %s %n" 2>/dev/null
sha256sum docker-compose.yml docker-compose.simple.yml 2>/dev/null
python3 "$SCRIPT_DIR/scripts/compute_aiprovider_dependency_fingerprint.py" 2>/dev/null
}
```
速度提升约 10 倍大量小文件场景误报率相同mtime+size 变化 ≡ 文件被修改)。
### 跳过重建的原理
fingerprint 一致时不执行 `docker compose build`,而是:
```bash
docker start planet_aiprovider # 启动已存在的容器,几秒内完成
```
`docker stop` 停容器,不删镜像;`cleanup_exit_containers` 删已退出容器,不删镜像。下次 `docker start` 会从现有镜像直接创建并启动容器。
## 问题二:杀端口速度慢
### 原因
`wait_for_port_release` 默认最多等 45 秒15 次 × 3 秒)。
### 修复
将后台进程清理场景的超时缩短至 3 秒TERM→1.5s→KILL→1.5s
```bash
PORT_RELEASE_ATTEMPTS=15
PORT_RELEASE_INTERVAL=0.2 # 每次等 0.2s,总计 3s
# cleanup_backend_processes / kill_port_if_requested
wait_for_port_release "$port" 15 0.2
```
`wait_for_port_release` 增加可选参数,允许不同场景使用不同超时:
```bash
wait_for_port_release() {
local port="$1"
local max_attempts="${2:-$PORT_RELEASE_ATTEMPTS}"
local interval="${3:-$PORT_RELEASE_INTERVAL}"
...
}
```
## 问题三:端口检测用 Python
### 原因
`can_bind_port``python3 -c "import socket..."` 检测端口,每次调用约 300ms。
### 修复
优先使用系统工具(~10msPython 作为兜底:
```bash
can_bind_port() {
local port="$1"
if command -v ss >/dev/null 2>&1; then
! ss -tlnH 2>/dev/null | awk '{print $4}' | grep -qE ":${port}$"
return
fi
if command -v lsof >/dev/null 2>&1; then
[ -z "$(lsof -tiTCP:"${port}" -sTCP:LISTEN 2>/dev/null)" ]
return
fi
python3 - "$port" <<'PY'
import sys, socket
p = int(sys.argv[1])
s = socket.socket()
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
try:
s.bind(("", p)); s.close(); sys.exit(0)
except OSError:
sys.exit(1)
PY
}
```
## 问题四restart 行为不一致
### 现象
- `restart -b`:停全部服务 → 检查 AI Provider fingerprint → 按需重建 → 启动
- `restart`(无参):停全部服务 → AI Provider 总是判定需要重建(因戳文件在 /tmp
### 修复
修复戳文件路径后,无参 `restart` 同样使用 `stop + start`fingerprint 检查正常生效,行为与 `restart -b` 完全一致。无需额外代码变更。
## 其他:移除不必要的 sleep
启动链路中两处 `sleep 3` 在实际已有健康检查覆盖的情况下多余,已移除:
- `start_backend_service`:数据库健康检查通过后的 `sleep 3`
- `restart_database_service`:重启后的等待 `sleep 3`
## 相关文件
- `planet.sh` — 全量修改
- `scripts/compute_aiprovider_dependency_fingerprint.py` — 依赖 fingerprint未改动

View File

@@ -0,0 +1,193 @@
# 快速开始
这份快速开始面向第一次启动 Planet 的开发者或演示操作者。目标是用最短路径把服务跑起来,并知道应该打开哪些入口。
## 前置条件
推荐在 WSL / Linux shell 中运行。
需要具备:
- Docker / Docker Compose 可用
- 当前 shell 能访问 `uv``bun`
- 仓库已 clone 到本机
如果是新机器,优先执行仓库自带初始化脚本:
```bash
./scripts/bootstrap-dev.sh
```
这个脚本会检查并同步常用依赖,并在缺少时生成:
- `backend/.env`
- `aiprovider/.env`
- `frontend/.env.local`
## 1. 启动服务
在仓库根目录执行:
```bash
./planet.sh start
```
启动完成后,常用入口是:
| 入口 | 默认地址 | 用途 |
| --- | --- | --- |
| Earth | `http://localhost:3000/earth` | 公开 3D Earth 可视化页面 |
| 控制台 | `http://localhost:3000/admin` | 登录后的管理后台 |
| 文档站 | `http://localhost:3000/docs` | 公开开发文档和使用手册 |
| AI Playground | `http://localhost:3000/playground` | 登录后的 AI 调试入口 |
| 后端 API 文档 | `http://localhost:8000/docs` | FastAPI / OpenAPI 接口文档 |
如果默认端口被占用,可以指定端口:
```bash
./planet.sh start -f 3001 -b 8001 -a 8101
```
## 2. 创建登录用户
控制台需要登录。首次使用可以执行:
```bash
./planet.sh createuser
```
按提示输入用户名、密码和角色。
## 3. 打开 Earth
访问:
```text
http://localhost:3000/earth
```
Earth 是公开页面,不需要登录。
进入后可以先确认:
- 地球正常显示
- 右侧图层控制可打开/关闭图层
- 搜索可以查找海缆、卫星、算力中心、BGP 事件
- 设置面板可以切换巡航模式、日夜模式、卫星显示风格
## 4. 打开控制台
访问:
```text
http://localhost:3000/admin
```
控制台用于数据源、采集数据、专题观测、告警、系统日志和配置管理。
首次排查建议查看:
- `/datasources`:数据源配置和采集状态
- `/data`:已采集数据
- `/bgp`BGP 专题观测
- `/alerts/system`:系统告警
- `/settings`:系统配置
## 5. 查看运行状态
```bash
./planet.sh health
```
这个命令会显示容器状态,并检查:
- 后端
- AI Provider
- 前端
## 6. 查看日志
最近日志:
```bash
./planet.sh log
```
持续查看某个服务:
```bash
./planet.sh log -f
./planet.sh log -b
./planet.sh log -a
```
含义:
- `-f`:前端日志
- `-b`:后端日志
- `-a`AI Provider 日志
## 7. 常用重启
只重启前端:
```bash
./planet.sh restart -f
```
只重启后端:
```bash
./planet.sh restart -b
```
只重启 AI Provider
```bash
./planet.sh restart -a
```
只重启数据库:
```bash
./planet.sh restart -d
```
全量重启:
```bash
./planet.sh restart
```
## 8. 局域网访问
如果希望 Windows 浏览器、手机或同一局域网的其他设备访问:
```bash
./planet.sh start --allow-lan
```
这会让前端和后端监听局域网可访问地址。
如果访问失败,先在运行 Planet 的 shell 中检查:
```bash
curl http://localhost:3000
curl http://localhost:8000/health
ss -ltnp | grep -E ':3000|:8000'
```
## 9. 停止服务
```bash
./planet.sh stop
```
停止后会关闭前端、后端、AI Provider、PostgreSQL 和 Redis。
## 下一步
- 完整操作说明见 [manual.md](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md)
- 控制台结构见 [frontend-admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)
- Earth 结构见 [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
- 后端采集器见 [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)

View File

@@ -16,12 +16,21 @@
## Current Version
- `main` 当前主线历史推导到:`0.16.5`
- `dev` 当前开发分支历史推导到:`0.40.3`
- `dev` 当前开发分支历史推导到:`0.43.0`
## Timeline
| Version | Type | Branch | Commit | Summary |
| --- | --- | --- | --- | --- |
| `0.43.0` | feature | `dev` | `pending` | 新增 Earth 船舶追踪、自定义数据源映射、外部集成配置中心、Markdown 渲染器增强,并整理规则/技能文档加载约束 |
| `0.42.2` | bugfix | `dev` | `pending` | Docs 中文模式补齐分组和文档标题翻译,并更新文档站品牌文案 |
| `0.42.1` | bugfix | `dev` | `pending` | 修正 release skill 的 feature 版本计算规则minor 进位时重置 patch 为 0 |
| `0.42.0` | feature | `dev` | `pending` | 新增公开 `/docs` 文档站、中英文技术/使用文档、搜索与主题切换,并补充公共组件复用和 Earth 无高清材质边缘提示 |
| `0.41.2` | improvement | `dev` | `pending` | 启动脚本新增 verbose 滚动日志与端口占用诊断Docker 构建支持镜像源覆盖,并修复 Earth 登陆点遮挡判断 |
| `0.41.1` | improvement | `dev` | `pending` | 修复新闻直播持久化失效、pin 边缘遮挡;图标抽取为 SVG 并建立规范 |
| `0.41.0` | feature | `dev` | `pending` | Earth 图层顺序拆分、基座海陆色块、国界交互、高清材质/云图/地形层级与样式文档落地 |
| `0.40.5` | improvement | `dev` | `pending` | 卫星 ribbon 拖尾、Iridium 覆盖球面投影填充+外圈、搜索自动聚焦修复 |
| `0.40.4` | bugfix | `dev` | `pending` | 修复页面后台恢复后卫星轨迹跳变与位置错位,统一轨迹重置路径 |
| `0.40.3` | improvement | `dev` | `pending` | 卫星点云升级 ShaderMaterial修复锁定环 depthTest 与位置漂移,新增悬停态缩放 |
| `0.40.2` | improvement | `dev` | `pending` | 卫星点大小随镜头缩放动态调整,调小默认基础尺寸 |
| `0.40.1` | improvement | `dev` | `pending` | 卫星选中标记配色跟随图例,修复 footprint 遮蔽卫星渲染问题,修复选中海缆误触发卫星高亮 |

View File

@@ -1,6 +1,6 @@
{
"name": "planet-frontend",
"version": "0.40.3",
"version": "0.43.0",
"private": true,
"packageManager": "bun@1",
"dependencies": {

View File

@@ -0,0 +1,19 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP route collector marker. Outer ring + Material access_point icon. -->
<!-- States: idle opacity=0.74, hover/locked use brighter blend (color controlled externally) -->
<!-- Outer ring -->
<circle cx="64" cy="64" r="22" fill="none" stroke="rgba(111,160,197,0.34)" stroke-width="1.2"/>
<!-- access_point icon: 24x24 path scaled 4x and offset to (16,16) in 128x128 canvas space -->
<g transform="translate(16 16) scale(4 4)"
fill="rgba(214,224,233,0.88)"
stroke="rgba(64,106,136,0.74)"
stroke-width="0.9"
stroke-linejoin="round"
stroke-linecap="round">
<path d="M4.93 4.93A9.97 9.97 0 0 0 2 12c0 2.76 1.12 5.26 2.93 7.07l1.41-1.41A7.94 7.94 0 0 1 4 12c0-2.21.89-4.22 2.34-5.66zm14.14 0l-1.41 1.41A7.96 7.96 0 0 1 20 12c0 2.22-.89 4.22-2.34 5.66l1.41 1.41A9.97 9.97 0 0 0 22 12c0-2.76-1.12-5.26-2.93-7.07M7.76 7.76A5.98 5.98 0 0 0 6 12c0 1.65.67 3.15 1.76 4.24l1.41-1.41A4 4 0 0 1 8 12c0-1.11.45-2.11 1.17-2.83zm8.48 0l-1.41 1.41A4 4 0 0 1 16 12c0 1.11-.45 2.11-1.17 2.83l1.41 1.41A5.98 5.98 0 0 0 18 12c0-1.65-.67-3.15-1.76-4.24M12 10a2 2 0 0 0-2 2a2 2 0 0 0 2 2a2 2 0 0 0 2-2a2 2 0 0 0-2-2"/>
<!-- Center dot override -->
<circle cx="12" cy="12" r="0.85" fill="rgba(222,231,239,0.72)" stroke="none"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 1.3 KiB

View File

@@ -0,0 +1,13 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP specific/burst anomaly symbol: 6 radial spokes (r 26→48) + center dot (r=16). -->
<!-- Spoke endpoints calculated as: inner=64+cos(angle)*26, outer=64+cos(angle)*48 for 6 angles -->
<g stroke="currentColor" stroke-width="10" stroke-linecap="round">
<line x1="90" y1="64" x2="112" y2="64"/>
<line x1="77" y1="86.5" x2="88" y2="105.6"/>
<line x1="51" y1="86.5" x2="40" y2="105.6"/>
<line x1="38" y1="64" x2="16" y2="64"/>
<line x1="51" y1="41.5" x2="40" y2="22.4"/>
<line x1="77" y1="41.5" x2="88" y2="22.4"/>
</g>
<circle cx="64" cy="64" r="16" fill="currentColor"/>
</svg>

After

Width:  |  Height:  |  Size: 714 B

View File

@@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP generic event symbol: filled circle. -->
<circle cx="64" cy="64" r="28" fill="currentColor"/>
</svg>

After

Width:  |  Height:  |  Size: 177 B

View File

@@ -0,0 +1,5 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP withdraw event symbol: exclamation mark (rounded bar + dot). -->
<rect x="52" y="22" width="24" height="62" rx="12" fill="currentColor"/>
<circle cx="64" cy="102" r="10" fill="currentColor"/>
</svg>

After

Width:  |  Height:  |  Size: 277 B

View File

@@ -0,0 +1,7 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP route leak symbol: two nested open triangular outlines (outer + inner chevron). -->
<g stroke="currentColor" stroke-width="10" stroke-linecap="round" stroke-linejoin="round" fill="none">
<polyline points="28,96 64,28 100,96"/>
<polyline points="40,82 64,54 88,82"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 364 B

View File

@@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- Ring overlay for BGP event markers. Stroked circle, no fill. -->
<circle cx="64" cy="64" r="44" fill="none" stroke="rgba(255,255,255,0.98)" stroke-width="6"/>
</svg>

After

Width:  |  Height:  |  Size: 238 B

View File

@@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP origin anomaly symbol: upward triangle. -->
<polygon points="64,18 110,106 18,106" fill="currentColor"/>
</svg>

After

Width:  |  Height:  |  Size: 188 B

View File

@@ -0,0 +1,4 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP flap event symbol: zigzag/wave (filled W shape, closed). -->
<polygon points="14,100 38,26 64,100 90,26 114,100" fill="currentColor"/>
</svg>

After

Width:  |  Height:  |  Size: 218 B

View File

@@ -0,0 +1,12 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- BGP collector base glow: radial gradient dot. Inner r=8 fully opaque, fades to transparent at r=56. -->
<defs>
<radialGradient id="bgp-glow" cx="64" cy="64" r="56" fx="64" fy="64" fr="8" gradientUnits="userSpaceOnUse">
<stop offset="0%" stop-color="white" stop-opacity="1"/>
<stop offset="24%" stop-color="white" stop-opacity="0.92"/>
<stop offset="58%" stop-color="white" stop-opacity="0.35"/>
<stop offset="100%" stop-color="white" stop-opacity="0"/>
</radialGradient>
</defs>
<circle cx="64" cy="64" r="56" fill="url(#bgp-glow)"/>
</svg>

After

Width:  |  Height:  |  Size: 653 B

View File

@@ -0,0 +1,16 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- GPU cluster marker: database/cylinder stack icon. -->
<!-- Color: #2dd4bf (teal) per COMPUTE_CENTER_CONFIG.colors.gpu_cluster -->
<!-- States: normal, estimated (adds a "?" badge drawn separately at canvas level) -->
<!-- Outer cylinder: top ellipse cap + side rect + bottom half-ellipse -->
<!-- Inner groove ring: smaller cylinder shape overlaid at same color (subtle shape layering) -->
<g fill="#2dd4bf">
<rect x="46" y="46" width="36" height="28"/>
<ellipse cx="64" cy="46" rx="18" ry="8"/>
<path d="M 82,74 A 18,8 0 0,1 46,74 Z"/>
<rect x="52" y="58" width="24" height="6"/>
<ellipse cx="64" cy="58" rx="12" ry="4.5"/>
<path d="M 76,64 A 12,4.5 0 0,1 52,64 Z"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 787 B

View File

@@ -0,0 +1,10 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 128 128">
<!-- Supercomputer marker: flat-screen monitor with neck and base stand. -->
<!-- Color: #38bdf8 (sky-blue) per COMPUTE_CENTER_CONFIG.colors.supercomputer -->
<!-- States: normal, estimated (adds a "?" badge drawn separately at canvas level) -->
<g fill="#38bdf8">
<rect x="40" y="42" width="48" height="30" rx="7"/>
<rect x="58" y="74" width="12" height="8" rx="3"/>
<rect x="50" y="84" width="28" height="5" rx="2.5"/>
</g>
</svg>

After

Width:  |  Height:  |  Size: 520 B

View File

@@ -0,0 +1,10 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="100 50 600 680">
<!-- Location pin with circular cutout. Coordinates in 1000-unit space (canvas scale: size/1000). -->
<!-- States: color via fill attribute — default white for 3D canvas, orange (#ffaa00) for normal, white for hover/locked -->
<path
fill-rule="evenodd"
fill="currentColor"
d="M400 704 C386 704 375 697 367 684 L173 378 C117 290 144 173 229 111 C278 75 337 57 400 57 C463 57 522 75 571 111 C656 173 683 290 627 378 L433 684 C425 697 414 704 400 704 Z
M400 320 m-86 0 a86 86 0 1 0 172 0 a86 86 0 1 0 -172 0"
/>
</svg>

After

Width:  |  Height:  |  Size: 611 B

View File

@@ -827,6 +827,12 @@
rgba(255, 255, 255, 0.04);
}
.earth-mobile-layer-card.is-disabled,
.earth-mobile-layer-card:disabled {
cursor: not-allowed;
opacity: 0.46;
}
.earth-mobile-layer-card-icon {
font-size: 22px;
color: var(--hud-accent-strong);
@@ -1401,6 +1407,12 @@
transform: translateX(16px);
}
label.is-disabled.earth-mobile-settings-card {
opacity: 0.38;
cursor: not-allowed;
pointer-events: none;
}
.earth-mobile-settings-slider-row {
display: flex;
align-items: center;
@@ -2478,6 +2490,12 @@
transform: translateX(calc(16px * var(--hud-scale)));
}
.earth-settings-item.is-disabled {
opacity: 0.38;
cursor: not-allowed;
pointer-events: none;
}
@media (max-width: 960px) {
.earth-settings-sheet {
top: 24px;

View File

@@ -271,6 +271,16 @@
opacity: 1;
}
.layer-row-toggle.is-disabled {
cursor: not-allowed;
opacity: 0.35;
}
.layer-row:has(.layer-row-toggle.is-disabled) .layer-row-label,
.layer-row:has(.layer-row-toggle.is-disabled) .layer-row-icon {
opacity: 0.4;
}
/* Thumb */
.layer-row-toggle-track::after {
content: "";

File diff suppressed because one or more lines are too long

View File

@@ -108,33 +108,13 @@
<!-- Layer rows -->
<div class="layer-panel-list" id="layer-panel-list">
<div class="layer-row" data-layer-name="地形 terrain">
<span class="material-symbols-rounded layer-row-icon">landscape</span>
<div class="layer-row" data-layer-name="海缆 subsea cables">
<span class="material-symbols-rounded layer-row-icon">cable</span>
<div class="layer-row-copy">
<span class="layer-row-label">地形</span>
<span class="layer-row-meta">Terrain</span>
<span class="layer-row-label">海缆</span>
<span class="layer-row-meta">Subsea Cables</span>
</div>
<button id="toggle-terrain" class="layer-row-toggle" type="button" role="switch" aria-checked="false" title="切换地形显示" data-status-target="terrain-status">
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="经纬线 graticule 经纬 latitude longitude">
<span class="material-symbols-rounded layer-row-icon">grid_4x4</span>
<div class="layer-row-copy">
<span class="layer-row-label">经纬线</span>
<span class="layer-row-meta">Graticule</span>
</div>
<button id="toggle-grid-lines" class="layer-row-toggle active" type="button" role="switch" aria-checked="true" title="切换经纬线显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="卫星 satellites">
<span class="material-symbols-rounded layer-row-icon">satellite_alt</span>
<div class="layer-row-copy">
<span class="layer-row-label">卫星</span>
<span class="layer-row-meta">Satellites</span>
</div>
<button id="toggle-satellites" class="layer-row-toggle" type="button" role="switch" aria-checked="false" title="切换卫星显示">
<button id="toggle-cables" class="layer-row-toggle active" type="button" role="switch" aria-checked="true" title="切换海缆显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
@@ -148,13 +128,13 @@
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="海缆 subsea cables">
<span class="material-symbols-rounded layer-row-icon">cable</span>
<div class="layer-row" data-layer-name="卫星 satellites">
<span class="material-symbols-rounded layer-row-icon">satellite_alt</span>
<div class="layer-row-copy">
<span class="layer-row-label">海缆</span>
<span class="layer-row-meta">Subsea Cables</span>
<span class="layer-row-label">卫星</span>
<span class="layer-row-meta">Satellites</span>
</div>
<button id="toggle-cables" class="layer-row-toggle active" type="button" role="switch" aria-checked="true" title="切换海缆显示">
<button id="toggle-satellites" class="layer-row-toggle" type="button" role="switch" aria-checked="false" title="切换卫星显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
@@ -168,6 +148,16 @@
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="船只 船舶 ais vessels ships maritime">
<span class="material-symbols-rounded layer-row-icon">directions_boat</span>
<div class="layer-row-copy">
<span class="layer-row-label">船只</span>
<span class="layer-row-meta">AIS Vessels</span>
</div>
<button id="toggle-vessels" class="layer-row-toggle" type="button" role="switch" aria-checked="false" title="切换船只显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="bgp观测 routing signals">
<span class="material-symbols-rounded layer-row-icon">hub</span>
<div class="layer-row-copy">
@@ -178,6 +168,56 @@
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="地形 terrain">
<span class="material-symbols-rounded layer-row-icon">landscape</span>
<div class="layer-row-copy">
<span class="layer-row-label">地形</span>
<span class="layer-row-meta">Terrain</span>
</div>
<button id="toggle-terrain" class="layer-row-toggle" type="button" role="switch" aria-checked="false" title="切换地形显示" data-status-target="terrain-status">
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="高清材质 纹理 texture hd earth">
<span class="material-symbols-rounded layer-row-icon">globe</span>
<div class="layer-row-copy">
<span class="layer-row-label">高清材质</span>
<span class="layer-row-meta">High-Res Texture</span>
</div>
<button id="toggle-earth-high-res-texture" class="layer-row-toggle active" type="button" role="switch" aria-checked="true" title="切换高清材质显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="大气 云图 云层 clouds atmosphere">
<span class="material-symbols-rounded layer-row-icon">cloud</span>
<div class="layer-row-copy">
<span class="layer-row-label">大气云图</span>
<span class="layer-row-meta">Cloud Layer</span>
</div>
<button id="toggle-atmosphere-clouds" class="layer-row-toggle active" type="button" role="switch" aria-checked="true" title="切换大气云图显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="国界 国家 borders countries boundary">
<span class="material-symbols-rounded layer-row-icon">public</span>
<div class="layer-row-copy">
<span class="layer-row-label">国界</span>
<span class="layer-row-meta">Country Borders</span>
</div>
<button id="toggle-country-boundaries" class="layer-row-toggle active" type="button" role="switch" aria-checked="true" title="切换国界显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
<div class="layer-row" data-layer-name="经纬线 graticule 经纬 latitude longitude">
<span class="material-symbols-rounded layer-row-icon">grid_4x4</span>
<div class="layer-row-copy">
<span class="layer-row-label">经纬线</span>
<span class="layer-row-meta">Graticule</span>
</div>
<button id="toggle-grid-lines" class="layer-row-toggle" type="button" role="switch" aria-checked="false" title="切换经纬线显示">
<span class="layer-row-toggle-track"></span>
</button>
</div>
</div>
<!-- Empty search state -->
@@ -333,6 +373,10 @@
<span class="stat-num" id="compute-center-count" data-earth-stat="compute-center-count"></span>
<span class="stat-label">算力中心</span>
</div>
<div class="stat-cell">
<span class="stat-num" id="vessel-count" data-earth-stat="vessel-count"></span>
<span class="stat-label">AIS 船只</span>
</div>
<div class="stat-cell">
<span class="stat-num" id="bgp-anomaly-count" data-earth-stat="bgp-anomaly-count"></span>
<span class="stat-label">BGP 事件</span>
@@ -695,8 +739,8 @@
<span class="earth-mobile-settings-subtitle">可选自身发光或真实地表覆盖两种选中表现</span>
</div>
<div class="earth-mobile-settings-segmented" role="group" aria-label="移动端选择卫星显示风格">
<button type="button" class="earth-mobile-settings-pill is-active" data-satellite-display-style="self_glow" aria-pressed="true">自身发光</button>
<button type="button" class="earth-mobile-settings-pill" data-satellite-display-style="ground_footprint" aria-pressed="false">真实地表覆盖</button>
<button type="button" class="earth-mobile-settings-pill" data-satellite-display-style="self_glow" aria-pressed="false">自身发光</button>
<button type="button" class="earth-mobile-settings-pill is-active" data-satellite-display-style="ground_footprint" aria-pressed="true">真实地表覆盖</button>
</div>
</div>
</div>
@@ -896,17 +940,17 @@
<div class="earth-settings-segmented" role="group" aria-label="选择卫星显示风格">
<button
type="button"
class="earth-settings-segmented-btn is-active"
class="earth-settings-segmented-btn"
data-satellite-display-style="self_glow"
aria-pressed="true"
aria-pressed="false"
>
自身发光
</button>
<button
type="button"
class="earth-settings-segmented-btn"
class="earth-settings-segmented-btn is-active"
data-satellite-display-style="ground_footprint"
aria-pressed="false"
aria-pressed="true"
>
真实地表覆盖
</button>

View File

@@ -20,7 +20,60 @@ export let lockedCable = null;
let cableIdMap = new Map();
let cableStates = new Map();
let cablesVisible = true;
let landingPointGeometry = null;
let landingPointTexture = null;
const _lpEarthWorldPos = new THREE.Vector3();
const _lpWorldPos = new THREE.Vector3();
const _lpCameraRel = new THREE.Vector3();
const _lpPointRel = new THREE.Vector3();
const _lpCameraToPoint = new THREE.Vector3();
function createLandingPointTexture() {
const size = CABLE_CONFIG.landingPoint.textureSize;
const canvas = document.createElement("canvas");
canvas.width = size;
canvas.height = size;
const ctx = canvas.getContext("2d");
const iconPath = new Path2D(
[
"M400 704",
"C386 704 375 697 367 684",
"L173 378",
"C117 290 144 173 229 111",
"C278 75 337 57 400 57",
"C463 57 522 75 571 111",
"C656 173 683 290 627 378",
"L433 684",
"C425 697 414 704 400 704",
"Z",
].join(" "),
);
ctx.clearRect(0, 0, size, size);
ctx.save();
ctx.translate(size * 0.12, size * 0.02);
ctx.scale(size / 1000, size / 1000);
ctx.fillStyle = "#ffffff";
ctx.fill(iconPath);
ctx.globalCompositeOperation = "destination-out";
ctx.beginPath();
ctx.arc(400, 320, 86, 0, Math.PI * 2);
ctx.fill();
ctx.restore();
const texture = new THREE.CanvasTexture(canvas);
texture.colorSpace = THREE.SRGBColorSpace;
texture.needsUpdate = true;
return texture;
}
function getLandingPointTexture() {
if (!landingPointTexture) {
landingPointTexture = createLandingPointTexture();
}
return landingPointTexture;
}
function clamp(value, min, max) {
return Math.min(max, Math.max(min, value));
@@ -49,7 +102,7 @@ function disposeMaterial(material) {
return;
}
if (material.map) {
if (material.map && !material.userData?.sharedMap) {
material.map.dispose();
}
material.dispose();
@@ -69,6 +122,22 @@ function disposeObject(object, parent) {
}
}
function setLandingPointMaterialState(point, { color, opacity, emissive, emissiveIntensity }) {
point.material.color.set(color);
point.material.opacity = opacity;
if (point.material.emissive && emissive !== undefined) {
point.material.emissive.setHex(emissive);
}
if ("emissiveIntensity" in point.material && emissiveIntensity !== undefined) {
point.material.emissiveIntensity = emissiveIntensity;
}
}
function setLandingPointScale(point, heightScale) {
const aspect = CABLE_CONFIG.landingPoint.iconAspectRatio;
point.scale.set(heightScale * aspect, heightScale, 1);
}
function getCableColor(properties) {
if (properties.color) {
if (
@@ -357,13 +426,6 @@ export async function loadLandingPoints(scene, earthObj, options = {}) {
clearLandingPoints(earthObj);
if (!landingPointGeometry) {
landingPointGeometry = new THREE.SphereGeometry(
CABLE_CONFIG.landingPoint.radius,
CABLE_CONFIG.landingPoint.widthSegments,
CABLE_CONFIG.landingPoint.heightSegments,
);
}
let validCount = 0;
for (const feature of data.features) {
@@ -396,29 +458,35 @@ export async function loadLandingPoints(scene, earthObj, options = {}) {
continue;
}
const sphere = new THREE.Mesh(
landingPointGeometry,
new THREE.MeshStandardMaterial({
const marker = new THREE.Sprite(
new THREE.SpriteMaterial({
map: getLandingPointTexture(),
color: CABLE_CONFIG.landingPoint.color,
emissive: CABLE_CONFIG.landingPoint.emissive,
emissiveIntensity: CABLE_CONFIG.landingPoint.emissiveIntensity,
transparent: true,
opacity: CABLE_CONFIG.landingPoint.opacity,
depthTest: false,
depthWrite: false,
}),
);
sphere.position.copy(position);
sphere.userData = {
marker.material.userData.sharedMap = true;
marker.renderOrder = CABLE_CONFIG.landingPoint.renderOrder;
marker.center.set(
CABLE_CONFIG.landingPoint.anchorX,
CABLE_CONFIG.landingPoint.anchorY,
);
marker.position.copy(position);
marker.userData = {
type: "landingPoint",
name: properties.name || "未知登陆站",
cableNames: properties.cable_names || [],
country: properties.country || "未知国家",
status: properties.status || "Unknown",
baseScale: CABLE_CONFIG.landingPoint.baseScale,
sharedGeometry: true,
};
setLandingPointScale(marker, CABLE_CONFIG.landingPoint.baseScale);
earthObj.add(sphere);
landingPoints.push(sphere);
earthObj.add(marker);
landingPoints.push(marker);
validCount++;
}
@@ -533,6 +601,42 @@ export function getAllLandingPoints() {
return landingPoints;
}
function isFacingCamera(lp, camera) {
lp.getWorldPosition(_lpWorldPos);
if (lp.parent) {
lp.parent.getWorldPosition(_lpEarthWorldPos);
} else {
_lpEarthWorldPos.set(0, 0, 0);
}
_lpCameraRel.copy(camera.position).sub(_lpEarthWorldPos);
_lpPointRel.copy(_lpWorldPos).sub(_lpEarthWorldPos);
_lpCameraToPoint.subVectors(_lpPointRel, _lpCameraRel);
const distanceSq = _lpCameraToPoint.lengthSq();
if (distanceSq <= 0) return true;
const distance = Math.sqrt(distanceSq);
_lpCameraToPoint.multiplyScalar(1 / distance);
// The pin sprite is rendered without depth testing so its full shape does
// not get sliced by the globe. Instead, hide it when the camera-to-anchor
// segment is occluded by a slightly inflated globe, matching the behavior of
// the BGP and compute-center markers near the limb.
const occlusionRadius =
CONFIG.earthRadius + CABLE_CONFIG.landingPoint.altitudeOffset * 0.45;
const cameraProjection = _lpCameraRel.dot(_lpCameraToPoint);
const cameraRadiusSq = _lpCameraRel.lengthSq();
const discriminant =
cameraProjection * cameraProjection -
(cameraRadiusSq - occlusionRadius * occlusionRadius);
if (discriminant < 0) return true;
const nearestIntersection = -cameraProjection - Math.sqrt(discriminant);
return nearestIntersection <= 0 || nearestIntersection >= distance;
}
export function applyLandingPointVisualState(lockedCableName, dimAll = false, camera = null) {
const pulse =
(Math.sin(Date.now() * CABLE_CONFIG.landingPointVisual.pulseSpeed) + 1) * 0.5;
@@ -544,58 +648,65 @@ export function applyLandingPointVisualState(lockedCableName, dimAll = false, ca
: [];
landingPoints.forEach((lp) => {
lp.visible = cablesVisible && (camera ? isFacingCamera(lp, camera) : true);
const isRelated =
!dimAll &&
Array.isArray(lp.userData.cableNames) &&
lp.userData.cableNames.some((name) => relatedNames.includes(name));
if (isRelated) {
lp.material.color.setHex(0xffd27a);
lp.material.emissive.setHex(0x7a4a00);
lp.material.emissiveIntensity =
CABLE_CONFIG.landingPointVisual.related.emissiveIntensityBase +
0.2 +
pulse * (CABLE_CONFIG.landingPointVisual.related.emissiveIntensityPulse + 0.2);
lp.material.opacity =
Math.max(
setLandingPointMaterialState(lp, {
color: 0xffd27a,
emissive: 0x7a4a00,
emissiveIntensity:
CABLE_CONFIG.landingPointVisual.related.emissiveIntensityBase +
0.2 +
pulse * (CABLE_CONFIG.landingPointVisual.related.emissiveIntensityPulse + 0.2),
opacity: Math.max(
0.92,
CABLE_CONFIG.landingPointVisual.related.opacityBase +
pulse * CABLE_CONFIG.landingPointVisual.related.opacityPulse,
);
),
});
const distanceScale = getLandingPointDistanceScale(lp, camera);
const baseScale = lp.userData?.baseScale || CABLE_CONFIG.landingPoint.baseScale;
lp.scale.setScalar(
setLandingPointScale(
lp,
(CABLE_CONFIG.landingPointVisual.related.scaleBase +
pulse * CABLE_CONFIG.landingPointVisual.related.scalePulse) *
baseScale *
distanceScale,
distanceScale,
);
} else {
const dimColor = CABLE_CONFIG.landingPointVisual.dimmed.colorRGB;
const r = dimColor.r * brightness;
const g = dimColor.g * brightness;
const b = dimColor.b * brightness;
lp.material.color.setRGB(r / 255, g / 255, b / 255);
lp.material.emissive.setHex(CABLE_CONFIG.landingPointVisual.dimmed.emissive);
lp.material.emissiveIntensity =
CABLE_CONFIG.landingPointVisual.dimmed.emissiveIntensity;
lp.material.opacity = CABLE_CONFIG.landingPointVisual.dimmed.opacity;
setLandingPointMaterialState(lp, {
color: new THREE.Color(r / 255, g / 255, b / 255),
emissive: CABLE_CONFIG.landingPointVisual.dimmed.emissive,
emissiveIntensity: CABLE_CONFIG.landingPointVisual.dimmed.emissiveIntensity,
opacity: CABLE_CONFIG.landingPointVisual.dimmed.opacity,
});
const distanceScale = getLandingPointDistanceScale(lp, camera);
const baseScale = lp.userData?.baseScale || CABLE_CONFIG.landingPoint.baseScale;
lp.scale.setScalar(baseScale * distanceScale);
setLandingPointScale(lp, baseScale * distanceScale);
}
});
}
export function resetLandingPointVisualState(camera = null) {
landingPoints.forEach((lp) => {
lp.material.color.setHex(CABLE_CONFIG.landingPoint.color);
lp.material.emissive.setHex(CABLE_CONFIG.landingPoint.emissive);
lp.material.emissiveIntensity = CABLE_CONFIG.landingPoint.emissiveIntensity;
lp.material.opacity = CABLE_CONFIG.landingPoint.opacity;
lp.visible = cablesVisible && (camera ? isFacingCamera(lp, camera) : true);
setLandingPointMaterialState(lp, {
color: CABLE_CONFIG.landingPoint.color,
emissive: CABLE_CONFIG.landingPoint.emissive,
emissiveIntensity: CABLE_CONFIG.landingPoint.emissiveIntensity,
opacity: CABLE_CONFIG.landingPoint.opacity,
});
const distanceScale = getLandingPointDistanceScale(lp, camera);
const baseScale = lp.userData?.baseScale || CABLE_CONFIG.landingPoint.baseScale;
lp.scale.setScalar(baseScale * distanceScale);
setLandingPointScale(lp, baseScale * distanceScale);
});
}

View File

@@ -5,6 +5,7 @@ import { getSurfaceMarkerCameraScale, latLonToVector3 } from "./utils.js";
const computeCenterGroup = new THREE.Group();
const computeCenterMarkers = [];
const COMPUTE_CENTER_RENDER_ORDER = 4.5;
const textureCache = new Map();
let showComputeCenters = true;
let supercomputerCount = 0;
@@ -196,7 +197,7 @@ function createComputeCenterMarker(markerData) {
),
);
marker.scale.setScalar(baseScale);
marker.renderOrder = 8;
marker.renderOrder = COMPUTE_CENTER_RENDER_ORDER;
marker.visible = showComputeCenters;
marker.userData = {
...markerData,

View File

@@ -31,7 +31,7 @@ export const SATELLITE_DISPLAY_STYLES = {
};
export const DEFAULT_SATELLITE_DISPLAY_STYLE =
SATELLITE_DISPLAY_STYLES.SELF_GLOW;
SATELLITE_DISPLAY_STYLES.GROUND_FOOTPRINT;
export const CRUISE_CONFIG = {
dwellMs: 7_000,
@@ -155,29 +155,88 @@ export const TERRAIN_CONFIG = {
baseZoom: 4,
geometryWidthSegments: 320,
geometryHeightSegments: 320,
baseRadiusOffset: 0.04,
baseRadiusOffset: 0.16,
exaggeration: 34,
landRevealFadeMeters: 220,
maxConcurrentRequests: 10,
opacity: 0.62,
color: 0x7f9d7f,
emissive: 0x061008,
specular: 0x233126,
shininess: 10,
opacity: 0.68,
color: 0x8aa884,
emissive: 0x030704,
specular: 0x344438,
shininess: 16,
urlTemplate:
"/api/v1/visualization/terrain/terrarium/{z}/{x}/{y}.png",
};
export const COUNTRY_BOUNDARY_CONFIG = {
dataPath: "/earth/data/countries-admin0.min.geojson",
lineAltitudeOffset: 0.24,
hoverAltitudeOffset: 0.32,
lineColor: 0x7fc7ff,
lineOpacity: 0.58,
lineRenderOrder: 2.2,
dimmedLineOpacity: 0.18,
hoverLineColor: 0xff3b1f,
hoverLineOpacity: 1.0,
hoverLineRenderOrder: 2.3,
hoverGlowOpacity: 0.38,
hoverGlowLineWidth: 3,
hoverGlowRenderOrderOffset: 0.01,
hoverGlowRadiusOffset: 0.04,
tintAltitudeOffset: 0.04,
tintColor: 0x0b1830,
tintRenderOrder: 0.2,
landColor: 0x080f1b,
landOpacity: 1.0,
landAltitudeOffset: 0.08,
landRenderOrder: 0.86,
landMaskWidth: 2048,
landMaskHeight: 1024,
};
export const PATHS = {
cablesApi: '/api/v1/visualization/geo/cables',
landingPointsApi: '/api/v1/visualization/geo/landing-points',
computeCentersApi: '/api/v1/visualization/geo/compute-centers',
vesselsApi: '/api/v1/visualization/geo/vessels',
vesselTrackApi: (mmsi) => `/api/v1/visualization/vessels/${encodeURIComponent(mmsi)}/track`,
bgpApi: '/api/v1/visualization/geo/bgp-anomalies',
bgpIncidentsApi: '/api/v1/visualization/geo/bgp-incidents',
bgpCollectorsApi: '/api/v1/visualization/geo/bgp-collectors',
earthSummaryApi: '/api/v1/visualization/geo/summary',
earthClientLogsApi: '/api/v1/system/logs/earth-client',
};
export const VESSEL_CONFIG = {
altitudeOffset: 0.56,
maxRenderedMarkers: 5000,
marker: {
baseScale: 7.5,
baseOpacity: 0.88,
hoverScale: 1.28,
lockedScale: 1.48,
dimmedScale: 0.78,
dimmedOpacity: 0.26,
},
colors: {
cargo: "#4A90D9",
tanker: "#E85D04",
passenger: "#06D6A0",
fishing: "#FFD166",
military: "#73797E",
other: "#9B9B9B",
},
sizeStabilization: {
min: 0.1,
max: 2.4,
},
track: {
altitudeOffset: 0.7,
color: 0x7dd3fc,
opacity: 0.82,
},
};
export const COMPUTE_CENTER_CONFIG = {
altitudeOffset: 0.48,
maxRenderedMarkers: 300,
@@ -233,15 +292,17 @@ export const CABLE_CONFIG = {
renderOrder: 1,
},
landingPoint: {
altitudeOffset: 0.1,
radius: 0.4,
widthSegments: 16,
heightSegments: 16,
baseScale: 2.5,
altitudeOffset: 0.48,
textureSize: 256,
iconAspectRatio: 0.82,
anchorX: 0.52,
anchorY: 0.276,
baseScale: 12,
color: 0xffaa00,
emissive: 0x442200,
emissiveIntensity: 0.5,
opacity: 1.0,
renderOrder: 4.5,
},
landingPointSizeStabilization: {
enabled: true,
@@ -251,7 +312,7 @@ export const CABLE_CONFIG = {
},
landingPointVisual: {
pulseSpeed: 0.003,
dimBrightness: 0.3,
dimBrightness: 0.62,
related: {
emissiveIntensityBase: 0.5,
emissiveIntensityPulse: 0.5,
@@ -261,10 +322,10 @@ export const CABLE_CONFIG = {
scalePulse: 0.3,
},
dimmed: {
colorRGB: { r: 255, g: 170, b: 0 },
emissive: 0x000000,
emissiveIntensity: 0,
opacity: 0.3,
colorRGB: { r: 180, g: 116, b: 28 },
emissive: 0x3a2200,
emissiveIntensity: 0.18,
opacity: 0.78,
},
},
};
@@ -277,9 +338,10 @@ export const CABLE_STATE = {
export const SATELLITE_CONFIG = {
maxCount: -1,
initialLoadCount: 2400,
hydrateFullAfterInitialLoad: true,
initialLoadCount: null,
hydrateFullAfterInitialLoad: false,
trailLength: 10,
trailLineWidth: 3,
displayAltitudeOffset: 8,
frontFacingDotThreshold: 0.015,
overlayRenderOrder: 12,
@@ -392,19 +454,45 @@ export const PREDICTED_ORBIT_CONFIG = {
};
export const GRID_CONFIG = {
latitudeStep: 10,
radiusOffset: 0.14,
color: 0xc0e0ff,
opacity: 0.08,
lineWidth: 1,
renderOrder: 2.05,
latitudeStep: 15,
longitudeStep: 30,
gridStep: 5
segmentStep: 5,
};
export const CLOUD_LAYER_CONFIG = {
radiusOffset: 3,
widthSegments: 64,
heightSegments: 64,
opacity: 0.15,
textureUrl: "./assets/earth_clouds_1024.png",
};
export const STARFIELD_CONFIG = {
count: 8000,
minRadius: 800,
radiusJitter: 200,
color: 0xffffff,
size: 0.5,
};
export const EARTH_MATERIAL_CONFIG = {
// Diffuse color multiplies with texture — pure white = full saturation,
// slightly grey-blue pulls perceived saturation down without a custom shader.
color: 0xcdd8e6,
// Base sphere sits below the country fill and high-res texture overlays.
// Keep it dark so a delayed overlay never flashes or reads as a white layer.
color: 0x010609,
specular: 0x1a2d45,
shininess: 12,
emissive: 0x050a12,
opacity: 0.96,
emissive: 0x010609,
opacity: 1,
textureOverlayAltitudeOffset: 0.1,
textureOverlayOpacity: 0.88,
textureOverlayRenderOrder: 0.96,
textureOverlaySpecular: 0x05080d,
textureOverlayShininess: 4,
// Depth-mask occluder keeps far-side objects hidden behind the earth
occluderRadiusFactor: 0.999,
@@ -418,11 +506,19 @@ export const EARTH_MATERIAL_CONFIG = {
atmosInnerIntensity: 0.18,
// Fresnel atmosphere glow — outer corona
atmosOuterRadiusFactor: 1.016,
atmosOuterRadiusFactor: 1.0025,
atmosOuterSegments: 48,
atmosOuterColor: [0.18, 0.45, 0.9],
atmosOuterRimPower: 5.0,
atmosOuterIntensity: 0.02,
atmosOuterRimPower: 9.0,
atmosOuterIntensity: 0.0025,
// Subtle Fresnel edge cue shown when the high-res texture is hidden or unavailable.
rimGlowColor: [0.42, 0.72, 1.0],
rimGlowRadiusFactor: 1.0035,
rimGlowPower: 3.4,
rimGlowIntensity: 0.24,
rimGlowSegments: 96,
rimGlowRenderOrder: 1.08,
// Texture candidates — tried in order, first success wins
textureUrls: [
@@ -434,12 +530,12 @@ export const EARTH_MATERIAL_CONFIG = {
dayNight: {
enabled: true,
sunDirection: { x: 1, y: 0.2, z: 0.4 },
nightFloor: 0.32,
dayBoost: 0.94,
twilightWidth: 0.24,
nightFloor: 0.24,
dayBoost: 1.12,
twilightWidth: 0.2,
twilightIntensity: 0.14,
twilightColor: 0x4ea0ff,
nightTintColor: 0x0b1830,
nightTintIntensity: 0.05,
nightTintIntensity: 0.08,
},
};

View File

@@ -29,8 +29,15 @@ import {
clearLockedObject,
clearLockedObjectAndInfo,
setCablesEnabled,
setCountryBoundariesEnabled,
setHighResTextureEnabled,
getHighResTextureEnabled,
setAtmosphereCloudsEnabled,
getAtmosphereCloudsEnabled,
setSatellitesEnabled,
getSatellitesEnabled,
setVesselsEnabled,
getVesselsEnabled,
} from "./main.js";
import {
toggleTrails,
@@ -41,11 +48,16 @@ import {
} from "./satellites.js";
import { getShowCables } from "./cables.js";
import { toggleBGP, getShowBGP, getBGPCount } from "./bgp.js";
import { getShowCountryBoundaries } from "./country-boundaries.js";
import {
toggleComputeCenters,
getShowComputeCenters,
getComputeCenterCount,
} from "./compute-centers.js";
import {
getShowVessels,
getVesselCount,
} from "./vessels.js";
import { ensureTVPanelReady, isTVPanelVisible, setTVPanelVisible } from "./tv.js";
import { createHUDPanel } from "./hud-panels.js";
import {
@@ -73,6 +85,7 @@ export let rotationMode = ROTATION_MODE.ROTATE;
let dayNightEnabled = true;
let defaultEarthZoom = CONFIG.defaultViewZoom;
let activeCamera = null;
let settingsApplyPromise = Promise.resolve();
let earthObj = null;
let listeners = [];
@@ -103,6 +116,10 @@ const SETTINGS_SHEET_MAX_SCALE_X = 0.22;
const SETTINGS_SHEET_MAX_SCALE_Y = 0.18;
const EARTH_SETTINGS_STORAGE_KEY = "planet.earth.settings.v2";
const LEGACY_EARTH_SETTINGS_STORAGE_KEY = "planet.earth.settings.v1";
const EARTH_SETTINGS_VERSION = 5;
const GRID_LINES_DEFAULT_VERSION = 3;
const SATELLITE_DISPLAY_DEFAULT_VERSION = 4;
const MEDIA_PANEL_DEFAULT_VERSION = 5;
const DEFAULT_EARTH_ZOOM_STEP = 0.01;
let settingsModalTimer = null;
let settingsSheetAnimation = null;
@@ -119,6 +136,7 @@ let activeMobileDrawerId = null;
let mobileDrawerOpen = false;
let mobileDrawerCard = "layers";
let mobileDrawerHintTimer = null;
let toolbarHubController = null;
const ALLOWED_CRUISE_MODULES = new Set(Object.values(CRUISE_MODULES));
const ALLOWED_SATELLITE_DISPLAY_STYLES = new Set(
Object.values(SATELLITE_DISPLAY_STYLES),
@@ -331,9 +349,31 @@ function getMobileLayerButtons(layerId) {
).filter((button) => button instanceof HTMLButtonElement);
}
function getLayerDisabledState(layerId) {
if (layerId === "trails" && !getSatellitesEnabled()) {
return {
disabled: true,
statusText: "不可用",
tooltip: "卫星关闭时不可用",
};
}
if (layerId === "terrain" && !getHighResTextureEnabled()) {
return {
disabled: true,
statusText: "不可用",
tooltip: "高清材质关闭时不可用",
};
}
return {
disabled: false,
statusText: null,
tooltip: null,
};
}
function syncMobileLayerCards() {
const summary = document.getElementById("mobile-layer-summary");
const definitions = getSortedLayerDefinitions();
const definitions = getDisplayLayerDefinitions();
let activeCount = 0;
definitions.forEach((definition) => {
@@ -342,11 +382,19 @@ function syncMobileLayerCards() {
activeCount += 1;
}
getMobileLayerButtons(definition.id).forEach((button) => {
const disabledState = getLayerDisabledState(definition.id);
button.classList.toggle("is-active", visible);
button.classList.toggle("is-disabled", disabledState.disabled);
button.disabled = disabledState.disabled;
button.setAttribute("aria-checked", visible ? "true" : "false");
if (disabledState.tooltip) {
button.title = disabledState.tooltip;
} else {
button.removeAttribute("title");
}
const status = button.querySelector("[data-mobile-layer-status]");
if (status) {
status.textContent = visible ? "开启" : "关闭";
status.textContent = disabledState.statusText || (visible ? "开启" : "关闭");
}
});
});
@@ -360,7 +408,7 @@ function renderMobileLayerCards() {
const list = document.getElementById("mobile-layer-list");
if (!(list instanceof HTMLElement)) return;
const definitions = getSortedLayerDefinitions();
const definitions = getDisplayLayerDefinitions();
list.innerHTML = definitions
.map((definition) => `
<button
@@ -384,6 +432,7 @@ function renderMobileLayerCards() {
bindListener(button, "click", async (event) => {
const target = event.currentTarget;
if (!(target instanceof HTMLButtonElement)) return;
if (target.disabled || target.classList.contains("is-disabled")) return;
const layerId = target.dataset.mobileLayerButton;
const definition = layerId ? getLayerDefinition(layerId) : null;
if (!definition) return;
@@ -594,6 +643,21 @@ function getSortedLayerDefinitions({ includeUnprioritized = true } = {}) {
.sort(compareLayerDefinitionsByStartupPriority);
}
function getDisplayLayerDefinitions() {
return Array.from(layerRegistry.values()).sort((left, right) => {
const leftOrder = Number.isFinite(left?.displayOrder)
? left.displayOrder
: Number.POSITIVE_INFINITY;
const rightOrder = Number.isFinite(right?.displayOrder)
? right.displayOrder
: Number.POSITIVE_INFINITY;
if (leftOrder !== rightOrder) {
return leftOrder - rightOrder;
}
return String(left?.id || "").localeCompare(String(right?.id || ""));
});
}
function shouldIncludeLayerInStartupLoad(definition) {
if (!Number.isFinite(definition?.startupPriority)) {
return false;
@@ -658,13 +722,22 @@ function getCurrentSharedSettingsSnapshot() {
};
}
function getDefaultLayerVisibilitySnapshot() {
return Object.fromEntries(
getPersistedLayers().map((layer) => [layer.id, Boolean(layer.defaultActive)]),
);
}
function captureEarthSettingsDefaults() {
if (!earthSettingsDefaults) {
const panelVisibility = getCurrentPanelVisibilitySnapshot();
const shared = getCurrentSharedSettingsSnapshot();
earthSettingsDefaults = {
version: 2,
shared,
version: EARTH_SETTINGS_VERSION,
shared: {
...shared,
layerVisibility: getDefaultLayerVisibilitySnapshot(),
},
views: {
desktop: {
panelVisibility: { ...panelVisibility },
@@ -680,7 +753,7 @@ function captureEarthSettingsDefaults() {
function cloneEarthSettings(settings) {
return {
version: 2,
version: EARTH_SETTINGS_VERSION,
shared: {
rotationMode: settings.shared.rotationMode,
cruiseModules: [...(settings.shared.cruiseModules || DEFAULT_CRUISE_MODULES)],
@@ -753,6 +826,13 @@ function normalizeEarthSettings(rawSettings, defaults) {
}
});
if ((rawSettings?.version || 0) < GRID_LINES_DEFAULT_VERSION && inputLayerVisibility.gridLines === true) {
normalizedLayerVisibility.gridLines = defaults.shared.layerVisibility.gridLines;
}
if ((rawSettings?.version || 0) < MEDIA_PANEL_DEFAULT_VERSION) {
normalizedDesktopPanelVisibility["media-panel"] = true;
}
const nextRotationMode =
sharedSettings?.rotationMode === ROTATION_MODE.CRUISE
? ROTATION_MODE.CRUISE
@@ -765,11 +845,17 @@ function normalizeEarthSettings(rawSettings, defaults) {
requestedCruiseModules.filter((moduleId) => ALLOWED_CRUISE_MODULES.has(moduleId)),
),
);
const nextSatelliteDisplayStyle = ALLOWED_SATELLITE_DISPLAY_STYLES.has(
let nextSatelliteDisplayStyle = ALLOWED_SATELLITE_DISPLAY_STYLES.has(
sharedSettings?.satelliteDisplayStyle,
)
? sharedSettings.satelliteDisplayStyle
: defaults.shared.satelliteDisplayStyle;
if (
(rawSettings?.version || 0) < SATELLITE_DISPLAY_DEFAULT_VERSION &&
nextSatelliteDisplayStyle === SATELLITE_DISPLAY_STYLES.SELF_GLOW
) {
nextSatelliteDisplayStyle = defaults.shared.satelliteDisplayStyle;
}
const nextTerrainOpacity = Number.parseFloat(sharedSettings?.terrainOpacity);
const nextDayNightEnabled = typeof sharedSettings?.dayNightEnabled === "boolean"
? sharedSettings.dayNightEnabled
@@ -779,7 +865,7 @@ function normalizeEarthSettings(rawSettings, defaults) {
);
return {
version: 2,
version: EARTH_SETTINGS_VERSION,
shared: {
rotationMode: nextRotationMode,
cruiseModules: nextCruiseModules.length > 0
@@ -805,7 +891,7 @@ function normalizeEarthSettings(rawSettings, defaults) {
}
function getPersistedLayers() {
return getSortedLayerDefinitions().filter((layer) => layer.persist !== false);
return getDisplayLayerDefinitions().filter((layer) => layer.persist !== false);
}
function getLayerDefinition(layerId) {
@@ -849,7 +935,10 @@ function syncEarthSettingsStateFromRuntime() {
const scope = getSettingsViewportScope();
nextSettings.shared = getCurrentSharedSettingsSnapshot();
nextSettings.views[scope].panelVisibility = getCurrentPanelVisibilitySnapshot();
// panelVisibility is maintained in earthSettingsState via setHudPanelVisibility.
// Do not re-snapshot from DOM here: transient hides (e.g. closeTransientMobileOverlays)
// change the DOM without going through setHudPanelVisibility and would corrupt the
// user's persisted preference.
earthSettingsState = nextSettings;
return nextSettings;
}
@@ -1149,6 +1238,7 @@ async function setSatellitesLayerEnabled(button, enabled, { persist = true, sile
} else if (enabled) {
setEarthStatValue("satellite-count", `${getSatelliteCount()}`);
}
syncTrailsAvailability();
syncMobileLayerCards();
if (persist) persistEarthSettings();
return enabled;
@@ -1179,6 +1269,61 @@ function setGridLinesLayerEnabled(button, enabled, { persist = true, silent = fa
return enabled;
}
async function setCountryBoundariesLayerEnabled(button, enabled, { persist = true, silent = false } = {}) {
try {
if (enabled) {
setLayerButtonState(button, {
active: false,
loading: true,
tooltip: "国界加载中...",
});
}
await setCountryBoundariesEnabled(enabled, { suppressStatus: silent });
setLayerButtonState(button, {
active: enabled,
loading: false,
tooltip: enabled ? "隐藏国界" : "显示国界",
});
syncMobileLayerCards();
if (persist) persistEarthSettings();
return enabled;
} catch (error) {
console.error("切换国界显示失败:", error);
setLayerButtonState(button, {
active: false,
loading: false,
tooltip: "显示国界",
});
syncMobileLayerCards();
if (persist) persistEarthSettings();
return false;
}
}
function setHighResTextureLayerEnabled(button, enabled, { persist = true, silent = false } = {}) {
setHighResTextureEnabled(enabled, { suppressStatus: silent });
setLayerButtonState(button, {
active: enabled,
loading: false,
tooltip: enabled ? "隐藏高清材质" : "显示高清材质",
});
syncMobileLayerCards();
if (persist) persistEarthSettings();
return enabled;
}
function setAtmosphereCloudsLayerEnabled(button, enabled, { persist = true, silent = false } = {}) {
setAtmosphereCloudsEnabled(enabled, { suppressStatus: silent });
setLayerButtonState(button, {
active: enabled,
loading: false,
tooltip: enabled ? "隐藏大气云图" : "显示大气云图",
});
syncMobileLayerCards();
if (persist) persistEarthSettings();
return enabled;
}
function setBGPLayerEnabled(button, enabled, { persist = true, silent = false } = {}) {
clearSelectionIfHiding(!enabled);
toggleBGP(enabled);
@@ -1214,11 +1359,46 @@ function setComputeCentersLayerEnabled(button, enabled, { persist = true, silent
return enabled;
}
async function setVesselsLayerEnabled(button, enabled, { persist = true, silent = false } = {}) {
clearSelectionIfHiding(!enabled);
try {
if (enabled) {
setLayerButtonState(button, {
active: false,
loading: true,
tooltip: "船只加载中...",
});
}
await setVesselsEnabled(enabled, { suppressStatus: silent, suppressLoadingUi: silent });
setLayerButtonState(button, {
active: enabled,
loading: false,
tooltip: enabled ? "隐藏船只" : "显示船只",
});
setEarthStatValue("vessel-count", `${getVesselCount()}`);
syncMobileLayerCards();
if (persist) persistEarthSettings();
return enabled;
} catch (error) {
console.error("切换船只显示失败:", error);
setLayerButtonState(button, {
active: false,
loading: false,
tooltip: "显示船只",
});
syncMobileLayerCards();
if (persist) persistEarthSettings();
return false;
}
}
function setTrailsLayerEnabled(button, enabled, { persist = true, silent = false } = {}) {
toggleTrails(enabled);
const disabledState = getLayerDisabledState("trails");
setLayerButtonState(button, {
active: enabled,
tooltip: enabled ? "隐藏轨迹" : "显示轨迹",
disabled: disabledState.disabled,
tooltip: disabledState.tooltip || (enabled ? "隐藏轨迹" : "显示轨迹"),
});
syncMobileLayerCards();
if (persist) persistEarthSettings();
@@ -1228,6 +1408,17 @@ function setTrailsLayerEnabled(button, enabled, { persist = true, silent = false
return enabled;
}
function syncTrailsAvailability() {
const trailsEnabled = getShowTrails();
const disabledState = getLayerDisabledState("trails");
setLayerButtonState(getLayerButton("trails"), {
active: trailsEnabled,
disabled: disabledState.disabled,
tooltip: disabledState.tooltip || (trailsEnabled ? "隐藏轨迹" : "显示轨迹"),
});
syncMobileLayerCards();
}
async function setCablesLayerEnabled(button, enabled, { persist = true, silent = false } = {}) {
clearSelectionIfHiding(!enabled);
try {
@@ -1253,23 +1444,6 @@ async function applyLayerVisibilitySettings(layerVisibility = {}, options = {})
function getBuiltinLayerDefinitions() {
return [
{
id: "terrain",
buttonId: "toggle-terrain",
icon: "landscape",
label: "地形",
meta: "Terrain",
keywords: "地形 terrain",
defaultActive: false,
startupPriority: null,
startupMode: "visible",
startupLabel: "地形",
startupMessage: "正在渲染地形...",
statusTarget: "terrain-status",
getVisible: () => showTerrain,
setVisible: (visible, options = {}) =>
setTerrainEnabled(getLayerButton("terrain"), visible, options),
},
{
id: "gridLines",
buttonId: "toggle-grid-lines",
@@ -1277,8 +1451,9 @@ function getBuiltinLayerDefinitions() {
label: "经纬线",
meta: "Graticule",
keywords: "经纬线 graticule 经纬 latitude longitude",
defaultActive: true,
startupPriority: null,
defaultActive: false,
displayOrder: 100,
startupPriority: 10,
startupMode: "visible",
startupLabel: "经纬线",
startupMessage: "",
@@ -1287,36 +1462,55 @@ function getBuiltinLayerDefinitions() {
setGridLinesLayerEnabled(getLayerButton("gridLines"), visible, options),
},
{
id: "satellites",
buttonId: "toggle-satellites",
icon: "satellite_alt",
label: "卫星",
meta: "Satellites",
keywords: "卫星 satellites",
defaultActive: false,
startupPriority: 30,
startupMode: "visible",
startupLabel: "卫星",
startupMessage: "正在加载卫星...",
getVisible: () => getSatellitesEnabled(),
id: "countryBoundaries",
buttonId: "toggle-country-boundaries",
icon: "public",
label: "国界",
meta: "Country Borders",
keywords: "国界 国家 borders countries boundary",
defaultActive: true,
displayOrder: 90,
startupPriority: 20,
startupMode: "preload",
startupLabel: "海陆基座",
startupMessage: "正在加载海陆基座...",
getVisible: () => getShowCountryBoundaries(),
setVisible: (visible, options = {}) =>
setSatellitesLayerEnabled(getLayerButton("satellites"), visible, options),
setCountryBoundariesLayerEnabled(getLayerButton("countryBoundaries"), visible, options),
},
{
id: "trails",
buttonId: "toggle-trails",
icon: "timeline",
label: "轨迹",
meta: "Trails",
keywords: "轨迹 trails",
id: "earthHighResTexture",
buttonId: "toggle-earth-high-res-texture",
icon: "globe",
label: "高清材质",
meta: "High-Res Texture",
keywords: "高清 材质 纹理 texture hd 地表 earth",
defaultActive: true,
startupPriority: null,
displayOrder: 70,
startupPriority: 30,
startupMode: "visible",
startupLabel: "轨迹",
startupMessage: "",
getVisible: () => getShowTrails(),
startupLabel: "高清材质",
startupMessage: "正在启用高清材质...",
getVisible: () => getHighResTextureEnabled(),
setVisible: (visible, options = {}) =>
setTrailsLayerEnabled(getLayerButton("trails"), visible, options),
setHighResTextureLayerEnabled(getLayerButton("earthHighResTexture"), visible, options),
},
{
id: "atmosphereClouds",
buttonId: "toggle-atmosphere-clouds",
icon: "cloud",
label: "大气云图",
meta: "Cloud Layer",
keywords: "大气 云图 云层 clouds atmosphere",
defaultActive: true,
displayOrder: 80,
startupPriority: 40,
startupMode: "visible",
startupLabel: "大气云图",
startupMessage: "",
getVisible: () => getAtmosphereCloudsEnabled(),
setVisible: (visible, options = {}) =>
setAtmosphereCloudsLayerEnabled(getLayerButton("atmosphereClouds"), visible, options),
},
{
id: "cables",
@@ -1326,7 +1520,8 @@ function getBuiltinLayerDefinitions() {
meta: "Subsea Cables",
keywords: "海缆 subsea cables",
defaultActive: true,
startupPriority: 20,
displayOrder: 10,
startupPriority: 50,
startupMode: "visible",
startupLabel: "海缆",
startupMessage: {
@@ -1345,7 +1540,8 @@ function getBuiltinLayerDefinitions() {
meta: "Compute Centers",
keywords: "算力中心 compute centers gpu 超算",
defaultActive: true,
startupPriority: 35,
displayOrder: 40,
startupPriority: 60,
startupMode: "preload",
startupLabel: "算力中心",
startupMessage: "正在加载算力中心...",
@@ -1361,7 +1557,8 @@ function getBuiltinLayerDefinitions() {
meta: "Routing Signals",
keywords: "bgp观测 routing signals",
defaultActive: true,
startupPriority: 40,
displayOrder: 50,
startupPriority: 70,
startupMode: "preload",
startupLabel: "BGP态势",
startupMessage: "正在加载BGP态势...",
@@ -1369,6 +1566,75 @@ function getBuiltinLayerDefinitions() {
setVisible: (visible, options = {}) =>
setBGPLayerEnabled(getLayerButton("bgp"), visible, options),
},
{
id: "vessels",
buttonId: "toggle-vessels",
icon: "directions_boat",
label: "船只",
meta: "AIS Vessels",
keywords: "船只 船舶 ais vessels ships maritime",
defaultActive: false,
displayOrder: 45,
startupPriority: 65,
startupMode: "visible",
startupLabel: "船只",
startupMessage: "正在加载船只...",
getVisible: () => getVesselsEnabled(),
setVisible: (visible, options = {}) =>
setVesselsLayerEnabled(getLayerButton("vessels"), visible, options),
},
{
id: "satellites",
buttonId: "toggle-satellites",
icon: "satellite_alt",
label: "卫星",
meta: "Satellites",
keywords: "卫星 satellites",
defaultActive: false,
displayOrder: 30,
startupPriority: 80,
startupMode: "visible",
startupLabel: "卫星",
startupMessage: "正在加载卫星...",
getVisible: () => getSatellitesEnabled(),
setVisible: (visible, options = {}) =>
setSatellitesLayerEnabled(getLayerButton("satellites"), visible, options),
},
{
id: "trails",
buttonId: "toggle-trails",
icon: "timeline",
label: "轨迹",
meta: "Trails",
keywords: "轨迹 trails",
defaultActive: true,
displayOrder: 20,
startupPriority: null,
startupMode: "visible",
startupLabel: "轨迹",
startupMessage: "",
getVisible: () => getShowTrails(),
setVisible: (visible, options = {}) =>
setTrailsLayerEnabled(getLayerButton("trails"), visible, options),
},
{
id: "terrain",
buttonId: "toggle-terrain",
icon: "landscape",
label: "地形",
meta: "Terrain",
keywords: "地形 terrain",
defaultActive: false,
displayOrder: 60,
startupPriority: null,
startupMode: "visible",
startupLabel: "地形",
startupMessage: "正在渲染地形...",
statusTarget: "terrain-status",
getVisible: () => showTerrain,
setVisible: (visible, options = {}) =>
setTerrainEnabled(getLayerButton("terrain"), visible, options),
},
];
}
@@ -1430,6 +1696,7 @@ function syncLayerRowDefinition(definition, { appendIfMissing = false } = {}) {
function registerLayerDefinition(definition, options = {}) {
const normalizedDefinition = {
persist: true,
displayOrder: null,
startupPriority: null,
startupMode: "visible",
startupLabel: "",
@@ -1818,6 +2085,31 @@ function applyDayNightEnabled(enabled, { persist = true } = {}) {
if (persist) persistEarthSettings();
}
export function setDayNightEnabledExternal(enabled, { persist = true } = {}) {
applyDayNightEnabled(enabled, { persist });
}
export function getDayNightEnabled() {
return dayNightEnabled;
}
export function setTerrainLayerInteractable(enabled) {
const button = getLayerButton("terrain");
setLayerButtonState(button, {
disabled: !enabled,
tooltip: enabled ? null : "高清材质关闭时不可用",
});
syncMobileLayerCards();
}
export function setDayNightInteractable(enabled) {
document.querySelectorAll("#toggle-daynight, [data-daynight-toggle]").forEach((input) => {
input.disabled = !enabled;
const label = input.closest("label");
if (label) label.classList.toggle("is-disabled", !enabled);
});
}
function setupSettingsControls() {
const settingsTrigger = document.getElementById("settings-trigger");
const settingsClose = document.getElementById("settings-close");
@@ -1960,7 +2252,7 @@ function setupSettingsControls() {
});
captureEarthSettingsDefaults();
applyEarthSettings(loadEarthSettings());
settingsApplyPromise = applyEarthSettings(loadEarthSettings());
syncAllHudPanelToggles();
syncRotationModeButtons();
syncCruiseModuleControls();
@@ -2283,7 +2575,7 @@ function resetCleanup() {
listeners = [];
}
export function setupControls(camera, renderer, scene, earth) {
export async function setupControls(camera, renderer, scene, earth) {
resetCleanup();
activeCamera = camera;
earthObj = earth;
@@ -2292,6 +2584,8 @@ export function setupControls(camera, renderer, scene, earth) {
setupWheelZoom(camera, renderer);
setupRotateControls(camera, earth);
setupTerrainControls();
await settingsApplyPromise;
syncTrailsAvailability();
setupLiquidGlassInteractions();
setupToolbarHubCluster();
setupKeyboardControls();
@@ -2622,7 +2916,11 @@ function bindLayerButton(row, definition) {
if (button.dataset.layerBound === "true") return;
bindListener(button, "click", async function () {
if (this.classList.contains("is-loading")) {
if (
this.disabled ||
this.classList.contains("is-loading") ||
this.classList.contains("is-disabled")
) {
return;
}
await definition.setVisible(!definition.getVisible());
@@ -2638,6 +2936,7 @@ export function registerLayer({
keywords = "",
defaultActive = false,
persist = true,
displayOrder = null,
startupPriority = null,
startupMode = "visible",
startupLabel = "",
@@ -2658,6 +2957,7 @@ export function registerLayer({
keywords,
defaultActive,
persist,
displayOrder,
startupPriority,
startupMode,
startupLabel,
@@ -2821,6 +3121,11 @@ function setupKeyboardControls() {
return;
}
if (toolbarHubController?.isOpen?.()) {
toolbarHubController.close();
return;
}
clearLockedObjectAndInfo();
});
}
@@ -3011,6 +3316,12 @@ function setupToolbarHubCluster() {
scheduleExpandedToolbarBoundsRefresh();
};
const closePinnedToolbar = () => {
hubPinnedOpen = false;
cancelCollapse();
setExpanded(false);
};
const scheduleCollapse = () => {
if (hubPinnedOpen) return;
if (collapseTimer) clearTimeout(collapseTimer);
@@ -3033,6 +3344,9 @@ function setupToolbarHubCluster() {
cancelAnimationFrame(refreshBoundsFrameId);
refreshBoundsFrameId = 0;
}
if (toolbarHubController?.cluster === cluster) {
toolbarHubController = null;
}
});
// Start collapsed — hub acts as the hover target to reveal the arc
@@ -3055,13 +3369,12 @@ function setupToolbarHubCluster() {
event.preventDefault();
event.stopPropagation();
cancelCollapse();
if (isMobileLayout()) {
hubPinnedOpen = !cluster.classList.contains("is-expanded");
setExpanded(hubPinnedOpen);
return;
if (hubPinnedOpen) {
closePinnedToolbar();
} else {
hubPinnedOpen = true;
setExpanded(true);
}
hubPinnedOpen = !cluster.classList.contains("is-expanded");
setExpanded(hubPinnedOpen);
});
const HOVER_PADDING_PX = 12;
@@ -3174,9 +3487,14 @@ function setupToolbarHubCluster() {
if (!hubPinnedOpen) return;
if (!(event.target instanceof Element)) return;
if (event.target.closest("#toolbar-cluster")) return;
hubPinnedOpen = false;
setExpanded(false);
closePinnedToolbar();
});
toolbarHubController = {
cluster,
isOpen: () => hubPinnedOpen || cluster.classList.contains("is-expanded"),
close: closePinnedToolbar,
};
}
export function teardownControls() {

View File

@@ -0,0 +1,500 @@
import * as THREE from "three";
import { CONFIG, COUNTRY_BOUNDARY_CONFIG } from "./constants.js";
import { latLonToVector3 } from "./utils.js";
// ─── Module state ──────────────────────────────────────────────────────────────
let _earthObj = null;
let _features = [];
let _landMesh = null;
let _tintMesh = null;
let _boundaryLines = null;
let _hoverGlowLines = null;
let _hoverLines = null;
let _hoveredFeature = null;
let _hoveredGroupKey = null;
let _visible = false;
let _landFillEnabled = true;
let _landFillSuppressed = false;
let _tintEnabled = false;
let _loaded = false;
let _loadPromise = null;
const OCEAN_HEX = 0x010609;
// ─── Equirectangular land/ocean fill texture ──────────────────────────────────
function hexToStyle(hex) {
return `#${hex.toString(16).padStart(6, "0")}`;
}
function hexToRgb(hex) {
return [
(hex >> 16) & 255,
(hex >> 8) & 255,
hex & 255,
];
}
function buildLandTexture(features) {
const width = COUNTRY_BOUNDARY_CONFIG.landMaskWidth;
const height = COUNTRY_BOUNDARY_CONFIG.landMaskHeight;
const canvas = document.createElement("canvas");
canvas.width = width;
canvas.height = height;
const ctx = canvas.getContext("2d");
const oceanRgb = hexToRgb(OCEAN_HEX);
if (!ctx) {
const oceanData = new Uint8Array(width * height * 4);
for (let i = 0; i < oceanData.length; i += 4) {
oceanData[i] = oceanRgb[0];
oceanData[i + 1] = oceanRgb[1];
oceanData[i + 2] = oceanRgb[2];
oceanData[i + 3] = 255;
}
const fallbackTexture = new THREE.DataTexture(
oceanData,
width,
height,
THREE.RGBAFormat,
);
fallbackTexture.needsUpdate = true;
return fallbackTexture;
}
// Ocean background
ctx.fillStyle = hexToStyle(OCEAN_HEX);
ctx.fillRect(0, 0, width, height);
// Land polygons using evenodd fill rule so holes (lakes, islands) work correctly
ctx.fillStyle = hexToStyle(COUNTRY_BOUNDARY_CONFIG.landColor);
for (const feat of features) {
const geom = feat.geometry;
if (!geom) continue;
const polys =
geom.type === "Polygon" ? [geom.coordinates] :
geom.type === "MultiPolygon" ? geom.coordinates : null;
if (!polys) continue;
for (const rings of polys) {
ctx.beginPath();
for (const ring of rings) {
for (let i = 0; i < ring.length; i++) {
// equirectangular: x = (lon+180)/360*width, y = (90-lat)/180*height
const px = ((ring[i][0] + 180) / 360) * width;
const py = ((90 - ring[i][1]) / 180) * height;
i === 0 ? ctx.moveTo(px, py) : ctx.lineTo(px, py);
}
ctx.closePath();
}
ctx.fill("evenodd");
}
}
const imageData = ctx.getImageData(0, 0, width, height);
const tex = new THREE.DataTexture(
new Uint8Array(imageData.data),
width,
height,
THREE.RGBAFormat,
);
tex.wrapS = THREE.ClampToEdgeWrapping;
tex.wrapT = THREE.ClampToEdgeWrapping;
tex.minFilter = THREE.LinearFilter;
tex.magFilter = THREE.LinearFilter;
tex.generateMipmaps = false;
tex.flipY = true;
tex.needsUpdate = true;
return tex;
}
// ─── Sphere mesh helpers ───────────────────────────────────────────────────────
function makeLandMesh(tex) {
const r = CONFIG.earthRadius + COUNTRY_BOUNDARY_CONFIG.landAltitudeOffset;
const geo = new THREE.SphereGeometry(r, 128, 64);
const mat = new THREE.MeshBasicMaterial({
color: 0xffffff,
map: tex,
transparent: COUNTRY_BOUNDARY_CONFIG.landOpacity < 1,
opacity: COUNTRY_BOUNDARY_CONFIG.landOpacity,
depthTest: true,
depthWrite: false,
});
const mesh = new THREE.Mesh(geo, mat);
mesh.name = "country-land-ocean";
mesh.renderOrder = COUNTRY_BOUNDARY_CONFIG.landRenderOrder;
mesh.visible = false;
mesh.raycast = () => {};
return mesh;
}
function makeTintMesh() {
const r = CONFIG.earthRadius + COUNTRY_BOUNDARY_CONFIG.tintAltitudeOffset;
const geo = new THREE.SphereGeometry(r, 64, 32);
const mat = new THREE.MeshBasicMaterial({ color: COUNTRY_BOUNDARY_CONFIG.tintColor, depthWrite: false });
const mesh = new THREE.Mesh(geo, mat);
mesh.name = "country-tint";
mesh.renderOrder = COUNTRY_BOUNDARY_CONFIG.tintRenderOrder;
mesh.visible = false;
mesh.raycast = () => {};
return mesh;
}
// ─── Boundary line geometry ────────────────────────────────────────────────────
function ringToSegments(ring, radius, out) {
const n = ring.length;
if (n < 2) return;
for (let i = 0; i < n - 1; i++) {
out.push(latLonToVector3(ring[i][1], ring[i][0], radius));
out.push(latLonToVector3(ring[i+1][1], ring[i+1][0], radius));
}
}
function featureToSegments(geom, radius) {
const pts = [];
if (!geom) return pts;
if (geom.type === "Polygon") {
geom.coordinates.forEach(ring => ringToSegments(ring, radius, pts));
} else if (geom.type === "MultiPolygon") {
geom.coordinates.forEach(poly => poly.forEach(ring => ringToSegments(ring, radius, pts)));
}
return pts;
}
function buildBoundaryLines(features) {
const r = CONFIG.earthRadius + COUNTRY_BOUNDARY_CONFIG.lineAltitudeOffset;
const mat = new THREE.LineBasicMaterial({
color: COUNTRY_BOUNDARY_CONFIG.lineColor,
transparent: true,
opacity: COUNTRY_BOUNDARY_CONFIG.lineOpacity,
depthTest: true,
depthWrite: false,
});
const all = [];
for (const feat of features) {
const pts = featureToSegments(feat.geometry, r);
all.push(...pts);
}
const geo = all.length > 0
? new THREE.BufferGeometry().setFromPoints(all)
: new THREE.BufferGeometry();
const lines = new THREE.LineSegments(geo, mat);
lines.name = "country-boundary-all";
lines.renderOrder = COUNTRY_BOUNDARY_CONFIG.lineRenderOrder;
lines.visible = false;
lines.raycast = () => {};
return lines;
}
function buildHoverLines() {
const mat = new THREE.LineBasicMaterial({
color: COUNTRY_BOUNDARY_CONFIG.hoverLineColor,
transparent: COUNTRY_BOUNDARY_CONFIG.hoverLineOpacity < 1,
opacity: COUNTRY_BOUNDARY_CONFIG.hoverLineOpacity,
depthTest: false,
depthWrite: false,
});
const lines = new THREE.LineSegments(new THREE.BufferGeometry(), mat);
lines.name = "country-hover";
lines.renderOrder = COUNTRY_BOUNDARY_CONFIG.hoverLineRenderOrder;
lines.visible = false;
lines.raycast = () => {};
return lines;
}
function buildHoverGlowLines() {
const mat = new THREE.LineBasicMaterial({
color: COUNTRY_BOUNDARY_CONFIG.hoverLineColor,
transparent: true,
opacity: COUNTRY_BOUNDARY_CONFIG.hoverGlowOpacity,
depthTest: false,
depthWrite: false,
blending: THREE.AdditiveBlending,
linewidth: COUNTRY_BOUNDARY_CONFIG.hoverGlowLineWidth,
});
const lines = new THREE.LineSegments(new THREE.BufferGeometry(), mat);
lines.name = "country-hover-glow";
lines.renderOrder =
COUNTRY_BOUNDARY_CONFIG.hoverLineRenderOrder -
COUNTRY_BOUNDARY_CONFIG.hoverGlowRenderOrderOffset;
lines.visible = false;
lines.raycast = () => {};
return lines;
}
function setBoundaryLinesDimmed(dimmed) {
if (!_boundaryLines?.material) return;
_boundaryLines.material.opacity = dimmed
? COUNTRY_BOUNDARY_CONFIG.dimmedLineOpacity
: COUNTRY_BOUNDARY_CONFIG.lineOpacity;
_boundaryLines.material.needsUpdate = true;
}
function clearHoverLineGeometries() {
if (_hoverGlowLines) _hoverGlowLines.geometry.setFromPoints([]);
if (_hoverLines) _hoverLines.geometry.setFromPoints([]);
}
function featureListToSegments(features, radius) {
return features.flatMap(f => featureToSegments(f.geometry, radius));
}
// ─── Point-in-polygon (lat/lon space) ─────────────────────────────────────────
function pointInRing(lat, lon, ring) {
let inside = false;
const n = ring.length;
for (let i = 0, j = n - 1; i < n; j = i++) {
const xi = ring[i][0], yi = ring[i][1];
const xj = ring[j][0], yj = ring[j][1];
if ((yi > lat) !== (yj > lat) && lon < (xj - xi) * (lat - yi) / (yj - yi) + xi) {
inside = !inside;
}
}
return inside;
}
function featureContains(lat, lon, feat) {
const geom = feat.geometry;
if (!geom) return false;
if (geom.type === "Polygon") {
if (!pointInRing(lat, lon, geom.coordinates[0])) return false;
return geom.coordinates.slice(1).every(h => !pointInRing(lat, lon, h));
}
if (geom.type === "MultiPolygon") {
return geom.coordinates.some(poly =>
pointInRing(lat, lon, poly[0]) &&
poly.slice(1).every(h => !pointInRing(lat, lon, h))
);
}
return false;
}
function makeCountryInfo(feat) {
if (!feat) return null;
const p = feat.properties || {};
return {
name: p.NAME_EN || p.NAME || p.ADMIN || "",
nameZh: p.NAME_ZH || null,
isoA3: p.ISO_A3 || p.ADM0_A3 || null,
isoA2: p.ISO_A2 || null,
continent: p.CONTINENT || null,
};
}
function getCountryHighlightGroupKey(feat) {
const p = feat?.properties || {};
const isoA3 = p.ISO_A3 || p.ADM0_A3 || "";
if (isoA3 === "CHN" || isoA3 === "TWN") {
return "CHN_TWN";
}
return isoA3 || p.ISO_A2 || p.NAME_EN || p.NAME || p.ADMIN || null;
}
function getHighlightFeatures(feat) {
const groupKey = getCountryHighlightGroupKey(feat);
if (!groupKey) return feat ? [feat] : [];
return _features.filter(f => getCountryHighlightGroupKey(f) === groupKey);
}
// ─── Public API ────────────────────────────────────────────────────────────────
/** Called during init (before data load). Creates the placeholder tint sphere. */
export function createCountryBoundaryLayer(earthObj) {
_earthObj = earthObj;
_tintMesh = makeTintMesh();
_earthObj.add(_tintMesh);
}
/** Fetch GeoJSON, build meshes. Idempotent; safe to call multiple times. */
export async function loadCountryBoundaries() {
if (_loaded) return _features.length;
if (_loadPromise) return _loadPromise;
_loadPromise = (async () => {
const resp = await fetch(COUNTRY_BOUNDARY_CONFIG.dataPath);
if (!resp.ok) throw new Error(`国界数据加载失败 HTTP ${resp.status}`);
const geojson = await resp.json();
_features = (geojson.features || []).filter(f => f.geometry);
const tex = buildLandTexture(_features);
_landMesh = makeLandMesh(tex);
_earthObj.add(_landMesh);
_boundaryLines = buildBoundaryLines(_features);
_earthObj.add(_boundaryLines);
_hoverGlowLines = buildHoverGlowLines();
_earthObj.add(_hoverGlowLines);
_hoverLines = buildHoverLines();
_earthObj.add(_hoverLines);
_loaded = true;
return _features.length;
})();
return _loadPromise;
}
/** Load if not yet loaded, then return feature count. */
export async function ensureCountryBoundariesReady() {
if (!_loaded) await loadCountryBoundaries();
return _features.length;
}
/**
* Show or hide the country boundary lines.
* The land/ocean fill is the base earth map and stays independent from this
* line visibility switch.
* @param {boolean} visible
* @param {{ showTint?: boolean, showLandFill?: boolean, suppressLandFill?: boolean }} [opts]
* showLandFill whether to show the base land/ocean fill.
* Defaults to the current stored value so callers that only
* care about visibility don't need to repeat it.
* suppressLandFill temporarily keep the fill below the high-res texture
* without changing the layer's own fill state.
*/
export function toggleCountryBoundaries(
visible,
{ showTint = false, showLandFill = null, suppressLandFill = null } = {},
) {
_visible = Boolean(visible);
if (showLandFill !== null) _landFillEnabled = Boolean(showLandFill);
if (suppressLandFill !== null) _landFillSuppressed = Boolean(suppressLandFill);
if (_landMesh) {
_landMesh.visible = _landFillEnabled && !_landFillSuppressed;
}
if (_boundaryLines) _boundaryLines.visible = _visible;
if (_hoverGlowLines) _hoverGlowLines.visible = _visible;
if (_hoverLines) _hoverLines.visible = _visible;
if (!_visible) {
_hoveredFeature = null;
_hoveredGroupKey = null;
setBoundaryLinesDimmed(false);
clearHoverLineGeometries();
}
if (_tintMesh) _tintMesh.visible = _visible && showTint && _tintEnabled;
}
/**
* Show or hide the land/ocean canvas fill independently of boundary lines.
*/
export function setLandFillEnabled(enabled) {
_landFillEnabled = Boolean(enabled);
if (_landMesh) _landMesh.visible = _landFillEnabled && !_landFillSuppressed;
}
export function setLandFillSuppressed(enabled) {
_landFillSuppressed = Boolean(enabled);
if (_landMesh) _landMesh.visible = _landFillEnabled && !_landFillSuppressed;
}
/**
* Enable / disable the solid dark tint overlay (used when high-res texture is off).
*/
export function setSurfaceTintEnabled(enabled) {
_tintEnabled = Boolean(enabled);
if (_tintMesh) _tintMesh.visible = _visible && _tintEnabled;
}
export function getShowCountryBoundaries() {
return _visible;
}
/** Clear the hover highlight without hiding the full layer. */
export function clearCountryBoundaryHover() {
if (!_hoveredFeature) return;
_hoveredFeature = null;
_hoveredGroupKey = null;
setBoundaryLinesDimmed(false);
clearHoverLineGeometries();
}
/**
* Update hover highlight for the given lat/lon coords.
* Returns a country-info object when hovering over land, or null over ocean.
*/
export function updateCountryBoundaryHover(coords) {
if (!_loaded || !_visible) return null;
const { lat, lon } = coords;
const found = _features.find(f => featureContains(lat, lon, f)) || null;
const groupKey = getCountryHighlightGroupKey(found);
if (found !== _hoveredFeature || groupKey !== _hoveredGroupKey) {
_hoveredFeature = found;
_hoveredGroupKey = groupKey;
if (_hoverLines) {
if (!found) {
setBoundaryLinesDimmed(false);
clearHoverLineGeometries();
} else {
setBoundaryLinesDimmed(true);
const highlightFeatures = getHighlightFeatures(found);
const coreRadius = CONFIG.earthRadius + COUNTRY_BOUNDARY_CONFIG.hoverAltitudeOffset;
const glowRadius = coreRadius + COUNTRY_BOUNDARY_CONFIG.hoverGlowRadiusOffset;
if (_hoverGlowLines) {
const glowPts = featureListToSegments(highlightFeatures, glowRadius);
_hoverGlowLines.geometry.setFromPoints(glowPts);
}
const corePts = featureListToSegments(highlightFeatures, coreRadius);
_hoverLines.geometry.setFromPoints(corePts);
}
}
}
return found ? makeCountryInfo(found) : null;
}
/** Dispose all Three.js objects and reset state. */
export function clearCountryBoundaryData() {
_hoveredFeature = null;
_hoveredGroupKey = null;
function disposeObj(obj) {
if (!obj) return;
if (_earthObj) _earthObj.remove(obj);
obj.geometry?.dispose();
if (obj.material) {
if (obj.material.map) obj.material.map.dispose();
obj.material.dispose();
}
}
disposeObj(_hoverLines);
disposeObj(_hoverGlowLines);
disposeObj(_boundaryLines);
disposeObj(_landMesh);
disposeObj(_tintMesh);
_hoverLines = null;
_hoverGlowLines = null;
_boundaryLines = null;
_landMesh = null;
_tintMesh = null;
_features = [];
_loaded = false;
_loadPromise = null;
_visible = false;
_landFillEnabled = true;
_landFillSuppressed = false;
_tintEnabled = false;
}
export function getCountryBoundaryLegendItems() {
return [
{ color: hexToStyle(COUNTRY_BOUNDARY_CONFIG.lineColor), label: "国界线" },
{ color: hexToStyle(COUNTRY_BOUNDARY_CONFIG.landColor), label: "陆地填色" },
{ color: hexToStyle(OCEAN_HEX), label: "海洋填色" },
];
}

View File

@@ -1,18 +1,32 @@
// earth.js - 3D Earth creation module
import * as THREE from 'three';
import { CONFIG, EARTH_CONFIG, EARTH_MATERIAL_CONFIG, TERRAIN_CONFIG } from './constants.js';
import {
CLOUD_LAYER_CONFIG,
CONFIG,
EARTH_CONFIG,
EARTH_MATERIAL_CONFIG,
GRID_CONFIG,
STARFIELD_CONFIG,
TERRAIN_CONFIG,
} from './constants.js';
import { latLonToVector3 } from './utils.js';
export let earth = null;
export let clouds = null;
export let terrain = null;
let showGridLines = true;
let showGridLines = false;
let showClouds = true;
const textureLoader = new THREE.TextureLoader();
let _earthMaterial = null;
let _earthShader = null;
let _earthTextureOverlay = null;
let _earthTextureOverlayMaterial = null;
let _earthShaders = [];
let _dayNightEnabled = true;
let _loadedTexture = null;
let _textureVisible = true;
let _earthRimGlow = null;
const _earthSunDirection = new THREE.Vector3(
EARTH_MATERIAL_CONFIG.dayNight.sunDirection.x,
EARTH_MATERIAL_CONFIG.dayNight.sunDirection.y,
@@ -26,7 +40,7 @@ function applyEarthDayNightShader(material) {
const nightTintColor = new THREE.Color(EARTH_MATERIAL_CONFIG.dayNight.nightTintColor);
material.onBeforeCompile = (shader) => {
_earthShader = shader;
_earthShaders.push(shader);
shader.uniforms.uSunDirectionWorld = { value: _earthSunDirection.clone() };
shader.uniforms.uNightFloor = { value: EARTH_MATERIAL_CONFIG.dayNight.nightFloor };
shader.uniforms.uDayBoost = { value: EARTH_MATERIAL_CONFIG.dayNight.dayBoost };
@@ -79,6 +93,7 @@ uniform float uDayNightEnabled;`,
dnLight *= mix(uNightFloor, uDayBoost, daylight);
dnLight += uTwilightColor * twilight * uTwilightIntensity;
dnLight += uNightTintColor * (1.0 - daylight) * uNightTintIntensity;
dnLight = dnLight / (vec3(1.0) + max(dnLight - vec3(0.68), vec3(0.0)) * 0.86);
// dn=0: emissive base (from material, set in JS) * camera-facing boost → always readable
// dn=1: full day/night solar lighting
@@ -94,6 +109,7 @@ uniform float uDayNightEnabled;`,
}
export function createEarth(scene) {
_earthShaders = [];
const geometry = new THREE.SphereGeometry(CONFIG.earthRadius, 128, 128);
const C = EARTH_MATERIAL_CONFIG;
@@ -103,7 +119,7 @@ export function createEarth(scene) {
specular: C.specular,
shininess: C.shininess,
emissive: C.emissive,
transparent: true,
transparent: C.opacity < 1,
opacity: C.opacity,
side: THREE.FrontSide,
depthWrite: true,
@@ -117,6 +133,31 @@ export function createEarth(scene) {
earth.rotation.x = EARTH_CONFIG.tiltRad;
scene.add(earth);
const textureOverlayGeometry = new THREE.SphereGeometry(
CONFIG.earthRadius + C.textureOverlayAltitudeOffset,
128,
128,
);
_earthTextureOverlayMaterial = new THREE.MeshPhongMaterial({
color: 0xffffff,
specular: C.textureOverlaySpecular,
shininess: C.textureOverlayShininess,
transparent: true,
opacity: C.textureOverlayOpacity,
side: THREE.FrontSide,
depthWrite: false,
depthTest: true,
});
applyEarthDayNightShader(_earthTextureOverlayMaterial);
_earthTextureOverlay = new THREE.Mesh(
textureOverlayGeometry,
_earthTextureOverlayMaterial,
);
_earthTextureOverlay.name = "earth-high-res-texture-overlay";
_earthTextureOverlay.renderOrder = C.textureOverlayRenderOrder;
_earthTextureOverlay.visible = false;
earth.add(_earthTextureOverlay);
// Depth-mask occluder — invisible sphere slightly inside the earth,
// writes to the depth buffer so far-side cables/satellites are occluded.
const occluderGeometry = new THREE.SphereGeometry(
@@ -132,7 +173,9 @@ export function createEarth(scene) {
occluder.renderOrder = -1;
earth.add(occluder);
// Shared Fresnel vertex shader for both atmosphere layers
// Keep the original atmosphere shells on the legacy camera-facing shader so
// they stay as a soft edge cue instead of becoming a visible transparent hull
// at close zoom levels.
const ATMOS_VERTEX_SHADER = `
varying vec3 vNormal;
void main() {
@@ -141,6 +184,17 @@ export function createEarth(scene) {
}
`;
const RIM_VERTEX_SHADER = `
varying vec3 vNormal;
varying vec3 vViewDirection;
void main() {
vNormal = normalize(normalMatrix * normal);
vec4 mvPosition = modelViewMatrix * vec4(position, 1.0);
vViewDirection = normalize(-mvPosition.xyz);
gl_Position = projectionMatrix * mvPosition;
}
`;
// Fresnel atmosphere — inner rim
const [ir, ig, ib] = C.atmosInnerColor;
const atmosInnerGeo = new THREE.SphereGeometry(
@@ -193,15 +247,52 @@ export function createEarth(scene) {
atmosOuter.renderOrder = 1;
earth.add(atmosOuter);
// Fresnel rim cue: an outer shell keeps the edge tied to the globe while bypassing
// the darker fill layers that can otherwise hide a same-radius glow. Unlike the
// legacy atmosphere shells, this one uses the real view direction so its highlight
// stays attached to the visible globe edge while zooming.
const [rr, rg, rb] = C.rimGlowColor;
const rimGlowGeo = new THREE.SphereGeometry(
CONFIG.earthRadius * C.rimGlowRadiusFactor,
C.rimGlowSegments,
C.rimGlowSegments,
);
const rimGlowMat = new THREE.ShaderMaterial({
vertexShader: RIM_VERTEX_SHADER,
fragmentShader: `
varying vec3 vNormal;
varying vec3 vViewDirection;
void main() {
float viewFacing = max(dot(normalize(vNormal), normalize(vViewDirection)), 0.0);
float rim = 1.0 - viewFacing;
float alpha = pow(rim, ${C.rimGlowPower.toFixed(1)}) * ${C.rimGlowIntensity.toFixed(2)};
gl_FragColor = vec4(${rr.toFixed(2)}, ${rg.toFixed(2)}, ${rb.toFixed(2)}, alpha);
}
`,
blending: THREE.AdditiveBlending,
side: THREE.FrontSide,
transparent: true,
depthTest: false,
depthWrite: false,
});
_earthRimGlow = new THREE.Mesh(rimGlowGeo, rimGlowMat);
_earthRimGlow.name = "earth-rim-glow";
_earthRimGlow.renderOrder = C.rimGlowRenderOrder;
earth.add(_earthRimGlow);
// Texture is loaded separately via loadEarthTexture() for staged loading
return earth;
}
export function createClouds(scene, earthObj) {
const geometry = new THREE.SphereGeometry(CONFIG.earthRadius + 3, 64, 64);
const geometry = new THREE.SphereGeometry(
CONFIG.earthRadius + CLOUD_LAYER_CONFIG.radiusOffset,
CLOUD_LAYER_CONFIG.widthSegments,
CLOUD_LAYER_CONFIG.heightSegments,
);
const material = new THREE.MeshPhongMaterial({
transparent: true,
opacity: 0.15,
opacity: CLOUD_LAYER_CONFIG.opacity,
depthTest: true,
depthWrite: false,
blending: THREE.AdditiveBlending,
@@ -209,10 +300,12 @@ export function createClouds(scene, earthObj) {
});
clouds = new THREE.Mesh(geometry, material);
clouds.name = "earth-atmosphere-clouds";
clouds.visible = showClouds;
earthObj.add(clouds);
textureLoader.load(
'./assets/earth_clouds_1024.png',
CLOUD_LAYER_CONFIG.textureUrl,
function(texture) {
material.map = texture;
material.needsUpdate = true;
@@ -226,6 +319,17 @@ export function createClouds(scene, earthObj) {
return clouds;
}
export function toggleClouds(visible) {
showClouds = Boolean(visible);
if (clouds) {
clouds.visible = showClouds;
}
}
export function getShowClouds() {
return showClouds;
}
export function createTerrain(earthObj) {
const geometry = new THREE.SphereGeometry(
CONFIG.earthRadius + TERRAIN_CONFIG.baseRadiusOffset,
@@ -251,7 +355,8 @@ export function createTerrain(earthObj) {
terrain = new THREE.Mesh(geometry, material);
terrain.name = "earth-real-terrain";
terrain.visible = false;
terrain.renderOrder = 0.5;
terrain.renderOrder = 1.2;
terrain.raycast = () => {};
earthObj.add(terrain);
return terrain;
@@ -265,11 +370,11 @@ export function toggleTerrain(visible) {
export function createStars(scene) {
const starGeometry = new THREE.BufferGeometry();
const starCount = 8000;
const starCount = STARFIELD_CONFIG.count;
const starPositions = new Float32Array(starCount * 3);
for (let i = 0; i < starCount * 3; i += 3) {
const r = 800 + Math.random() * 200;
const r = STARFIELD_CONFIG.minRadius + Math.random() * STARFIELD_CONFIG.radiusJitter;
const theta = Math.random() * Math.PI * 2;
const phi = Math.acos(2 * Math.random() - 1);
@@ -281,8 +386,8 @@ export function createStars(scene) {
starGeometry.setAttribute('position', new THREE.BufferAttribute(starPositions, 3));
const starMaterial = new THREE.PointsMaterial({
color: 0xffffff,
size: 0.5,
color: STARFIELD_CONFIG.color,
size: STARFIELD_CONFIG.size,
transparent: true,
blending: THREE.AdditiveBlending
});
@@ -302,17 +407,19 @@ export function createGridLines(scene, earthObj) {
latitudeLines = [];
longitudeLines = [];
const earthRadius = 100.1;
const earthRadius = CONFIG.earthRadius + GRID_CONFIG.radiusOffset;
const gridMaterial = new THREE.LineBasicMaterial({
color: 0x44aaff,
color: GRID_CONFIG.color,
transparent: true,
opacity: 0.2,
linewidth: 1
opacity: GRID_CONFIG.opacity,
linewidth: GRID_CONFIG.lineWidth,
depthTest: true,
depthWrite: false,
});
for (let lat = -75; lat <= 75; lat += 15) {
for (let lat = -75; lat <= 75; lat += GRID_CONFIG.latitudeStep) {
const points = [];
for (let lon = -180; lon <= 180; lon += 5) {
for (let lon = -180; lon <= 180; lon += GRID_CONFIG.segmentStep) {
const point = latLonToVector3(lat, lon, earthRadius);
points.push(point);
}
@@ -320,14 +427,15 @@ export function createGridLines(scene, earthObj) {
const geometry = new THREE.BufferGeometry().setFromPoints(points);
const line = new THREE.Line(geometry, gridMaterial);
line.userData = { type: 'latitude', value: lat };
line.renderOrder = GRID_CONFIG.renderOrder;
line.visible = showGridLines;
earthObj.add(line);
latitudeLines.push(line);
}
for (let lon = -180; lon <= 180; lon += 30) {
for (let lon = -180; lon <= 180; lon += GRID_CONFIG.longitudeStep) {
const points = [];
for (let lat = -90; lat <= 90; lat += 5) {
for (let lat = -90; lat <= 90; lat += GRID_CONFIG.segmentStep) {
const point = latLonToVector3(lat, lon, earthRadius);
points.push(point);
}
@@ -335,6 +443,7 @@ export function createGridLines(scene, earthObj) {
const geometry = new THREE.BufferGeometry().setFromPoints(points);
const line = new THREE.Line(geometry, gridMaterial);
line.userData = { type: 'longitude', value: lon };
line.renderOrder = GRID_CONFIG.renderOrder;
line.visible = showGridLines;
earthObj.add(line);
longitudeLines.push(line);
@@ -359,29 +468,43 @@ export function getEarth() {
return earth;
}
export function getEarthSurfacePickTarget() {
return _earthTextureOverlay?.visible ? _earthTextureOverlay : earth;
}
export function getClouds() {
return clouds;
}
export function clearEarthTexture() {
if (!_earthMaterial) return;
_earthMaterial.map = null;
_earthMaterial.needsUpdate = true;
_loadedTexture = null;
if (_earthTextureOverlayMaterial) {
_earthTextureOverlayMaterial.map = null;
_earthTextureOverlayMaterial.needsUpdate = true;
}
if (_earthTextureOverlay) {
_earthTextureOverlay.visible = false;
}
if (_earthRimGlow) {
_earthRimGlow.visible = true;
}
}
export function setEarthSunDirection(direction) {
if (!direction) return;
_earthSunDirection.copy(direction).normalize();
if (_earthShader?.uniforms?.uSunDirectionWorld) {
_earthShader.uniforms.uSunDirectionWorld.value.copy(_earthSunDirection);
}
_earthShaders.forEach((shader) => {
shader?.uniforms?.uSunDirectionWorld?.value?.copy(_earthSunDirection);
});
}
export function setDayNightEnabled(enabled) {
_dayNightEnabled = enabled;
if (_earthShader?.uniforms?.uDayNightEnabled) {
_earthShader.uniforms.uDayNightEnabled.value = enabled ? 1.0 : 0.0;
}
_earthShaders.forEach((shader) => {
if (shader?.uniforms?.uDayNightEnabled) {
shader.uniforms.uDayNightEnabled.value = enabled ? 1.0 : 0.0;
}
});
if (_earthMaterial) {
if (enabled) {
// Restore normal Phong lighting + custom day/night shader
@@ -390,10 +513,9 @@ export function setDayNightEnabled(enabled) {
_earthMaterial.emissiveMap = null;
} else {
// Full bright: zero diffuse so directional light has no effect;
// use original color as emissive map to show texture uniformly.
_earthMaterial.color.setRGB(0, 0, 0);
_earthMaterial.emissive.setHex(EARTH_MATERIAL_CONFIG.color);
_earthMaterial.emissiveMap = _earthMaterial.map;
_earthMaterial.emissiveMap = null;
}
_earthMaterial.needsUpdate = true;
}
@@ -401,7 +523,7 @@ export function setDayNightEnabled(enabled) {
export function loadEarthTexture() {
return new Promise((resolve) => {
if (!_earthMaterial) { resolve(); return; }
if (!_earthTextureOverlayMaterial) { resolve(); return; }
const urls = EARTH_MATERIAL_CONFIG.textureUrls;
const tryLoad = (index) => {
@@ -418,12 +540,15 @@ export function loadEarthTexture() {
texture.anisotropy = 16;
texture.minFilter = THREE.LinearMipmapLinearFilter;
texture.magFilter = THREE.LinearFilter;
_earthMaterial.map = texture;
// If day/night is currently disabled, sync emissiveMap to the newly loaded texture
if (!_dayNightEnabled) {
_earthMaterial.emissiveMap = texture;
_loadedTexture = texture;
_earthTextureOverlayMaterial.map = texture;
_earthTextureOverlayMaterial.needsUpdate = true;
if (_earthTextureOverlay) {
_earthTextureOverlay.visible = _textureVisible;
}
if (_earthRimGlow) {
_earthRimGlow.visible = !_textureVisible;
}
_earthMaterial.needsUpdate = true;
resolve();
},
null,
@@ -433,3 +558,22 @@ export function loadEarthTexture() {
tryLoad(0);
});
}
export function setEarthTextureVisible(visible) {
_textureVisible = Boolean(visible);
const textureShowing = _textureVisible && Boolean(_loadedTexture);
if (_earthTextureOverlay) {
_earthTextureOverlay.visible = textureShowing;
}
if (_earthTextureOverlayMaterial) {
_earthTextureOverlayMaterial.map = _loadedTexture || null;
_earthTextureOverlayMaterial.needsUpdate = true;
}
if (_earthRimGlow) {
_earthRimGlow.visible = !textureShowing;
}
}
export function getEarthTextureVisible() {
return _textureVisible;
}

View File

@@ -201,6 +201,7 @@ function getMobilePopupTitle(type, data) {
case 'bgp_collector': return data.collector || 'BGP观测站';
case 'supercomputer': return data.name || '超算';
case 'gpu_cluster': return data.name || 'GPU集群';
case 'vessel': return data.name || '船只';
default: return '详情';
}
}
@@ -215,6 +216,7 @@ function getMobilePopupSubtitle(type, data) {
case 'bgp_collector': return data.location || 'BGP观测站';
case 'supercomputer': return data.country || '超级计算机';
case 'gpu_cluster': return data.country || 'GPU集群';
case 'vessel': return data.vessel_type || 'AIS 船只';
default: return '';
}
}
@@ -545,6 +547,23 @@ const CARD_CONFIG = {
{ key: 'source', label: '来源' },
{ key: 'updated_at', label: '更新时间' }
]
},
vessel: {
icon: '🚢',
title: '船只详情',
className: 'vessel',
fields: [
{ key: 'name', label: '名称' },
{ key: 'mmsi', label: 'MMSI' },
{ key: 'imo', label: 'IMO' },
{ key: 'flag', label: '旗帜' },
{ key: 'vessel_type', label: '船型' },
{ key: 'speed', label: '当前航速', unit: 'kn' },
{ key: 'course', label: '航向', unit: '°' },
{ key: 'status', label: '状态' },
{ key: 'length', label: '船长', unit: 'm' },
{ key: 'received_at', label: '更新时间' }
]
}
};

View File

@@ -5,7 +5,11 @@ const SURFACE_SCALE = 1.003;
const SURFACE_OFFSET = 0.72;
const CLUSTER_DIAMETER_KM_APPROX = 4500;
const CLUSTER_RADIUS_KM_BASE = CLUSTER_DIAMETER_KM_APPROX / 2;
const SURFACE_AXIS = new THREE.Vector3(0, 0, 1);
const IRIDIUM_OVERLAY_COLOR = 0x5faeff;
const IRIDIUM_REFERENCE_ALTITUDE_KM = 780;
const FILL_RINGS = 12;
const FILL_SEGMENTS = 48;
const RING_SEGMENTS = 72;
function disposeMaterial(material) {
if (!material) return;
@@ -19,34 +23,27 @@ function disposeMaterial(material) {
function disposeObjectTree(object) {
if (!object) return;
object.traverse((child) => {
if (child.geometry) {
child.geometry.dispose();
}
if (child.material) {
disposeMaterial(child.material);
}
if (child.geometry) child.geometry.dispose();
if (child.material) disposeMaterial(child.material);
});
}
function createIridiumClusterMaterial() {
function createIridiumFillMaterial() {
return new THREE.ShaderMaterial({
transparent: true,
side: THREE.DoubleSide,
depthTest: true,
depthWrite: false,
polygonOffset: true,
polygonOffsetFactor: -3,
polygonOffsetUnits: -3,
blending: THREE.AdditiveBlending,
uniforms: {
uColor: { value: new THREE.Color(0x5faeff) },
uOpacity: { value: 0.24 },
uColor: { value: new THREE.Color(IRIDIUM_OVERLAY_COLOR) },
uOpacity: { value: 0.55 },
},
vertexShader: `
attribute vec2 aUv;
varying vec2 vUv;
void main() {
vUv = uv;
vUv = aUv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`,
@@ -54,12 +51,10 @@ function createIridiumClusterMaterial() {
uniform vec3 uColor;
uniform float uOpacity;
varying vec2 vUv;
void main() {
vec2 p = vUv * 2.0 - 1.0;
float ellipseMetric = p.x * p.x * 0.82 + p.y * p.y * 1.06;
float alpha = exp(-ellipseMetric * 1.05) * (1.0 - smoothstep(0.86, 1.24, ellipseMetric));
alpha *= uOpacity;
float r2 = dot(vUv, vUv);
float glow = exp(-r2 * 1.4) * (1.0 - smoothstep(0.72, 1.0, r2));
float alpha = glow * uOpacity;
if (alpha <= 0.001) discard;
gl_FragColor = vec4(uColor, alpha);
}
@@ -67,6 +62,17 @@ function createIridiumClusterMaterial() {
});
}
function createIridiumRingMaterial() {
return new THREE.LineBasicMaterial({
color: new THREE.Color(IRIDIUM_OVERLAY_COLOR),
transparent: true,
opacity: 0.75,
blending: THREE.AdditiveBlending,
depthTest: true,
depthWrite: false,
});
}
function projectOffsetToSurface(
centerNormal,
alongTrack,
@@ -88,13 +94,55 @@ function projectOffsetToSurface(
function computeClusterRadiusKm(altitudeKm) {
const altitudeScale = THREE.MathUtils.clamp(
(Number(altitudeKm) || 780) / 780,
(Number(altitudeKm) || IRIDIUM_REFERENCE_ALTITUDE_KM) / IRIDIUM_REFERENCE_ALTITUDE_KM,
0.88,
1.18,
);
return CLUSTER_RADIUS_KM_BASE * altitudeScale;
}
function buildFillGeometry() {
// Radial grid: center + FILL_RINGS rings × FILL_SEGMENTS points each.
// Positions are updated in world space each frame; indices are static.
const vertexCount = 1 + FILL_RINGS * FILL_SEGMENTS;
const positions = new Float32Array(vertexCount * 3);
const uvs = new Float32Array(vertexCount * 2);
// Center vertex: uv = (0,0)
// Edge vertices: uv on unit circle, r = ring/FILL_RINGS
const indices = [];
// Center to first ring: triangle fan
for (let s = 0; s < FILL_SEGMENTS; s++) {
const a = 1 + s;
const b = 1 + (s + 1) % FILL_SEGMENTS;
indices.push(0, a, b);
}
// Ring to ring
for (let r = 0; r < FILL_RINGS - 1; r++) {
const ringBase = 1 + r * FILL_SEGMENTS;
const nextBase = ringBase + FILL_SEGMENTS;
for (let s = 0; s < FILL_SEGMENTS; s++) {
const s1 = (s + 1) % FILL_SEGMENTS;
indices.push(ringBase + s, nextBase + s, ringBase + s1);
indices.push(nextBase + s, nextBase + s1, ringBase + s1);
}
}
const geometry = new THREE.BufferGeometry();
geometry.setAttribute("position", new THREE.BufferAttribute(positions, 3));
geometry.setAttribute("aUv", new THREE.BufferAttribute(uvs, 2));
geometry.setIndex(indices);
return geometry;
}
function buildRingGeometry() {
const positions = new Float32Array(RING_SEGMENTS * 3);
const geometry = new THREE.BufferGeometry();
geometry.setAttribute("position", new THREE.BufferAttribute(positions, 3));
return geometry;
}
export function createIridiumFootprintAdapter({
earthObj,
earthRadiusWorld,
@@ -105,19 +153,21 @@ export function createIridiumFootprintAdapter({
const group = new THREE.Group();
group.name = "iridium-footprint-overlay";
group.renderOrder = renderOrder;
group.userData = {
earthRadiusWorld,
clusterGlow: null,
};
group.userData = { earthRadiusWorld, fill: null, outerRing: null };
const clusterGlow = new THREE.Mesh(
new THREE.CircleGeometry(1, 72),
createIridiumClusterMaterial(),
);
clusterGlow.name = "iridium-cluster-glow";
clusterGlow.renderOrder = renderOrder - 1;
group.add(clusterGlow);
group.userData.clusterGlow = clusterGlow;
const fill = new THREE.Mesh(buildFillGeometry(), createIridiumFillMaterial());
fill.name = "iridium-cluster-fill";
fill.renderOrder = renderOrder;
fill.frustumCulled = false;
group.add(fill);
group.userData.fill = fill;
const outerRing = new THREE.LineLoop(buildRingGeometry(), createIridiumRingMaterial());
outerRing.name = "iridium-outer-ring";
outerRing.renderOrder = renderOrder;
outerRing.frustumCulled = false;
group.add(outerRing);
group.userData.outerRing = outerRing;
earthObj.add(group);
return group;
@@ -129,30 +179,65 @@ export function updateIridiumFootprintAdapter(
) {
if (!group || !position || !alongTrack || !crossTrack) return;
const earthRadiusWorld =
group.userData?.earthRadiusWorld || EARTH_RADIUS_KM;
const earthRadiusWorld = group.userData?.earthRadiusWorld || EARTH_RADIUS_KM;
const centerNormal = position.clone().normalize();
const clusterRadiusKm = computeClusterRadiusKm(altitudeKm);
const clusterGlow = group.userData?.clusterGlow || null;
const worldUnitsPerKm = earthRadiusWorld / EARTH_RADIUS_KM;
const alongRadiusKm = clusterRadiusKm * 1.18;
const crossRadiusKm = clusterRadiusKm * 0.96;
if (clusterGlow) {
const clusterCenter = projectOffsetToSurface(
centerNormal,
alongTrack,
crossTrack,
0,
0,
earthRadiusWorld,
);
const clusterNormal = clusterCenter.clone().normalize();
clusterGlow.position.copy(clusterCenter);
clusterGlow.quaternion.setFromUnitVectors(SURFACE_AXIS, clusterNormal);
clusterGlow.scale.set(
clusterRadiusKm * worldUnitsPerKm * 1.18,
clusterRadiusKm * worldUnitsPerKm * 0.96,
1,
const fill = group.userData?.fill;
if (fill) {
const posAttr = fill.geometry.attributes.position;
const uvAttr = fill.geometry.attributes.aUv;
// Center vertex
const center = projectOffsetToSurface(
centerNormal, alongTrack, crossTrack, 0, 0, earthRadiusWorld,
);
posAttr.setXYZ(0, center.x, center.y, center.z);
uvAttr.setXY(0, 0, 0);
// Ring vertices
for (let r = 1; r <= FILL_RINGS; r++) {
const t = r / FILL_RINGS;
const aKm = alongRadiusKm * t;
const cKm = crossRadiusKm * t;
for (let s = 0; s < FILL_SEGMENTS; s++) {
const angle = (s / FILL_SEGMENTS) * Math.PI * 2;
const cosA = Math.cos(angle);
const sinA = Math.sin(angle);
const pt = projectOffsetToSurface(
centerNormal, alongTrack, crossTrack,
aKm * cosA,
cKm * sinA,
earthRadiusWorld,
);
const vi = 1 + (r - 1) * FILL_SEGMENTS + s;
posAttr.setXYZ(vi, pt.x, pt.y, pt.z);
uvAttr.setXY(vi, t * cosA, t * sinA);
}
}
posAttr.needsUpdate = true;
uvAttr.needsUpdate = true;
fill.geometry.computeBoundingSphere();
}
const outerRing = group.userData?.outerRing;
if (outerRing) {
const posAttr = outerRing.geometry.attributes.position;
for (let k = 0; k < RING_SEGMENTS; k++) {
const angle = (k / RING_SEGMENTS) * Math.PI * 2;
const pt = projectOffsetToSurface(
centerNormal, alongTrack, crossTrack,
alongRadiusKm * Math.cos(angle),
crossRadiusKm * Math.sin(angle),
earthRadiusWorld,
);
posAttr.setXYZ(k, pt.x, pt.y, pt.z);
}
posAttr.needsUpdate = true;
outerRing.geometry.computeBoundingSphere();
}
}

Some files were not shown because too many files have changed in this diff Show More