release: bump version to 0.47.0
This commit is contained in:
@@ -8,6 +8,23 @@ This project follows the repository versioning rule:
|
||||
- `improvement` -> `+0.0.1`(bugfix + 小功能混合)
|
||||
- `bugfix` -> `+0.0.1`
|
||||
|
||||
## [0.47.0] — 2026-04-30
|
||||
|
||||
Released: 2026-04-30
|
||||
|
||||
### ✨ Highlights
|
||||
- 新增 AISStream WebSocket 船只采集器,并将 AIS 多源数据写入原始观测层,由聚合接口统一去重、合并和解释字段来源。
|
||||
- 设置页新增 AISStream API Key、采集范围 preset、运行状态、连接验证和凭证教程入口,让全球 AIS 采集链路可配置、可观察。
|
||||
- Earth 船只图层默认不再限制 5000 艘,并统一 marker 颜色、详情卡、hover 和搜索结果的船型归一化显示。
|
||||
|
||||
### 🔧 Improvements
|
||||
- 聚合接口新增 `field_sources`、`selected_reasons`、`source_summary`、`quality_flags` 和冲突记录调试接口,动态字段默认优先采用更新的实时流观测。
|
||||
- AISStream 标准化支持 `MetaData.ShipName` 船名兜底,并将 AIS 数字船型映射为 Cargo / Tanker / Passenger / Fishing / Military。
|
||||
- 将仓库 docs 技能改为通用文档工作流,Planet 专属白名单、双语、裸文件标题和凭证教程规则迁移到 `docs/documentation-coverage-rules.md`。
|
||||
- 更新 AIS v4/v5 TODO 与计划文档,明确后续聚合策略配置、船舶资料 enrichment 和媒体缓存边界。
|
||||
|
||||
---
|
||||
|
||||
## [0.46.3] — 2026-04-30
|
||||
|
||||
Released: 2026-04-30
|
||||
|
||||
117
docs/documentation-coverage-rules.md
Normal file
117
docs/documentation-coverage-rules.md
Normal file
@@ -0,0 +1,117 @@
|
||||
# Documentation Coverage Rules
|
||||
|
||||
This file contains Planet-specific documentation coverage rules. Documentation skills and agents should read this file before deciding which docs to update. Keep tool-specific workflow in skills; keep product and repository rules here.
|
||||
|
||||
## Scope Rules
|
||||
|
||||
- User-visible workflow changes must update `docs/technical/zh/manual.md` and usually `docs/technical/zh/quickstart.md`.
|
||||
- If an English counterpart exists for user-facing docs such as `manual.md` or `quickstart.md`, update `docs/technical/en/...` enough that it does not contradict the Chinese source.
|
||||
- Control console page responsibility changes must update `docs/technical/zh/frontend-admin-frontend-context.md`.
|
||||
- Earth frontend behavior changes must update `docs/technical/zh/earth-frontend-context.md`.
|
||||
- Earth layer additions, `renderOrder`, altitude/radius offsets, depth strategy, pointer picking, legend modes, or layer panel/startup ordering must update `docs/technical/zh/earth-render-layer-order.md`.
|
||||
- Earth layer visual style or legend symbol/color semantics should also update `docs/technical/zh/earth-layer-style-reference.md` when that reference is affected.
|
||||
- Collector, datasource, credential, settings, connectivity, scheduler, or API changes must update the relevant backend docs, especially `docs/technical/zh/backend-collectors.md` and any datasource/settings-specific doc.
|
||||
- When a change turns an old plan assumption into current behavior, update the relevant `docs/plans/*.md` with a status note instead of leaving contradictory instructions.
|
||||
- Search docs for stale terms introduced by the change, for example old tab names, old route responsibilities, obsolete auth assumptions, or renamed UI labels.
|
||||
|
||||
## Public Docs Rules
|
||||
|
||||
- If adding a new technical document, add it to `docs/technical/zh/README.md` when it should be discoverable from the technical docs index.
|
||||
- If a technical document should be visible in the public Docs page or linked from a technical README, register it in `frontend/src/pages/Docs/docs-content.ts` under `DOCS_METADATA`. Files under `docs/technical/{zh,en}/` are not automatically routable.
|
||||
- For every public technical doc, keep the bilingual file pair in sync by filename: `docs/technical/zh/<name>.md` and `docs/technical/en/<name>.md`. If content is intentionally Chinese-only or English-only, state that intentionally in the final note.
|
||||
- Public docs should use readable link text, not raw filenames such as `manual.md`.
|
||||
|
||||
## Credential Collector Rules
|
||||
|
||||
- Any built-in collector marked `requires_credentials: true` and `credential_status: supported` must have:
|
||||
- a `credential_provider` in `backend/app/core/datasource_defaults.py`;
|
||||
- a default credential guide in `backend/app/services/credential_guides.py`;
|
||||
- a supported connectivity provider in `backend/app/services/datasource_connectivity.py`;
|
||||
- settings UI guidance or a credential form in `frontend/src/pages/Settings/Settings.tsx`;
|
||||
- a regression test that fails if the guide/provider is missing.
|
||||
|
||||
## Recommended Checks
|
||||
|
||||
Run the checks that match the affected docs.
|
||||
|
||||
### Duplicate Bilingual Docs
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
from pathlib import Path
|
||||
same = []
|
||||
for en in sorted(Path("docs/technical/en").glob("*.md")):
|
||||
zh = Path("docs/technical/zh") / en.name
|
||||
if zh.exists() and en.read_text() == zh.read_text():
|
||||
same.append(en.name)
|
||||
if same:
|
||||
raise SystemExit("identical en/zh docs: " + ", ".join(same))
|
||||
print("no identical en/zh docs")
|
||||
PY
|
||||
```
|
||||
|
||||
### Language-Less Technical Links
|
||||
|
||||
```bash
|
||||
rg -n "/home/ray/dev/linkong/planet/docs/technical/(?!zh|en)" docs/technical/zh --pcre2
|
||||
```
|
||||
|
||||
This should return no matches.
|
||||
|
||||
### Public Docs Registry
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
metadata = Path("frontend/src/pages/Docs/docs-content.ts").read_text()
|
||||
known = set(re.findall(r"'([^']+\.md)':\s*\{", metadata))
|
||||
known.add("README.md")
|
||||
|
||||
missing = []
|
||||
for readme in [Path("docs/technical/zh/README.md"), Path("docs/technical/en/README.md")]:
|
||||
if not readme.exists():
|
||||
continue
|
||||
for href in re.findall(r"\]\(([^)]+\.md)\)", readme.read_text()):
|
||||
path = Path(href)
|
||||
if "docs/technical/" not in href:
|
||||
continue
|
||||
filename = path.name
|
||||
if filename not in known:
|
||||
missing.append(f"{readme}: {filename}")
|
||||
|
||||
if missing:
|
||||
raise SystemExit("docs README links missing DOCS_METADATA: " + ", ".join(missing))
|
||||
print("docs README links are whitelisted")
|
||||
PY
|
||||
```
|
||||
|
||||
### Public Bilingual Pairs
|
||||
|
||||
```bash
|
||||
python - <<'PY'
|
||||
import re
|
||||
from pathlib import Path
|
||||
|
||||
metadata = Path("frontend/src/pages/Docs/docs-content.ts").read_text()
|
||||
filenames = sorted(set(re.findall(r"'([^']+\.md)':\s*\{", metadata)) - {"README.md"})
|
||||
missing = []
|
||||
for filename in filenames:
|
||||
for lang in ("zh", "en"):
|
||||
path = Path("docs/technical") / lang / filename
|
||||
if not path.exists():
|
||||
missing.append(str(path))
|
||||
if missing:
|
||||
raise SystemExit("missing bilingual docs: " + ", ".join(missing))
|
||||
print("public docs have zh/en file pairs")
|
||||
PY
|
||||
```
|
||||
|
||||
### Raw Filename Link Titles
|
||||
|
||||
```bash
|
||||
rg -n "\[[^]]+\.md\]\(" docs/technical/zh docs/technical/en
|
||||
```
|
||||
|
||||
This should return no matches for polished public docs.
|
||||
@@ -1,6 +1,6 @@
|
||||
# AIS 多源采集、冲突记录与聚合接口计划
|
||||
|
||||
**状态**:规划中
|
||||
**状态**:v0-v3 已实现,v4+ 规划中
|
||||
**创建日期**:2026-04-30
|
||||
**核心原则**:采集器只写原始观测;去重、合并、冲突解释放在聚合接口中完成
|
||||
|
||||
@@ -14,6 +14,8 @@
|
||||
| 冲突处理 | 先记录冲突事实和当前选择原因,后续再开放用户规则配置 |
|
||||
| 默认可信度 | 同类 AIS 数据源优先按 `delivery_mode` 评估:`realtime_stream` 优于 `batch_stream`,再优于 `polling` 和 `snapshot` |
|
||||
| 过期保护 | 实时流源断流超过 freshness 窗口后,不能仅凭“实时源”身份压过更新的轮询数据 |
|
||||
| 源健康状态 | 聚合时必须参考采集器健康状态,不能只看配置中的理论优先级 |
|
||||
| 媒体富化 | 船只图片等媒体信息不进入 AIS 实时聚合主链路,后续单独做 enrichment |
|
||||
|
||||
## 背景
|
||||
|
||||
@@ -24,7 +26,7 @@
|
||||
- WebSocket 或其他实时流通常更接近实时,但也可能断流或批量延迟。
|
||||
- 如果每个 collector 自己做去重合并,规则会分散、不可审计,也很难让用户后续配置“某个字段信任哪个来源”。
|
||||
|
||||
因此 v1 不应让采集器直接覆盖最终船只表。更稳的方式是先保留观测事实,再由聚合接口统一给出当前展示视图。
|
||||
因此第一阶段不应让采集器直接覆盖最终船只表。更稳的方式是先保留观测事实,再由聚合接口统一给出当前展示视图。
|
||||
|
||||
## 目标架构
|
||||
|
||||
@@ -53,11 +55,36 @@ flowchart LR
|
||||
| `transport` | `websocket`、`sse`、`http`、`file` 等 |
|
||||
| `observed_at` | 上游数据时间,优先使用 AIS 消息时间 |
|
||||
| `collected_at` | 本系统接收或采集时间 |
|
||||
| `source_message_id` | 上游消息 ID 或可推导 ID,没有则为空 |
|
||||
| `observation_hash` | 幂等去重指纹,用于防止同一来源重复写入同一条观测 |
|
||||
| `normalized_payload` | 标准化后的 AIS JSON |
|
||||
| `raw_payload` | 可选,保存原始或裁剪后的上游记录 |
|
||||
| `quality_flags` | 观测级质量标记,例如 `stale`、`position_jump`、`future_timestamp` |
|
||||
|
||||
`delivery_mode` 和 `transport` 不应混为一谈。WebSocket 是传输方式;streaming 是交付模式。聚合可信度主要看 `delivery_mode`,`transport` 只作为辅助信息。
|
||||
|
||||
原始观测层需要做存储级幂等去重,但这里的去重不是业务合并。推荐使用 `source + entity_key + message_type + observed_at + payload_hash` 或上游稳定消息 ID 作为唯一约束,避免 WebSocket 重连、HTTP 重试或批量回放导致同一事实重复入库。
|
||||
|
||||
### 源健康状态
|
||||
|
||||
每个采集器应维护独立的健康状态,供聚合服务读取:
|
||||
|
||||
| 字段 | 用途 |
|
||||
|-----|------|
|
||||
| `source` | 采集器标识 |
|
||||
| `connection_state` | `connected`、`reconnecting`、`disconnected`、`disabled` 等 |
|
||||
| `last_seen_at` | 最近收到上游消息或响应的时间 |
|
||||
| `last_success_at` | 最近成功写入观测的时间 |
|
||||
| `last_error` | 最近错误摘要 |
|
||||
| `message_rate` | 最近窗口内的消息速率 |
|
||||
| `lag_seconds` | 上游观测时间与本系统接收时间的延迟 |
|
||||
|
||||
聚合优先级不能只看 `source_priority`。例如 `aisstream_vessels` 默认优先于 `barentswatch_vessels`,但如果它处于 `disconnected` 或 `lag_seconds` 超过 freshness 窗口,则动态字段应回退到更新的可用来源。
|
||||
|
||||
### 身份键边界
|
||||
|
||||
v1 可以继续用 MMSI 作为 `entity_key`,因为它是 AIS 动态消息里最稳定、最容易获得的主键。但文档和模型都要为后续扩展留出口:MMSI 可能复用、填错或缺少静态信息,后续身份解析应结合 `mmsi + imo + callsign + name + dimensions` 判断是否需要拆分或合并实体。
|
||||
|
||||
### 冲突记录层
|
||||
|
||||
聚合服务发现同一个实体、同一个字段存在多个非空不同值时,写入冲突记录。冲突记录不代表错误,只代表“有多个可用候选值”。
|
||||
@@ -90,7 +117,8 @@ flowchart LR
|
||||
| 动态位置 | `lat`、`lon`、`sog`、`cog`、`heading`、`nav_status` | 优先最新 `observed_at`,同时间再按来源优先级 |
|
||||
| 静态身份 | `name`、`callsign`、`imo`、`flag` | 非空优先,再按字段策略或来源优先级 |
|
||||
| 静态规格 | `vessel_type`、`vessel_type_name`、`length`、`width`、`draught` | 非空优先;冲突时记录候选值 |
|
||||
| 元信息 | `field_sources`、`conflict_count`、`selected_reasons` | 聚合接口生成,便于调试和后续 UI 展示 |
|
||||
| 轨迹点 | `track_points` | 按时间线合并;同一时间窗口内相近点去重;保留点级 `source` |
|
||||
| 元信息 | `field_sources`、`conflict_count`、`selected_reasons`、`quality_flags` | 聚合接口生成,便于调试和后续 UI 展示 |
|
||||
|
||||
### 默认优先级
|
||||
|
||||
@@ -124,6 +152,27 @@ freshness:
|
||||
|
||||
如果 `aisstream_vessels` 最近 15 分钟没有该 MMSI 的新观测,而 BarentsWatch 轮询源有更新位置,则位置类字段应采用 BarentsWatch 的更新观测,并记录选择原因 `newest_observation` 或 `freshness_fallback`。
|
||||
|
||||
### 异常位置保护
|
||||
|
||||
多源 AIS 接入后,聚合服务必须过滤或降权明显异常的位置观测:
|
||||
|
||||
- 经纬度必须在合法范围内。
|
||||
- `observed_at` 不能明显来自未来。
|
||||
- 同一 MMSI 短时间内跨越不合理距离时,标记 `position_jump`,默认不直接采用该点。
|
||||
- 当异常点来自当前优先源时,应记录 `selected_reason = anomaly_rejected`,再回退到其他可用来源。
|
||||
|
||||
异常保护不应静默丢弃事实。原始观测仍应保留,聚合结果通过 `quality_flags` 和冲突记录解释为什么没有采用它。
|
||||
|
||||
### 轨迹聚合
|
||||
|
||||
轨迹接口不能简单拼接所有来源,否则前端会出现折返、抖动和重复点。默认规则:
|
||||
|
||||
- 以 `observed_at` 排序,生成统一时间线。
|
||||
- 同一来源的完全重复点通过 `observation_hash` 去重。
|
||||
- 多来源在短时间窗口内上报的相近位置视为同一轨迹点,优先选择 freshness 和 source priority 更高的一条。
|
||||
- 每个轨迹点保留 `source`、`selected_reason` 和必要的 `quality_flags`。
|
||||
- 对被判定为 `position_jump` 的点,默认不进入展示轨迹,但可通过调试参数查看。
|
||||
|
||||
## 聚合接口
|
||||
|
||||
现有展示接口应逐步改为消费聚合服务,而不是自己直接拼 `VesselPosition + VesselStatic`。
|
||||
@@ -155,6 +204,7 @@ GeoJSON properties 建议增加:
|
||||
"lat": "newest_observation",
|
||||
"vessel_type": "non_empty_priority"
|
||||
},
|
||||
"quality_flags": [],
|
||||
"conflict_count": 2
|
||||
}
|
||||
```
|
||||
@@ -231,15 +281,89 @@ aisstream_vessels:
|
||||
- ShipStaticData
|
||||
```
|
||||
|
||||
## 实施顺序
|
||||
默认不建议直接订阅全球范围。AISStream 采集器应支持以下订阅策略:
|
||||
|
||||
1. 新增原始观测模型和冲突记录模型。
|
||||
2. 实现 AIS 聚合服务,先从现有 `vessel_position` / `vessel_static` 兼容读取,再逐步切换到原始观测层。
|
||||
3. 将 `/geo/vessels` 和 `/vessels/{mmsi}` 改为走聚合服务。
|
||||
4. 改造 BarentsWatch 保存逻辑,让它写入原始观测,同时保留现有表作为兼容缓存。
|
||||
5. 实现 AISStream WebSocket collector。
|
||||
6. 接入系统设置中的聚合策略配置。
|
||||
7. 做冲突治理 UI。
|
||||
- 使用配置的固定 `bounding_boxes`。
|
||||
- 后续支持按 Earth 当前视口或关注区域动态调整订阅范围。
|
||||
- 支持限制 `message_types`,避免静态信息、位置报告和扩展消息全量涌入。
|
||||
- 断线后使用指数退避重连,并把连接状态写入源健康状态。
|
||||
- 重连后可能收到重复或回放消息,因此必须依赖原始观测层的幂等去重。
|
||||
|
||||
### 媒体富化边界
|
||||
|
||||
VesselFinder 等服务里的船只图片不属于 AIS 实时数据本身。图片、船籍详情、公司信息等后续应作为独立 enrichment 链路:
|
||||
|
||||
- 通过 MMSI、IMO、船名等字段异步查询。
|
||||
- 使用独立缓存和授权配置。
|
||||
- 不阻塞 `vessel_ais` 实时观测入库。
|
||||
- 聚合接口只暴露已经缓存好的媒体引用,不在请求链路中现场抓取。
|
||||
|
||||
## 版本拆分
|
||||
|
||||
计划按 5 个版本推进:
|
||||
|
||||
### v0 — 聚合基础设施(已实现)
|
||||
|
||||
目标是不改变前端展示行为,先把数据底座铺好。
|
||||
|
||||
1. 新增原始观测模型、冲突记录模型和源健康状态模型。
|
||||
2. 为现有 BarentsWatch collector 写入原始观测,同时保留现有 `vessel_position` / `vessel_static` 兼容写入。
|
||||
3. 实现存储级 `observation_hash` 幂等去重。
|
||||
4. 补基础管理命令或调试接口,用于查看某个 MMSI 的原始观测和冲突候选。
|
||||
|
||||
### v1 — 聚合读接口(已实现)
|
||||
|
||||
目标是让展示接口开始消费聚合结果,但前端形状保持兼容。
|
||||
|
||||
1. 实现 AIS 聚合服务,先兼容读取现有表,再逐步切换到原始观测层。
|
||||
2. 将 `/geo/vessels` 和 `/vessels/{mmsi}` 改为走聚合服务。
|
||||
3. 将 `/vessels/{mmsi}/track` 改为走轨迹聚合逻辑。
|
||||
4. 返回 `field_sources`、`selected_reasons`、`quality_flags`、`conflict_count`。
|
||||
5. 加入 freshness fallback 和异常位置保护。
|
||||
|
||||
### v2 — AISStream WebSocket collector(已实现)
|
||||
|
||||
目标是接入第二个真实 AIS 来源,并验证多源冲突和回退逻辑。
|
||||
|
||||
1. 实现 `aisstream_vessels` collector。
|
||||
2. 支持 API key、订阅范围、消息类型、重连和限流配置。
|
||||
3. 将 AISStream 写入原始观测层,不直接 upsert 最终展示表。
|
||||
4. 接入源健康状态和 message rate 统计。
|
||||
5. 提供 AISStream API Key 获取教程、设置页入口和连接验证支持。
|
||||
6. 为重复消息、断流回退、WS 优先级写集成测试。
|
||||
|
||||
### v3 — AISStream 可用性与配置体验(已实现)
|
||||
|
||||
目标是让 AISStream 从“能采集”变成日常可观察、可调试、可配置的数据源。
|
||||
|
||||
1. 设置页展示 AISStream 运行状态:连接状态、最近收到、最近成功、本轮消息数、延迟和最近错误。
|
||||
2. AISStream 设置页提供常用采集范围 preset,并保留自定义 Bounding Boxes JSON。
|
||||
3. 聚合结果返回 `source_summary`,展示每艘船的来源、观测数量、最新观测时间、传输模式和消息类型。
|
||||
4. 保留 `field_sources` 和 `selected_reasons`,用于解释动态字段来自实时流、静态字段来自可用非空来源。
|
||||
5. 船名标准化会读取 AISStream `MetaData.ShipName`;船型展示会从 `vessel_type_name` 和 AIS 数字 `vessel_type` 共同归一化,保证 marker 颜色、详情卡、hover 和搜索结果一致。
|
||||
6. `/geo/vessels` 不再默认限制 5000 艘;不传 `limit` 或传 `limit=0` 表示全量返回,前端默认也不再二次裁剪到 5000。
|
||||
|
||||
### v4 — 策略配置
|
||||
|
||||
目标是开放系统级配置,但仍以安全默认值兜底。
|
||||
|
||||
1. 接入系统设置中的聚合策略配置。
|
||||
2. 支持 source priority、字段级规则、freshness 窗口和高级保护开关。
|
||||
3. 保存配置时校验未知字段、非法模式和危险动态字段锁定。
|
||||
4. 聚合接口返回当前命中的配置版本,方便排查。
|
||||
|
||||
### v5 — 船舶资料 enrichment 与冲突治理
|
||||
|
||||
目标是把 AIS 实时流里不稳定或低频出现的静态信息,补成可缓存、可审计的船舶资料层,同时把冲突解释变成可操作能力。
|
||||
|
||||
1. 做冲突治理 UI。
|
||||
2. 支持把人工选择沉淀成字段级规则。
|
||||
3. 支持恢复默认策略。
|
||||
4. 设计 `vessel_profile_enrichment`,按 `mmsi + imo + name + callsign` 异步补充船名、船型细分、AIS 大类、旗国、尺寸、建造年份、运营方等静态资料。
|
||||
5. 设计 `vessel_media_enrichment`,异步补充船只图片和外部详情缓存。
|
||||
6. enrichment 结果必须带 `source`、`fetched_at`、`expires_at`、`confidence` 和原始引用,不覆盖 AIS 原始观测。
|
||||
7. 聚合接口只读取已缓存 enrichment;请求链路不现场抓取第三方页面,避免慢请求和授权风险。
|
||||
8. 前端船只详情面板展示已缓存资料和媒体,并标注字段来源,不阻塞 AIS 实时链路。
|
||||
|
||||
## 测试计划
|
||||
|
||||
@@ -247,8 +371,12 @@ aisstream_vessels:
|
||||
- 多来源同一 MMSI 的位置字段优先选择最新观测。
|
||||
- 实时流和轮询源同时间冲突时,实时流优先。
|
||||
- 实时流过期后,更新的轮询源可以接管动态字段。
|
||||
- 实时流源健康状态异常时,动态字段可以回退到更新的可用来源。
|
||||
- 静态字段不会被空值覆盖。
|
||||
- 静态字段冲突会写入冲突记录。
|
||||
- 明显异常位置不会进入默认展示轨迹,并会留下 `quality_flags`。
|
||||
- 同一时间窗口内多来源相近轨迹点只展示一个点。
|
||||
- AISStream 重连或回放导致的重复消息不会重复进入聚合结果。
|
||||
- 字段级配置可以覆盖默认来源优先级。
|
||||
- 聚合接口在没有冲突表时仍可返回兼容 GeoJSON。
|
||||
|
||||
|
||||
@@ -124,7 +124,7 @@ CREATE UNIQUE INDEX ON vessel_latest(mmsi);
|
||||
GET /api/v1/visualization/geo/vessels
|
||||
?bbox=lon_min,lat_min,lon_max,lat_max # 视口裁剪
|
||||
?type=cargo,tanker,passenger # 船型过滤
|
||||
?limit=5000
|
||||
?limit=0 # 可选;不传或 0 表示不裁剪数量
|
||||
→ GeoJSON FeatureCollection(Point)
|
||||
|
||||
GET /api/v1/visualization/vessels/{mmsi} # 单船详情
|
||||
@@ -163,6 +163,8 @@ GeoJSON Feature 格式:
|
||||
- 后端 BarentsWatch collector 继续以 HTTP polling 方式采集
|
||||
- AISStream 等实时源以独立 WebSocket collector 写入原始观测层
|
||||
- 展示接口从聚合服务读取当前船只视图,而不是由单个 collector 决定最终展示值
|
||||
- 前端默认不再给 `/geo/vessels` 传 `limit=5000`,`VESSEL_CONFIG.maxRenderedMarkers = 0` 表示不做前端数量裁剪;后续如性能不足再引入显式 LOD 上限
|
||||
- marker 颜色、详情卡、hover 和搜索结果必须共享 `vessel_type_display` 船型归一化结果,避免 AIS 数字类型码已驱动颜色但卡片仍显示 `Other`
|
||||
- 前端是否升级为 WebSocket delta push 是独立优化,不影响后端采集器可以使用 WebSocket 接上游实时源
|
||||
|
||||
---
|
||||
@@ -196,8 +198,8 @@ GeoJSON Feature 格式:
|
||||
|
||||
| 相机距离 | 渲染策略 |
|
||||
|---------|---------|
|
||||
| > 400 | 仅渲染 top 1000 艘(按数据新鲜度 + 船型优先级) |
|
||||
| 200–400 | 渲染 top 5000 艘 |
|
||||
| > 400 | 默认渲染当前接口返回的全部船只;如性能不足,再引入可配置 LOD 上限 |
|
||||
| 200–400 | 默认渲染当前接口返回的全部船只;如性能不足,再引入可配置 LOD 上限 |
|
||||
| < 200 | 渲染当前视口 bbox 内全部船只 |
|
||||
|
||||
前端根据相机位置动态计算 bbox,附加到 API 请求中。
|
||||
|
||||
@@ -84,6 +84,8 @@ async def run(self, db):
|
||||
| HuggingFace Spaces | space | Demo applications | 1 day |
|
||||
| PeeringDB | ixp/network/facility | Internet exchange points / networks / facilities | 1-2 days |
|
||||
| TeleGeography | submarine_cable | Submarine cable information | 7 days |
|
||||
| BarentsWatch AIS | vessel | AIS vessel positions, speed, heading, MMSI, and related fields | Collector settings |
|
||||
| AISStream Vessels | vessel_ais | AIS WebSocket realtime stream, written to the raw observation layer and displayed through aggregation | Collector settings |
|
||||
|
||||
## IV. Data Format (stored in CollectedData table)
|
||||
|
||||
|
||||
@@ -260,6 +260,30 @@ AIS request rules:
|
||||
|
||||
`VesselAISCollector` no longer reads environment variables directly. It goes through `resolve_barentswatch_config()` and `fetch_barentswatch_access_token()` so settings, connectivity validation, and collection do not fork into three credential flows.
|
||||
|
||||
## AISStream Collector Chain
|
||||
|
||||
Files:
|
||||
|
||||
- [aisstream.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/aisstream.py)
|
||||
- [vessel_ais_aggregation.py](/home/ray/dev/linkong/planet/backend/app/services/vessel_ais_aggregation.py)
|
||||
|
||||
AISStream uses a WebSocket realtime stream. The collector writes only to the `ais_raw_observations` raw observation layer; it does not directly overwrite the final vessel display table. The aggregation API handles multi-source deduplication, field selection, and conflict records.
|
||||
|
||||
Configuration:
|
||||
|
||||
- `api_key`: stored in `DataSourceConfig.auth_config`, or provided through `AISSTREAM_API_KEY`.
|
||||
- `endpoint`: defaults to `wss://stream.aisstream.io/v0/stream`.
|
||||
- `message_types`: defaults to `PositionReport` and `ShipStaticData`.
|
||||
- `bounding_boxes`: AISStream format is `[[[lat_min, lon_min], [lat_max, lon_max]]]`; the settings page provides global, Norway / North Sea, Europe coast, East Asia, and North America coast presets.
|
||||
- `max_messages` and `receive_timeout_seconds`: control the batch-style WebSocket collection window.
|
||||
|
||||
Normalization:
|
||||
|
||||
- `PositionReport` mainly provides position, speed, course, heading, and navigation status.
|
||||
- Vessel names can be filled from `MetaData.ShipName` even when the message body has no `name`.
|
||||
- Vessel type usually comes from lower-frequency `ShipStaticData.Type`; the backend maps AIS numeric type codes to Cargo / Tanker / Passenger / Fishing / Military.
|
||||
- If a vessel has not yet produced a static message, its aggregated type can still be `Other`; v5 vessel profile enrichment is planned to fill that gap.
|
||||
|
||||
## Credential Guide
|
||||
|
||||
File:
|
||||
@@ -277,6 +301,7 @@ POST /api/v1/settings/credential-guides/{provider}/reset
|
||||
Currently supported:
|
||||
|
||||
- `barentswatch`
|
||||
- `aisstream`
|
||||
|
||||
The default guide includes the official BarentsWatch tutorial:
|
||||
|
||||
|
||||
@@ -96,6 +96,7 @@ Responsibilities:
|
||||
|
||||
- [satellites.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/satellites.js)
|
||||
- [cables.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cables.js)
|
||||
- [vessels.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/vessels.js)
|
||||
- [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js)
|
||||
- [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
|
||||
- [compute-centers.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/compute-centers.js)
|
||||
@@ -108,6 +109,14 @@ Each module is responsible for its own:
|
||||
- State tracking (loaded, visible, hover, locked)
|
||||
- Self-cleanup (dispose on scene destroy)
|
||||
|
||||
### AIS Vessel Layer
|
||||
|
||||
The vessel layer fetches `/api/v1/visualization/geo/vessels` and renders the aggregated AIS GeoJSON through `createInteractableLayer()`. By default it does not send a `limit` parameter, and `VESSEL_CONFIG.maxRenderedMarkers = 0` means the frontend does not clip the result to 5000 vessels. A positive `options.limit` or positive `maxRenderedMarkers` can still be used as an explicit temporary cap.
|
||||
|
||||
Vessel color and vessel type text must use the same normalized classification. `vessels.js` derives `type` from both `vessel_type_name` and the AIS numeric `vessel_type` code; that `type` drives marker color. It also derives `vessel_type_display`, which `main.js` uses for the info card, hover summary, and search result subtitle. Do not make the info card read only the raw `vessel_type_name`, because AISStream can provide a numeric type while the raw name is still `Other`.
|
||||
|
||||
AISStream `PositionReport` messages commonly carry live position and `MetaData.ShipName`, while vessel type usually comes from lower-frequency `ShipStaticData.Type`. The backend normalizes `MetaData.ShipName` into the vessel name and maps numeric type codes into Cargo / Tanker / Passenger / Fishing / Military where available. Missing type detail should wait for a static AIS message or the planned vessel profile enrichment; the frontend should not invent a more specific type.
|
||||
|
||||
### 7. HUD Panels and Search
|
||||
|
||||
- [hud-panels.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/hud-panels.js)
|
||||
|
||||
@@ -182,11 +182,14 @@ The land/ocean base is an Earth base-map asset and preloads at startup; the "Bor
|
||||
| Vessel renderOrder | local `VESSEL_RENDER_ORDER` | `4.4` | Normal marker and interactive overlay |
|
||||
| Vessel track renderOrder | `VESSEL_RENDER_ORDER - 0.1` | `4.3` | Below vessel markers |
|
||||
| Vessel point pixel size | local `VESSEL_POINT_SIZE` | `34` | Shared size for normal markers and hover / locked overlays |
|
||||
| Default vessel render cap | `VESSEL_CONFIG.maxRenderedMarkers` | `0` | `0` means the frontend does not clip by default; positive values send `limit` and clip markers |
|
||||
| Vessel texture canvas size | local `VESSEL_ATLAS_CELL_SIZE` | `128` | Canvas point texture |
|
||||
| Course bucket count | local `VESSEL_COURSE_BINS` | `32` | Moving vessels are bucketed by COG to reduce draw calls while preserving direction |
|
||||
| Vessel hover picking throttle | local `VESSEL_HOVER_PICK_INTERVAL_MS` | `100` | `main.js` hover picking |
|
||||
| Vessel screen hit radius | local `VESSEL_POINTER_RADIUS_PX` | `22` | `main.js` screen-space picking |
|
||||
|
||||
AIS vessel markers use batched `THREE.Points`, not one `THREE.Sprite` per vessel. Moving vessels stay triangular, anchored or slow vessels stay circular, and hover / locked states add a same-size glow overlay. Vessel type color and info-card type text must come from the same normalized result: `vessels.js` reads both backend `vessel_type_name` and AIS numeric `vessel_type`, derives the color-driving `type`, then exposes `vessel_type_display` for the info card, hover summary, and search results.
|
||||
|
||||
## Compute Centers
|
||||
|
||||
| Name | Variable | Current Value | Location / Notes |
|
||||
|
||||
@@ -86,6 +86,7 @@ async def run(self, db):
|
||||
| TeleGeography | submarine_cable | 海底光缆信息 | 7天 |
|
||||
| Space-Track TLE | satellite_tle | 卫星轨道 TLE 数据 | 依采集器配置 |
|
||||
| BarentsWatch AIS | vessel | 船只位置、航速、航向、MMSI 等 AIS 数据 | 依采集器配置 |
|
||||
| AISStream Vessels | vessel_ais | AIS WebSocket 实时流,写入原始观测层并由聚合接口展示 | 依采集器配置 |
|
||||
|
||||
## 四、数据格式 (统一存储到 CollectedData 表)
|
||||
|
||||
@@ -238,7 +239,8 @@ backend/app/services/collectors/
|
||||
├── huggingface.py # HuggingFace采集器
|
||||
├── peeringdb.py # PeeringDB采集器
|
||||
├── telegeraphy.py # TeleGeography海底光缆采集器
|
||||
└── vessel_ais.py # BarentsWatch AIS 船只采集器
|
||||
├── vessel_ais.py # BarentsWatch AIS 船只采集器
|
||||
└── aisstream.py # AISStream WebSocket 船只采集器
|
||||
|
||||
backend/app/models/
|
||||
└── collected_data.py # 统一数据模型
|
||||
@@ -251,6 +253,7 @@ backend/app/models/
|
||||
| 采集器 | credential provider | 凭证来源 |
|
||||
| --- | --- | --- |
|
||||
| `barentswatch_vessels` | `barentswatch` | 控制台采集器设置、环境变量、`~/.zshrc` |
|
||||
| `aisstream_vessels` | `aisstream` | 控制台采集器设置、环境变量 |
|
||||
| `spacetrack_tle` | `spacetrack` | 环境变量、`~/.zshrc` |
|
||||
|
||||
### BarentsWatch AIS
|
||||
|
||||
@@ -262,6 +262,30 @@ AIS 请求规则:
|
||||
|
||||
`VesselAISCollector` 不再自己读取环境变量,而是统一走 `resolve_barentswatch_config()` 和 `fetch_barentswatch_access_token()`,避免设置页、连接验证和采集器三套凭证逻辑分叉。
|
||||
|
||||
## AISStream 采集器链路
|
||||
|
||||
文件:
|
||||
|
||||
- [aisstream.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/aisstream.py)
|
||||
- [vessel_ais_aggregation.py](/home/ray/dev/linkong/planet/backend/app/services/vessel_ais_aggregation.py)
|
||||
|
||||
AISStream 使用 WebSocket 实时流,采集器只写入 `ais_raw_observations` 原始观测层,不直接覆盖最终船只展示表。聚合接口负责多源去重、字段选择和冲突记录。
|
||||
|
||||
配置项:
|
||||
|
||||
- `api_key`:保存在 `DataSourceConfig.auth_config`,也可用环境变量 `AISSTREAM_API_KEY`。
|
||||
- `endpoint`:默认 `wss://stream.aisstream.io/v0/stream`。
|
||||
- `message_types`:默认 `PositionReport` 和 `ShipStaticData`。
|
||||
- `bounding_boxes`:AISStream 格式为 `[[[lat_min, lon_min], [lat_max, lon_max]]]`,设置页提供全球、挪威 / 北海、欧洲近海、东亚、北美东西海岸 preset。
|
||||
- `max_messages` 和 `receive_timeout_seconds`:控制单次批次式 WebSocket 采集窗口。
|
||||
|
||||
标准化规则:
|
||||
|
||||
- `PositionReport` 主要提供位置、速度、航向和状态。
|
||||
- 船名可以从 `MetaData.ShipName` 补入,即使消息体本身没有 `name`。
|
||||
- 船型通常来自低频 `ShipStaticData.Type`;后端会把 AIS 数字类型码映射为 Cargo / Tanker / Passenger / Fishing / Military。
|
||||
- 如果某艘船尚未收到静态消息,聚合结果的船型仍可能是 `Other`,后续由 v5 船舶资料 enrichment 补齐。
|
||||
|
||||
## 凭证教程
|
||||
|
||||
文件:
|
||||
@@ -279,6 +303,7 @@ POST /api/v1/settings/credential-guides/{provider}/reset
|
||||
当前支持:
|
||||
|
||||
- `barentswatch`
|
||||
- `aisstream`
|
||||
|
||||
默认教程包含 BarentsWatch 官方 tutorial 地址:
|
||||
|
||||
|
||||
@@ -254,10 +254,10 @@ AIS 船只图层入口:
|
||||
船只图层当前负责:
|
||||
|
||||
- 请求 `/api/v1/visualization/geo/vessels`
|
||||
- 将 BarentsWatch AIS GeoJSON 转为地球局部坐标 marker 数据
|
||||
- 将聚合后的 AIS GeoJSON 转为地球局部坐标 marker 数据;请求默认不传 `limit`,后端和前端都不再默认裁剪到 5000 艘
|
||||
- 通过 `createInteractableLayer()` 注册 Interactable 图标层
|
||||
- 用按航向分桶的 `THREE.Points` 批量渲染普通船只 marker
|
||||
- 按船型映射颜色
|
||||
- 按船型映射颜色;`vessels.js` 会用 `vessel_type_name` 和 AIS `vessel_type` 数字共同归一化船型
|
||||
- 根据航行/停泊状态绘制三角形或圆点纹理
|
||||
- 用单点 `THREE.Points` overlay 承载 hover / locked glow
|
||||
- 支持 hover、lock、轨迹加载和视觉聚焦
|
||||
@@ -273,6 +273,10 @@ AIS 船只图层入口:
|
||||
|
||||
方向标准以 AIS `course / cog` 为准:从正北开始顺时针。普通态和交互态都通过同一套 canvas 旋转规则生成纹理,避免 hover 后箭头方向和原 marker 不一致。
|
||||
|
||||
船型展示也必须复用同一套归一化结果。`buildVesselMarkerData()` 会把后端的 `vessel_type_name` 和 AIS 数字类型码归一化为 `type`,用于 marker 颜色;同时生成 `vessel_type_display`,供详情卡、hover 简述和搜索结果显示。不要让详情卡直接只读原始 `vessel_type_name`,否则会出现 marker 已按 Cargo/Tanker 等颜色显示、卡片仍写 `Other` 的不一致。
|
||||
|
||||
AISStream 的 `PositionReport` 常带实时位置和 `MetaData.ShipName`,但船型通常来自低频 `ShipStaticData.Type`。后端会把 `MetaData.ShipName` 补进船名,并将类型码映射为 Cargo / Tanker / Passenger / Fishing / Military;仍缺失的船型需要等待静态 AIS 消息或后续船舶资料 enrichment,不能在前端凭颜色之外的信息臆造细分类。
|
||||
|
||||
船只 hover / click 也不再对渲染对象做 `raycaster.intersectObjects()`。`main.js` 只负责传入当前 Earth、camera、pointer 和命中半径,实际命中计算由 `interactable.js` 的图标层接口完成:
|
||||
|
||||
1. 拖动地球或惯性旋转时跳过 hover picking。
|
||||
|
||||
@@ -194,6 +194,7 @@
|
||||
| 船只 renderOrder | local `VESSEL_RENDER_ORDER` | `4.4` | 普通 marker 和交互 overlay |
|
||||
| 船只轨迹 renderOrder | `VESSEL_RENDER_ORDER - 0.1` | `4.3` | 低于船只 marker |
|
||||
| 船只点像素尺寸 | local `VESSEL_POINT_SIZE` | `34` | 普通 marker 与 hover / locked overlay 共享尺寸 |
|
||||
| 船只默认渲染上限 | `VESSEL_CONFIG.maxRenderedMarkers` | `0` | `0` 表示不在前端默认裁剪;正数才会给接口传 `limit` 并裁剪 marker |
|
||||
| 船只纹理画布尺寸 | local `VESSEL_ATLAS_CELL_SIZE` | `128` | canvas 点纹理 |
|
||||
| 航向分桶数 | local `VESSEL_COURSE_BINS` | `32` | moving 船只按 COG 分桶,降低 draw call 同时保留方向 |
|
||||
| 船只 hover 拾取节流 | local `VESSEL_HOVER_PICK_INTERVAL_MS` | `100` | `main.js` hover picking |
|
||||
@@ -206,6 +207,8 @@
|
||||
|
||||
AIS 船只普通态使用批量 `THREE.Points`,不是逐船 `THREE.Sprite`。航行船只保持三角形,停泊或低速船只保持圆点;普通态不带 glow,hover / locked 时在同一屏幕尺寸上叠加带 glow 的单点 overlay。AIS 航向按 `course / cog` 从正北顺时针解释,普通态和交互态必须使用同一套 canvas 旋转规则。
|
||||
|
||||
船型颜色和详情卡船型文本必须来自同一套归一化结果:`vessels.js` 同时读取后端 `vessel_type_name` 和 AIS 数字 `vessel_type`,先得到颜色用的 `type`,再生成 `vessel_type_display` 给详情卡、hover 和搜索使用。
|
||||
|
||||
## 算力中心
|
||||
|
||||
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
|
||||
|
||||
@@ -16,12 +16,13 @@
|
||||
## Current Version
|
||||
|
||||
- `main` 当前主线历史推导到:`0.16.5`
|
||||
- `dev` 当前开发分支历史推导到:`0.46.3`
|
||||
- `dev` 当前开发分支历史推导到:`0.47.0`
|
||||
|
||||
## Timeline
|
||||
|
||||
| Version | Type | Branch | Commit | Summary |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `0.47.0` | feature | `dev` | `pending` | 新增 AISStream WebSocket 船只采集器、多源 AIS 原始观测聚合、采集器状态配置、船型显示修正和文档规则解耦 |
|
||||
| `0.46.3` | bugfix | `dev` | `pending` | 优化 Starlink footprint 拖拽性能,避免旋转地球时重复重建覆盖网格,并恢复线缆点击呼吸动画 |
|
||||
| `0.46.2` | bugfix | `dev` | `pending` | 修复 Earth 启动加载顺序、图层 localStorage 恢复、国界线底图语义、媒体面板、船只轨迹和 Iridium footprint 显示问题,并补充 AIS 聚合计划 |
|
||||
| `0.46.1` | bugfix | `dev` | `pending` | 修复新增 Docs 技术文档未进前端白名单导致页面不可访问的问题,补齐英文文档并固化白名单/双语/裸文件标题检查 |
|
||||
|
||||
Reference in New Issue
Block a user