Compare commits
5 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ba54545ac7 | ||
|
|
9dafbf4f6e | ||
|
|
a87537e903 | ||
|
|
87594a95ff | ||
|
|
2da25376bd |
@@ -50,6 +50,25 @@ ls docs/technical/zh/ # 查看现有文档
|
||||
更新:docs/technical/zh/backend-datasources-api-performance.md — 补充并行化细节
|
||||
```
|
||||
|
||||
### Step 2.5 — 覆盖范围检查
|
||||
|
||||
写文档前必须按变更类型检查配套文档,不要只更新一篇专题文档:
|
||||
|
||||
- 用户可见流程变化:更新 `docs/technical/zh/manual.md`,通常也更新 `docs/technical/zh/quickstart.md`。
|
||||
- `manual.md`、`quickstart.md` 这类用户手册存在英文版时,同步更新 `docs/technical/en/...`,至少避免英文版与中文版互相矛盾。
|
||||
- 控制台页面职责、路由入口、表格/抽屉/设置页行为变化:更新 `docs/technical/zh/frontend-admin-frontend-context.md`。
|
||||
- Earth 前端行为、HUD、巡航、图层、图例、交互变化:更新 `docs/technical/zh/earth-frontend-context.md`。
|
||||
- 新增 Earth 图层、调整 `renderOrder`、半径/高度偏移、深度策略、拾取策略、legend mode、图层面板顺序或启动加载顺序:更新 `docs/technical/zh/earth-render-layer-order.md`。
|
||||
- Earth 图层视觉样式、颜色、图例符号语义变化:若影响样式索引,同步更新 `docs/technical/zh/earth-layer-style-reference.md`。
|
||||
- 采集器、数据源、凭证、设置页、连接检查、scheduler、后端 API 变化:更新相关后端文档,优先检查 `docs/technical/zh/backend-collectors.md` 和 datasource/settings 专题文档。
|
||||
- 如果某个旧 plan 的假设已经被当前实现推翻,在对应 `docs/plans/*.md` 增加现状修正或更新该段,不要让计划文档继续给出相反方向。
|
||||
- 新增 technical 文档后,如果需要被发现,更新 `docs/technical/zh/README.md`。
|
||||
- 对本次变更提取旧词做 stale search,例如旧 tab 名、旧路由职责、旧认证假设、改名前 UI 文案:
|
||||
|
||||
```bash
|
||||
rg -n "旧文案|旧路由职责|旧认证假设" docs/technical docs/plans
|
||||
```
|
||||
|
||||
### Step 3 — 写文档
|
||||
|
||||
遵循以下原则:
|
||||
|
||||
@@ -43,7 +43,20 @@ rg -n "class |def |function |export |router|@router|interface |type " <path>
|
||||
- `ops-planet-sh-startup.md`
|
||||
- `earth-bgp-context.md`
|
||||
|
||||
3. Write the doc in Chinese:
|
||||
3. Apply the documentation coverage checklist before writing:
|
||||
|
||||
- User-visible workflow changes must update `docs/technical/zh/manual.md` and usually `docs/technical/zh/quickstart.md`.
|
||||
- If an English counterpart exists for user-facing docs such as `manual.md` or `quickstart.md`, update `docs/technical/en/...` enough that it does not contradict the Chinese source.
|
||||
- Control console page responsibility changes must update `docs/technical/zh/frontend-admin-frontend-context.md`.
|
||||
- Earth frontend behavior changes must update `docs/technical/zh/earth-frontend-context.md`.
|
||||
- Earth layer additions, `renderOrder`, altitude/radius offsets, depth strategy, pointer picking, legend modes, or layer panel/startup ordering must update `docs/technical/zh/earth-render-layer-order.md`.
|
||||
- Earth layer visual style or legend symbol/color semantics should also update `docs/technical/zh/earth-layer-style-reference.md` when that reference is affected.
|
||||
- Collector, datasource, credential, settings, connectivity, scheduler, or API changes must update the relevant backend docs, especially `docs/technical/zh/backend-collectors.md` and any datasource/settings-specific doc.
|
||||
- When a change turns an old plan assumption into current behavior, update the relevant `docs/plans/*.md` with a status note instead of leaving contradictory instructions.
|
||||
- If adding a new technical document, add it to `docs/technical/zh/README.md` when it should be discoverable from the technical docs index.
|
||||
- Search docs for stale terms introduced by the change, for example old tab names, old route responsibilities, obsolete auth assumptions, or renamed UI labels.
|
||||
|
||||
4. Write the doc in Chinese:
|
||||
|
||||
- Write Chinese prose for `docs/technical/zh/`.
|
||||
- Keep technical identifiers, API paths, config keys, code symbols, and standard product names in English where appropriate.
|
||||
@@ -51,14 +64,14 @@ rg -n "class |def |function |export |router|@router|interface |type " <path>
|
||||
- Use fenced code blocks with language tags.
|
||||
- Use tables when comparing options or listing parameters.
|
||||
|
||||
4. Required content:
|
||||
5. Required content:
|
||||
|
||||
- Background/problem: what was wrong before and why the change was needed.
|
||||
- Core design decisions and rationale.
|
||||
- Key code snippets, preferably before/after or focused excerpts.
|
||||
- Related files and what each file contributes.
|
||||
|
||||
5. Verification:
|
||||
6. Verification:
|
||||
|
||||
- Read the completed doc and check that the reasoning is clear.
|
||||
- Verify important referenced paths exist.
|
||||
@@ -93,6 +106,12 @@ If checking many links, prefer deterministic extraction:
|
||||
rg -n "\]\(([^)]+)\)" docs/technical/zh/<doc>.md
|
||||
```
|
||||
|
||||
Also run focused stale-term searches derived from the change, for example:
|
||||
|
||||
```bash
|
||||
rg -n "old label|old route purpose|obsolete provider assumption" docs/technical docs/plans
|
||||
```
|
||||
|
||||
## Hard Constraints
|
||||
|
||||
- A file under `docs/technical/zh/` must not be an English source file copied as a placeholder.
|
||||
|
||||
13
.dockerignore
Normal file
13
.dockerignore
Normal file
@@ -0,0 +1,13 @@
|
||||
**
|
||||
|
||||
!pyproject.toml
|
||||
!uv.lock
|
||||
!aiprovider/
|
||||
!aiprovider/**
|
||||
|
||||
aiprovider/.env
|
||||
aiprovider/.env.*
|
||||
!aiprovider/.env.example
|
||||
**/__pycache__/
|
||||
**/*.pyc
|
||||
**/*.pyo
|
||||
1
TODO.md
1
TODO.md
@@ -23,6 +23,7 @@
|
||||
- [ ] 保持 Earth 当前这批纯个人偏好设置继续走本地持久化:`旋转模式`、HUD 面板显示/隐藏、`地形透明度` 暂不升级到后端系统设置,避免把设备级偏好过早做成全局配置
|
||||
- [ ] 如果后续明确需要“账号级同步 Earth 偏好”,再单独设计 `Earth user preferences`:优先按用户维度而不是全局系统设置保存,并规划 `localStorage -> backend` 的平滑迁移策略
|
||||
- [ ] 为 Planet / Earth 补一个可用的日志查看系统:先明确前后端/AI Provider/采集任务的日志入口、最近日志聚合、筛选与 tail 能力,再决定是先做脚本级统一入口还是控制台内置日志面板
|
||||
- [ ] 重写控制台 UI,逐步抛弃 Ant Design,建立自有组件体系,并统一采用 `tabler.io` / Tabler Icons 作为控制台主图标库
|
||||
- [ ] 把 Earth 态势新闻源从 [earth_news.py](/home/ray/dev/linkong/planet/backend/app/services/earth_news.py) 的硬编码列表抽成可配置目录,优先保持当前“实时聚合”链路不变,只先解决新闻源不可配置的问题
|
||||
- [ ] 为 Earth 态势新闻设计后续采集器化方案:明确新闻数据模型、去重策略、区域映射、过期清理和 Earth/AI 复用方式,再决定何时把新闻从实时抓取升级成正式 collector
|
||||
- [ ] 为 Earth 地球表面增加一层与基础纹理对齐的材质/纹理 overlay,并在同层叠加国界轮廓参考线;要求国界线与底图稳定对齐,且 hover 到国家轮廓时能高亮当前国家,便于校准地表和增强交互
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
# syntax=docker/dockerfile:1.7
|
||||
|
||||
ARG PYTHON_IMAGE=python:3.14-slim
|
||||
ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
|
||||
|
||||
@@ -18,9 +20,10 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY pyproject.toml uv.lock /app/
|
||||
RUN uv sync --frozen --no-dev
|
||||
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
uv sync --frozen --no-dev
|
||||
|
||||
COPY . /app
|
||||
COPY aiprovider /app/aiprovider
|
||||
|
||||
EXPOSE 8010
|
||||
|
||||
|
||||
@@ -12,6 +12,7 @@ from pydantic import BaseModel, Field
|
||||
import httpx
|
||||
|
||||
from app.core.target_schema_registry import get_target_schema, list_target_schemas
|
||||
from app.core.datasource_defaults import DEFAULT_DATASOURCES
|
||||
from app.db.session import get_db
|
||||
from app.models.user import User
|
||||
from app.models.datasource_config import DataSourceConfig
|
||||
@@ -29,6 +30,12 @@ from app.services.datasource_mapping import (
|
||||
redact_for_llm,
|
||||
stable_payload_hash,
|
||||
)
|
||||
from app.services.datasource_connectivity import (
|
||||
get_builtin_connection_status,
|
||||
save_connectivity_success,
|
||||
strip_connectivity_validation,
|
||||
test_builtin_connectivity,
|
||||
)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -73,6 +80,32 @@ class DataSourceConfigResponse(BaseModel):
|
||||
from_attributes = True
|
||||
|
||||
|
||||
def _is_builtin_config_name(name: str | None) -> bool:
|
||||
return bool(name and name in DEFAULT_DATASOURCES)
|
||||
|
||||
|
||||
async def _ensure_builtin_connection_verified(
|
||||
db: AsyncSession,
|
||||
config_data: DataSourceConfigCreate,
|
||||
) -> None:
|
||||
if not _is_builtin_config_name(config_data.name):
|
||||
return
|
||||
|
||||
status_result = await get_builtin_connection_status(
|
||||
db,
|
||||
config_data.name,
|
||||
config_data.endpoint,
|
||||
config_data.auth_type,
|
||||
config_data.headers,
|
||||
config_data.config,
|
||||
)
|
||||
if not status_result.get("connected"):
|
||||
raise HTTPException(
|
||||
status_code=400,
|
||||
detail=status_result.get("message") or "请先完成连接验证,再保存内置采集器配置。",
|
||||
)
|
||||
|
||||
|
||||
class CustomSampleRequest(BaseModel):
|
||||
datasource_config_id: Optional[int] = None
|
||||
config: Optional[DataSourceConfigCreate] = None
|
||||
@@ -312,6 +345,47 @@ async def list_configs(
|
||||
}
|
||||
|
||||
|
||||
@router.get("/configs/all")
|
||||
async def list_all_datasources(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""List all data sources: YAML defaults + DB overrides"""
|
||||
from app.core.data_sources import COLLECTOR_URL_KEYS, get_data_sources_config
|
||||
|
||||
config = get_data_sources_config()
|
||||
|
||||
db_query = await db.execute(select(DataSourceConfig))
|
||||
db_configs = {c.name: c for c in db_query.scalars().all()}
|
||||
|
||||
result = []
|
||||
for name, yaml_key in COLLECTOR_URL_KEYS.items():
|
||||
yaml_url = config.get_yaml_url(name)
|
||||
db_config = db_configs.get(name)
|
||||
|
||||
result.append(
|
||||
{
|
||||
"name": name,
|
||||
"default_url": yaml_url,
|
||||
"endpoint": db_config.endpoint if db_config else yaml_url,
|
||||
"is_overridden": db_config is not None and db_config.endpoint != yaml_url
|
||||
if yaml_url
|
||||
else db_config is not None,
|
||||
"is_active": db_config.is_active if db_config else True,
|
||||
"source_type": db_config.source_type if db_config else "http",
|
||||
"auth_type": db_config.auth_type if db_config else "none",
|
||||
"headers": db_config.headers if db_config else {},
|
||||
"config": strip_connectivity_validation(db_config.config if db_config else {}),
|
||||
"config_id": db_config.id if db_config else None,
|
||||
"description": db_config.description
|
||||
if db_config
|
||||
else f"Data source from YAML: {yaml_key}",
|
||||
}
|
||||
)
|
||||
|
||||
return {"total": len(result), "data": result}
|
||||
|
||||
|
||||
@router.get("/configs/{config_id}")
|
||||
async def get_config(
|
||||
config_id: int,
|
||||
@@ -356,7 +430,7 @@ async def create_config(
|
||||
auth_type=config_data.auth_type,
|
||||
auth_config=config_data.auth_config,
|
||||
headers=config_data.headers,
|
||||
config=config_data.config,
|
||||
config=strip_connectivity_validation(config_data.config),
|
||||
)
|
||||
|
||||
db.add(config)
|
||||
@@ -388,6 +462,8 @@ async def update_config(
|
||||
|
||||
update_data = config_data.model_dump(exclude_unset=True)
|
||||
for field, value in update_data.items():
|
||||
if field == "config":
|
||||
value = strip_connectivity_validation(value)
|
||||
setattr(config, field, value)
|
||||
|
||||
await db.commit()
|
||||
@@ -490,41 +566,61 @@ async def test_new_config(
|
||||
}
|
||||
|
||||
|
||||
@router.get("/configs/all")
|
||||
async def list_all_datasources(
|
||||
@router.post("/configs/builtin/connection-status")
|
||||
async def get_builtin_config_connection_status(
|
||||
config_data: DataSourceConfigCreate,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""List all data sources: YAML defaults + DB overrides"""
|
||||
from app.core.data_sources import COLLECTOR_URL_KEYS, get_data_sources_config
|
||||
if not _is_builtin_config_name(config_data.name):
|
||||
raise HTTPException(status_code=400, detail="Only built-in datasource configs are supported.")
|
||||
|
||||
config = get_data_sources_config()
|
||||
return await get_builtin_connection_status(
|
||||
db,
|
||||
config_data.name,
|
||||
config_data.endpoint,
|
||||
config_data.auth_type,
|
||||
config_data.headers,
|
||||
config_data.config,
|
||||
)
|
||||
|
||||
db_query = await db.execute(select(DataSourceConfig))
|
||||
db_configs = {c.name: c for c in db_query.scalars().all()}
|
||||
|
||||
result = []
|
||||
for name, yaml_key in COLLECTOR_URL_KEYS.items():
|
||||
yaml_url = config.get_yaml_url(name)
|
||||
db_config = db_configs.get(name)
|
||||
@router.post("/configs/builtin/connect")
|
||||
async def connect_builtin_config(
|
||||
config_data: DataSourceConfigCreate,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
if not _is_builtin_config_name(config_data.name):
|
||||
raise HTTPException(status_code=400, detail="Only built-in datasource configs are supported.")
|
||||
|
||||
result.append(
|
||||
{
|
||||
"name": name,
|
||||
"default_url": yaml_url,
|
||||
"endpoint": db_config.endpoint if db_config else yaml_url,
|
||||
"is_overridden": db_config is not None and db_config.endpoint != yaml_url
|
||||
if yaml_url
|
||||
else db_config is not None,
|
||||
"is_active": db_config.is_active if db_config else True,
|
||||
"source_type": db_config.source_type if db_config else "http",
|
||||
"description": db_config.description
|
||||
if db_config
|
||||
else f"Data source from YAML: {yaml_key}",
|
||||
}
|
||||
result = await test_builtin_connectivity(
|
||||
config_data.name,
|
||||
config_data.endpoint,
|
||||
config_data.auth_type,
|
||||
config_data.headers,
|
||||
config_data.config,
|
||||
db,
|
||||
)
|
||||
if result.get("success") and result.get("checksum"):
|
||||
validation = await save_connectivity_success(
|
||||
db,
|
||||
config_data.name,
|
||||
result["checksum"],
|
||||
result,
|
||||
connected_by="connection_button",
|
||||
)
|
||||
await db.commit()
|
||||
return {
|
||||
**result,
|
||||
"connected": True,
|
||||
"validation": validation,
|
||||
}
|
||||
|
||||
return {"total": len(result), "data": result}
|
||||
return {
|
||||
**result,
|
||||
"connected": False,
|
||||
}
|
||||
|
||||
|
||||
@router.post("/custom/sample")
|
||||
|
||||
@@ -398,6 +398,11 @@ async def list_datasources(
|
||||
"task_id": running_task.id if running_task else None,
|
||||
"progress": running_task.progress if running_task else None,
|
||||
"phase": running_task.phase if running_task else None,
|
||||
"phase_progress": running_task.phase_progress if running_task else None,
|
||||
"phase_message": running_task.phase_message if running_task else None,
|
||||
"phase_current": running_task.phase_current if running_task else None,
|
||||
"phase_total": running_task.phase_total if running_task else None,
|
||||
"phase_unit": running_task.phase_unit if running_task else None,
|
||||
"records_processed": running_task.records_processed if running_task else None,
|
||||
"total_records": running_task.total_records if running_task else None,
|
||||
}
|
||||
@@ -626,6 +631,11 @@ async def trigger_datasource(
|
||||
"message": "当前采集任务尚未完成,重新触发会丢失本次未完成进度。是否强制重新采集?",
|
||||
"task_id": running_task.id,
|
||||
"phase": running_task.phase,
|
||||
"phase_progress": running_task.phase_progress,
|
||||
"phase_message": running_task.phase_message,
|
||||
"phase_current": running_task.phase_current,
|
||||
"phase_total": running_task.phase_total,
|
||||
"phase_unit": running_task.phase_unit,
|
||||
"progress": running_task.progress,
|
||||
"records_processed": running_task.records_processed,
|
||||
"total_records": running_task.total_records,
|
||||
@@ -709,13 +719,29 @@ async def get_task_status(
|
||||
task = await get_running_task(db, datasource.id)
|
||||
|
||||
if not task:
|
||||
return {"is_running": False, "task_id": None, "progress": None, "phase": None, "status": "idle"}
|
||||
return {
|
||||
"is_running": False,
|
||||
"task_id": None,
|
||||
"progress": None,
|
||||
"phase": None,
|
||||
"phase_progress": None,
|
||||
"phase_message": None,
|
||||
"phase_current": None,
|
||||
"phase_total": None,
|
||||
"phase_unit": None,
|
||||
"status": "idle",
|
||||
}
|
||||
|
||||
return {
|
||||
"is_running": task.status == "running",
|
||||
"task_id": task.id,
|
||||
"progress": task.progress,
|
||||
"phase": task.phase,
|
||||
"phase_progress": task.phase_progress,
|
||||
"phase_message": task.phase_message,
|
||||
"phase_current": task.phase_current,
|
||||
"phase_total": task.phase_total,
|
||||
"phase_unit": task.phase_unit,
|
||||
"records_processed": task.records_processed,
|
||||
"total_records": task.total_records,
|
||||
"status": task.status,
|
||||
|
||||
@@ -17,6 +17,23 @@ from app.models.datasource import DataSource
|
||||
from app.models.datasource_config import DataSourceConfig
|
||||
from app.models.system_setting import SystemSetting
|
||||
from app.models.user import User
|
||||
from app.services.barentswatch import (
|
||||
BarentsWatchConfig,
|
||||
check_barentswatch_config,
|
||||
check_barentswatch_connectivity,
|
||||
get_barentswatch_datasource_record,
|
||||
resolve_barentswatch_config,
|
||||
)
|
||||
from app.services.credential_guides import (
|
||||
generate_credential_guide,
|
||||
get_credential_guide,
|
||||
reset_credential_guide,
|
||||
)
|
||||
from app.services.datasource_connectivity import (
|
||||
build_builtin_connectivity_checksum,
|
||||
save_connectivity_success,
|
||||
)
|
||||
from app.services.ai_client import AIProviderClient, get_ai_provider_client
|
||||
from app.services.llm_provider_catalog import (
|
||||
get_fallback_llm_provider_preset,
|
||||
list_fallback_llm_provider_presets,
|
||||
@@ -245,12 +262,7 @@ async def get_runtime_ai_provider_config(db: AsyncSession) -> dict:
|
||||
|
||||
|
||||
async def get_barentswatch_config_record(db: AsyncSession) -> Optional[DataSourceConfig]:
|
||||
result = await db.execute(
|
||||
select(DataSourceConfig)
|
||||
.where(DataSourceConfig.name == "barentswatch_vessels")
|
||||
.where(DataSourceConfig.is_active.is_(True))
|
||||
)
|
||||
return result.scalar_one_or_none()
|
||||
return await get_barentswatch_datasource_record(db)
|
||||
|
||||
|
||||
async def serialize_external_integrations(db: AsyncSession) -> dict:
|
||||
@@ -258,9 +270,9 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
|
||||
runtime_setting = await get_setting_record(db, "external_integrations")
|
||||
display_llm_config = ai_config["llm_config"] or DEFAULT_SETTINGS["external_integrations"]["ai_provider"]
|
||||
barentswatch_record = await get_barentswatch_config_record(db)
|
||||
yaml_config = get_data_sources_config()
|
||||
barentswatch_auth = barentswatch_record.auth_config if barentswatch_record else {}
|
||||
barentswatch_auth = barentswatch_auth or {}
|
||||
resolved_barentswatch = await resolve_barentswatch_config(db)
|
||||
return {
|
||||
"ai_provider": {
|
||||
"service_url": ai_config["service_url"],
|
||||
@@ -277,14 +289,12 @@ async def serialize_external_integrations(db: AsyncSession) -> dict:
|
||||
"source": "runtime" if runtime_setting else "env",
|
||||
},
|
||||
"barentswatch": {
|
||||
"endpoint": (
|
||||
barentswatch_record.endpoint
|
||||
if barentswatch_record and barentswatch_record.endpoint
|
||||
else yaml_config.get_yaml_url("barentswatch_vessels")
|
||||
"endpoint": resolved_barentswatch.endpoint,
|
||||
"client_id": barentswatch_auth.get("client_id") or resolved_barentswatch.client_id,
|
||||
"client_secret": _mask_secret(
|
||||
barentswatch_auth.get("client_secret") or resolved_barentswatch.client_secret
|
||||
),
|
||||
"client_id": barentswatch_auth.get("client_id") or "",
|
||||
"client_secret": _mask_secret(barentswatch_auth.get("client_secret")),
|
||||
"source": "datasource_config" if barentswatch_record else "default",
|
||||
"source": resolved_barentswatch.credential_source,
|
||||
},
|
||||
}
|
||||
|
||||
@@ -328,7 +338,7 @@ async def save_external_integrations_payload(
|
||||
description="BarentsWatch Live AIS credentials",
|
||||
source_type="api",
|
||||
endpoint=update.barentswatch.endpoint.strip() or default_endpoint,
|
||||
auth_type="oauth_client_credentials",
|
||||
auth_type="oauth_client",
|
||||
auth_config={},
|
||||
headers={},
|
||||
config={},
|
||||
@@ -343,7 +353,7 @@ async def save_external_integrations_payload(
|
||||
current_auth["client_secret"] = update.barentswatch.client_secret
|
||||
current_auth["client_id"] = update.barentswatch.client_id.strip()
|
||||
barentswatch_record.endpoint = update.barentswatch.endpoint.strip() or default_endpoint
|
||||
barentswatch_record.auth_type = "oauth_client_credentials"
|
||||
barentswatch_record.auth_type = "oauth_client"
|
||||
barentswatch_record.auth_config = current_auth
|
||||
await db.commit()
|
||||
|
||||
@@ -461,6 +471,95 @@ async def get_external_integrations(
|
||||
return {"integrations": await serialize_external_integrations(db)}
|
||||
|
||||
|
||||
@router.get("/integrations/barentswatch/connectivity")
|
||||
async def get_barentswatch_connectivity(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
return await check_barentswatch_connectivity(db)
|
||||
|
||||
|
||||
@router.post("/integrations/barentswatch/connect")
|
||||
async def connect_barentswatch_integration(
|
||||
payload: BarentsWatchIntegrationUpdate,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
current = await resolve_barentswatch_config(db)
|
||||
config = BarentsWatchConfig(
|
||||
endpoint=payload.endpoint.strip() or current.endpoint,
|
||||
client_id=payload.client_id.strip() or current.client_id,
|
||||
client_secret=(
|
||||
""
|
||||
if payload.clear_client_secret
|
||||
else payload.client_secret or current.client_secret
|
||||
),
|
||||
credential_source="draft",
|
||||
endpoint_source="draft",
|
||||
)
|
||||
result = await check_barentswatch_config(config)
|
||||
if result.get("success"):
|
||||
checksum, _context = await build_builtin_connectivity_checksum(
|
||||
"barentswatch_vessels",
|
||||
config.endpoint,
|
||||
"none",
|
||||
{},
|
||||
{},
|
||||
db,
|
||||
credential_override={
|
||||
"client_id": config.client_id,
|
||||
"client_secret": config.client_secret,
|
||||
},
|
||||
)
|
||||
validation = await save_connectivity_success(
|
||||
db,
|
||||
"barentswatch_vessels",
|
||||
checksum,
|
||||
result,
|
||||
connected_by="connection_button",
|
||||
)
|
||||
await db.commit()
|
||||
return {**result, "connected": True, "validation": validation}
|
||||
return {**result, "connected": False}
|
||||
|
||||
|
||||
@router.get("/credential-guides/{provider}")
|
||||
async def read_credential_guide(
|
||||
provider: str,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
try:
|
||||
return {"guide": await get_credential_guide(db, provider)}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/credential-guides/{provider}/generate")
|
||||
async def generate_provider_credential_guide(
|
||||
provider: str,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
ai_client: AIProviderClient = Depends(get_ai_provider_client),
|
||||
):
|
||||
try:
|
||||
return {"guide": await generate_credential_guide(db, provider, ai_client)}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.post("/credential-guides/{provider}/reset")
|
||||
async def reset_provider_credential_guide(
|
||||
provider: str,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
try:
|
||||
return {"guide": await reset_credential_guide(db, provider)}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@router.get("/integrations/ai-provider/presets")
|
||||
async def get_ai_provider_presets(
|
||||
current_user: User = Depends(get_current_user),
|
||||
|
||||
@@ -27,7 +27,9 @@ async def list_tasks(
|
||||
offset = (page - 1) * page_size
|
||||
query = """
|
||||
SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
|
||||
ct.started_at, ct.completed_at, ct.records_processed, ct.error_message
|
||||
ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
|
||||
ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
|
||||
ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
|
||||
FROM collection_tasks ct
|
||||
JOIN data_sources ds ON ct.datasource_id = ds.id
|
||||
WHERE 1=1
|
||||
@@ -66,6 +68,14 @@ async def list_tasks(
|
||||
"completed_at": to_iso8601_utc(t[5]),
|
||||
"records_processed": t[6],
|
||||
"error_message": t[7],
|
||||
"phase": t[8],
|
||||
"phase_progress": t[9],
|
||||
"phase_message": t[10],
|
||||
"phase_current": t[11],
|
||||
"phase_total": t[12],
|
||||
"phase_unit": t[13],
|
||||
"total_records": t[14],
|
||||
"progress": t[15],
|
||||
}
|
||||
for t in tasks
|
||||
],
|
||||
@@ -81,7 +91,9 @@ async def get_task(
|
||||
result = await db.execute(
|
||||
text("""
|
||||
SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
|
||||
ct.started_at, ct.completed_at, ct.records_processed, ct.error_message
|
||||
ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
|
||||
ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
|
||||
ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
|
||||
FROM collection_tasks ct
|
||||
JOIN data_sources ds ON ct.datasource_id = ds.id
|
||||
WHERE ct.id = :id
|
||||
@@ -105,6 +117,14 @@ async def get_task(
|
||||
"completed_at": to_iso8601_utc(task[5]),
|
||||
"records_processed": task[6],
|
||||
"error_message": task[7],
|
||||
"phase": task[8],
|
||||
"phase_progress": task[9],
|
||||
"phase_message": task[10],
|
||||
"phase_current": task[11],
|
||||
"phase_total": task[12],
|
||||
"phase_unit": task[13],
|
||||
"total_records": task[14],
|
||||
"progress": task[15],
|
||||
}
|
||||
|
||||
|
||||
|
||||
@@ -146,7 +146,12 @@ async def init_db():
|
||||
text(
|
||||
"""
|
||||
ALTER TABLE collection_tasks
|
||||
ADD COLUMN IF NOT EXISTS phase VARCHAR(30) DEFAULT 'queued'
|
||||
ADD COLUMN IF NOT EXISTS phase VARCHAR(30) DEFAULT 'queued',
|
||||
ADD COLUMN IF NOT EXISTS phase_progress DOUBLE PRECISION,
|
||||
ADD COLUMN IF NOT EXISTS phase_message VARCHAR(255),
|
||||
ADD COLUMN IF NOT EXISTS phase_current BIGINT,
|
||||
ADD COLUMN IF NOT EXISTS phase_total BIGINT,
|
||||
ADD COLUMN IF NOT EXISTS phase_unit VARCHAR(30)
|
||||
"""
|
||||
)
|
||||
)
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
"""Collection Task model"""
|
||||
|
||||
from sqlalchemy import Column, DateTime, Integer, String, Text, Float
|
||||
from sqlalchemy import BigInteger, Column, DateTime, Integer, String, Text, Float
|
||||
from sqlalchemy.sql import func
|
||||
|
||||
from app.db.session import Base
|
||||
@@ -13,6 +13,11 @@ class CollectionTask(Base):
|
||||
datasource_id = Column(Integer, nullable=False, index=True)
|
||||
status = Column(String(20), nullable=False) # pending, running, success, failed, cancelled
|
||||
phase = Column(String(30), default="queued")
|
||||
phase_progress = Column(Float)
|
||||
phase_message = Column(String(255))
|
||||
phase_current = Column(BigInteger)
|
||||
phase_total = Column(BigInteger)
|
||||
phase_unit = Column(String(30))
|
||||
started_at = Column(DateTime(timezone=True))
|
||||
completed_at = Column(DateTime(timezone=True))
|
||||
records_processed = Column(Integer, default=0)
|
||||
|
||||
209
backend/app/services/barentswatch.py
Normal file
209
backend/app/services/barentswatch.py
Normal file
@@ -0,0 +1,209 @@
|
||||
"""BarentsWatch AIS credential resolution and connectivity checks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import shlex
|
||||
from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.data_sources import get_data_sources_config
|
||||
from app.models.datasource_config import DataSourceConfig
|
||||
|
||||
|
||||
BARENTSWATCH_LATEST_URL = "https://live.ais.barentswatch.no/v1/latest/combined"
|
||||
BARENTSWATCH_TOKEN_URL = "https://id.barentswatch.no/connect/token"
|
||||
BARENTSWATCH_DATASOURCE_NAME = "barentswatch_vessels"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class BarentsWatchConfig:
|
||||
endpoint: str
|
||||
client_id: str
|
||||
client_secret: str
|
||||
credential_source: str
|
||||
endpoint_source: str
|
||||
|
||||
|
||||
def _read_zshrc_env(path: Path | None = None) -> dict[str, str]:
|
||||
zshrc_path = path or Path.home() / ".zshrc"
|
||||
if not zshrc_path.exists():
|
||||
return {}
|
||||
|
||||
values: dict[str, str] = {}
|
||||
for raw_line in zshrc_path.read_text(encoding="utf-8", errors="ignore").splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
if line.startswith("export "):
|
||||
line = line[len("export ") :].strip()
|
||||
if "=" not in line:
|
||||
continue
|
||||
|
||||
key, value = line.split("=", 1)
|
||||
key = key.strip()
|
||||
if not key or not key.replace("_", "").isalnum() or not key[0].isalpha():
|
||||
continue
|
||||
|
||||
try:
|
||||
parsed = shlex.split(value, comments=True, posix=True)
|
||||
except ValueError:
|
||||
parsed = [value.strip().strip("'\"")]
|
||||
if parsed:
|
||||
values[key] = parsed[0]
|
||||
return values
|
||||
|
||||
|
||||
def _first_env_value(zshrc_env: dict[str, str], *keys: str) -> tuple[str, str]:
|
||||
for key in keys:
|
||||
value = os.getenv(key)
|
||||
if value:
|
||||
return value, "environment"
|
||||
for key in keys:
|
||||
value = zshrc_env.get(key)
|
||||
if value:
|
||||
return value, "~/.zshrc"
|
||||
return "", ""
|
||||
|
||||
|
||||
async def get_barentswatch_datasource_record(db: AsyncSession) -> DataSourceConfig | None:
|
||||
result = await db.execute(
|
||||
select(DataSourceConfig)
|
||||
.where(DataSourceConfig.name == BARENTSWATCH_DATASOURCE_NAME)
|
||||
.where(DataSourceConfig.is_active.is_(True))
|
||||
)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
|
||||
async def resolve_barentswatch_config(db: AsyncSession | None = None) -> BarentsWatchConfig:
|
||||
record = await get_barentswatch_datasource_record(db) if db else None
|
||||
auth_config = dict(record.auth_config or {}) if record else {}
|
||||
config = dict(record.config or {}) if record else {}
|
||||
zshrc_env = _read_zshrc_env()
|
||||
|
||||
env_client_id, env_source = _first_env_value(
|
||||
zshrc_env,
|
||||
"BARENTSWATCH_CLIENT_ID",
|
||||
"BARRENTSWATCH_CLIENT_ID",
|
||||
)
|
||||
env_client_secret, secret_env_source = _first_env_value(
|
||||
zshrc_env,
|
||||
"BARENTSWATCH_CLIENT_SECRET",
|
||||
"BARRENTSWATCH_CLIENT_SECRET",
|
||||
)
|
||||
client_id = auth_config.get("client_id") or config.get("client_id") or env_client_id
|
||||
client_secret = (
|
||||
auth_config.get("client_secret") or config.get("client_secret") or env_client_secret
|
||||
)
|
||||
|
||||
credential_source = ""
|
||||
if auth_config.get("client_id") or auth_config.get("client_secret"):
|
||||
credential_source = "datasource_config"
|
||||
elif config.get("client_id") or config.get("client_secret"):
|
||||
credential_source = "datasource_runtime_config"
|
||||
elif env_source or secret_env_source:
|
||||
credential_source = env_source or secret_env_source
|
||||
|
||||
yaml_endpoint = get_data_sources_config().get_yaml_url(BARENTSWATCH_DATASOURCE_NAME)
|
||||
endpoint = record.endpoint if record and record.endpoint else yaml_endpoint
|
||||
return BarentsWatchConfig(
|
||||
endpoint=endpoint or BARENTSWATCH_LATEST_URL,
|
||||
client_id=str(client_id or ""),
|
||||
client_secret=str(client_secret or ""),
|
||||
credential_source=credential_source or "missing",
|
||||
endpoint_source="datasource_config" if record and record.endpoint else "default",
|
||||
)
|
||||
|
||||
|
||||
async def fetch_barentswatch_access_token(
|
||||
client: httpx.AsyncClient,
|
||||
config: BarentsWatchConfig,
|
||||
) -> str | None:
|
||||
if not config.client_id or not config.client_secret:
|
||||
return None
|
||||
|
||||
response = await client.post(
|
||||
BARENTSWATCH_TOKEN_URL,
|
||||
data={
|
||||
"client_id": config.client_id,
|
||||
"client_secret": config.client_secret,
|
||||
"scope": "ais",
|
||||
"grant_type": "client_credentials",
|
||||
},
|
||||
headers={"Content-Type": "application/x-www-form-urlencoded"},
|
||||
)
|
||||
response.raise_for_status()
|
||||
payload = response.json()
|
||||
token = payload.get("access_token")
|
||||
return str(token) if token else None
|
||||
|
||||
|
||||
async def check_barentswatch_connectivity(db: AsyncSession) -> dict[str, Any]:
|
||||
config = await resolve_barentswatch_config(db)
|
||||
return await check_barentswatch_config(config)
|
||||
|
||||
|
||||
async def check_barentswatch_config(config: BarentsWatchConfig) -> dict[str, Any]:
|
||||
if not config.client_id or not config.client_secret:
|
||||
return {
|
||||
"success": False,
|
||||
"stage": "credentials",
|
||||
"message": "未找到 BarentsWatch client id/client secret,请先配置采集器凭证。",
|
||||
"endpoint": config.endpoint,
|
||||
"credential_source": config.credential_source,
|
||||
"settings_tab": "collector_credentials",
|
||||
}
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=20.0) as client:
|
||||
token = await fetch_barentswatch_access_token(client, config)
|
||||
if not token:
|
||||
return {
|
||||
"success": False,
|
||||
"stage": "token",
|
||||
"message": "BarentsWatch token 响应中没有 access_token,请检查凭证。",
|
||||
"endpoint": config.endpoint,
|
||||
"credential_source": config.credential_source,
|
||||
"settings_tab": "collector_credentials",
|
||||
}
|
||||
|
||||
async with client.stream(
|
||||
"GET",
|
||||
config.endpoint,
|
||||
headers={"Authorization": f"Bearer {token}"},
|
||||
) as response:
|
||||
response.raise_for_status()
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"stage": "endpoint",
|
||||
"message": "BarentsWatch AIS token 和数据接口均可连通。",
|
||||
"endpoint": config.endpoint,
|
||||
"credential_source": config.credential_source,
|
||||
"endpoint_source": config.endpoint_source,
|
||||
}
|
||||
except httpx.HTTPStatusError as exc:
|
||||
status_code = exc.response.status_code
|
||||
stage = "token" if str(exc.request.url) == BARENTSWATCH_TOKEN_URL else "endpoint"
|
||||
return {
|
||||
"success": False,
|
||||
"stage": stage,
|
||||
"message": f"BarentsWatch {stage} 请求返回 HTTP {status_code},请检查凭证或接口地址。",
|
||||
"endpoint": config.endpoint,
|
||||
"credential_source": config.credential_source,
|
||||
"settings_tab": "collector_credentials",
|
||||
}
|
||||
except httpx.HTTPError as exc:
|
||||
return {
|
||||
"success": False,
|
||||
"stage": "network",
|
||||
"message": f"BarentsWatch 链路检查失败:{exc.__class__.__name__}",
|
||||
"endpoint": config.endpoint,
|
||||
"credential_source": config.credential_source,
|
||||
"settings_tab": "collector_credentials",
|
||||
}
|
||||
@@ -54,6 +54,11 @@ class BaseCollector(ABC):
|
||||
"task_id": self._current_task.id,
|
||||
"status": self._current_task.status,
|
||||
"phase": self._current_task.phase,
|
||||
"phase_progress": self._current_task.phase_progress,
|
||||
"phase_message": self._current_task.phase_message,
|
||||
"phase_current": self._current_task.phase_current,
|
||||
"phase_total": self._current_task.phase_total,
|
||||
"phase_unit": self._current_task.phase_unit,
|
||||
"progress": progress,
|
||||
"records_processed": self._current_task.records_processed,
|
||||
"total_records": self._current_task.total_records,
|
||||
@@ -80,12 +85,52 @@ class BaseCollector(ABC):
|
||||
|
||||
await self._publish_task_update(force=force)
|
||||
|
||||
async def set_phase(self, phase: str):
|
||||
async def set_phase(self, phase: str, *, message: str | None = None, reset_progress: bool = True):
|
||||
if self._current_task and self._db_session:
|
||||
self._current_task.phase = phase
|
||||
self._current_task.phase_message = message
|
||||
if reset_progress:
|
||||
self._current_task.phase_progress = None
|
||||
self._current_task.phase_current = None
|
||||
self._current_task.phase_total = None
|
||||
self._current_task.phase_unit = None
|
||||
await self._db_session.commit()
|
||||
await self._publish_task_update(force=True)
|
||||
|
||||
async def update_phase_progress(
|
||||
self,
|
||||
*,
|
||||
current: int | None = None,
|
||||
total: int | None = None,
|
||||
unit: str | None = None,
|
||||
message: str | None = None,
|
||||
progress: float | None = None,
|
||||
commit: bool = False,
|
||||
force: bool = False,
|
||||
):
|
||||
"""Update progress for the current phase without changing task totals."""
|
||||
if not self._current_task or not self._db_session:
|
||||
return
|
||||
|
||||
if progress is None and current is not None and total and total > 0:
|
||||
progress = (current / total) * 100
|
||||
|
||||
if progress is not None:
|
||||
self._current_task.phase_progress = max(0.0, min(float(progress), 100.0))
|
||||
if current is not None:
|
||||
self._current_task.phase_current = max(0, int(current))
|
||||
if total is not None:
|
||||
self._current_task.phase_total = max(0, int(total))
|
||||
if unit is not None:
|
||||
self._current_task.phase_unit = unit
|
||||
if message is not None:
|
||||
self._current_task.phase_message = message
|
||||
|
||||
if commit:
|
||||
await self._db_session.commit()
|
||||
|
||||
await self._publish_task_update(force=force)
|
||||
|
||||
@abstractmethod
|
||||
async def fetch(self) -> List[Dict[str, Any]]:
|
||||
"""Fetch raw data from source"""
|
||||
@@ -251,7 +296,7 @@ class BaseCollector(ABC):
|
||||
await self._publish_task_update(force=True)
|
||||
|
||||
try:
|
||||
await self.set_phase("fetching")
|
||||
await self.set_phase("fetching", message="正在拉取原始数据")
|
||||
raw_data = await self.fetch()
|
||||
task.total_records = len(raw_data)
|
||||
await db.commit()
|
||||
@@ -260,15 +305,20 @@ class BaseCollector(ABC):
|
||||
if self.fail_on_empty and not raw_data:
|
||||
raise RuntimeError(f"Collector {self.name} returned no data")
|
||||
|
||||
await self.set_phase("transforming")
|
||||
await self.set_phase("transforming", message="正在转换采集数据")
|
||||
data = self.transform(raw_data)
|
||||
snapshot_id = await self._create_snapshot(db, task_id, data, start_time)
|
||||
|
||||
await self.set_phase("saving")
|
||||
await self.set_phase("saving", message="正在保存采集数据")
|
||||
records_count = await self._save_data(db, data, task_id=task_id, snapshot_id=snapshot_id)
|
||||
|
||||
task.status = "success"
|
||||
task.phase = "completed"
|
||||
task.phase_progress = 100.0
|
||||
task.phase_message = "采集完成"
|
||||
task.phase_current = records_count
|
||||
task.phase_total = records_count
|
||||
task.phase_unit = "records"
|
||||
task.records_processed = records_count
|
||||
task.progress = 100.0
|
||||
task.completed_at = datetime.now(UTC)
|
||||
@@ -285,6 +335,7 @@ class BaseCollector(ABC):
|
||||
await db.rollback()
|
||||
task.status = "cancelled"
|
||||
task.phase = "cancelled"
|
||||
task.phase_message = "采集已取消"
|
||||
task.error_message = "Collection cancelled by operator and rolled back"
|
||||
task.completed_at = datetime.now(UTC)
|
||||
if snapshot_id is not None:
|
||||
@@ -301,6 +352,7 @@ class BaseCollector(ABC):
|
||||
await db.rollback()
|
||||
task.status = "failed"
|
||||
task.phase = "failed"
|
||||
task.phase_message = str(e)
|
||||
task.error_message = str(e)
|
||||
task.completed_at = datetime.now(UTC)
|
||||
if snapshot_id is not None:
|
||||
|
||||
@@ -108,6 +108,11 @@ class IPtoASNPrefixGeoCollector(BaseCollector):
|
||||
self._current_task.total_records = total_expected
|
||||
self._current_task.records_processed = 0
|
||||
self._current_task.progress = 0.0
|
||||
self._current_task.phase_progress = 0.0
|
||||
self._current_task.phase_message = "正在下载 IPtoASN 数据"
|
||||
self._current_task.phase_current = 0
|
||||
self._current_task.phase_total = total_expected
|
||||
self._current_task.phase_unit = "bytes"
|
||||
await self._db_session.commit()
|
||||
await self._publish_task_update(force=True)
|
||||
|
||||
@@ -135,7 +140,14 @@ class IPtoASNPrefixGeoCollector(BaseCollector):
|
||||
return
|
||||
last_emit["value"] = aggregated
|
||||
last_emit["t"] = now
|
||||
await self.update_progress(min(aggregated, total_expected), commit=True)
|
||||
current = min(aggregated, total_expected)
|
||||
await self.update_phase_progress(
|
||||
current=current,
|
||||
total=total_expected,
|
||||
unit="bytes",
|
||||
message="正在下载 IPtoASN 数据",
|
||||
)
|
||||
await self.update_progress(current, commit=True)
|
||||
|
||||
batches = await asyncio.gather(
|
||||
*(
|
||||
@@ -148,6 +160,12 @@ class IPtoASNPrefixGeoCollector(BaseCollector):
|
||||
)
|
||||
)
|
||||
if total_expected > 0:
|
||||
await self.update_phase_progress(
|
||||
current=total_expected,
|
||||
total=total_expected,
|
||||
unit="bytes",
|
||||
message="IPtoASN 数据下载完成",
|
||||
)
|
||||
await self.update_progress(total_expected, commit=True, force=True)
|
||||
|
||||
rows: list[dict[str, Any]] = []
|
||||
|
||||
@@ -39,12 +39,23 @@ class NRODelegatedPrefixGeoCollector(BaseCollector):
|
||||
self._current_task.total_records = total_expected
|
||||
self._current_task.records_processed = 0
|
||||
self._current_task.progress = 0.0
|
||||
self._current_task.phase_progress = 0.0
|
||||
self._current_task.phase_message = "正在下载 NRO delegated 数据"
|
||||
self._current_task.phase_current = 0
|
||||
self._current_task.phase_total = total_expected
|
||||
self._current_task.phase_unit = "bytes"
|
||||
await self._db_session.commit()
|
||||
await self._publish_task_update(force=True)
|
||||
|
||||
async def on_progress(downloaded: int, total: int | None) -> None:
|
||||
if not total or total <= 0:
|
||||
return
|
||||
await self.update_phase_progress(
|
||||
current=min(downloaded, total),
|
||||
total=total,
|
||||
unit="bytes",
|
||||
message="正在下载 NRO delegated 数据",
|
||||
)
|
||||
await self.update_progress(min(downloaded, total), commit=True)
|
||||
|
||||
body_path = await self._downloader.download_file(
|
||||
|
||||
@@ -40,12 +40,23 @@ class OpenGeoFeedPrefixGeoCollector(BaseCollector):
|
||||
self._current_task.total_records = total_expected
|
||||
self._current_task.records_processed = 0
|
||||
self._current_task.progress = 0.0
|
||||
self._current_task.phase_progress = 0.0
|
||||
self._current_task.phase_message = "正在下载 OpenGeoFeed 数据"
|
||||
self._current_task.phase_current = 0
|
||||
self._current_task.phase_total = total_expected
|
||||
self._current_task.phase_unit = "bytes"
|
||||
await self._db_session.commit()
|
||||
await self._publish_task_update(force=True)
|
||||
|
||||
async def on_progress(downloaded: int, total: int | None) -> None:
|
||||
if not total or total <= 0:
|
||||
return
|
||||
await self.update_phase_progress(
|
||||
current=min(downloaded, total),
|
||||
total=total,
|
||||
unit="bytes",
|
||||
message="正在下载 OpenGeoFeed 数据",
|
||||
)
|
||||
await self.update_progress(min(downloaded, total), commit=True)
|
||||
|
||||
body_path = await self._downloader.download_file(
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
"""BarentsWatch AIS collector for vessel tracking."""
|
||||
|
||||
from datetime import UTC, datetime, timedelta
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
@@ -9,13 +8,14 @@ from sqlalchemy import delete, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.vessel import VesselPosition, VesselStatic
|
||||
from app.services.barentswatch import (
|
||||
BARENTSWATCH_LATEST_URL,
|
||||
fetch_barentswatch_access_token,
|
||||
resolve_barentswatch_config,
|
||||
)
|
||||
from app.services.collectors.base import BaseCollector
|
||||
|
||||
|
||||
BARENTSWATCH_LATEST_URL = "https://live.ais.barentswatch.no/v1/latest/combined"
|
||||
BARENTSWATCH_TOKEN_URL = "https://id.barentswatch.no/connect/token"
|
||||
|
||||
|
||||
VESSEL_TYPE_NAMES = {
|
||||
30: "Fishing",
|
||||
35: "Military",
|
||||
@@ -38,62 +38,9 @@ class VesselAISCollector(BaseCollector):
|
||||
def base_url(self) -> str:
|
||||
return self._resolved_url or BARENTSWATCH_LATEST_URL
|
||||
|
||||
async def _load_datasource_config(self) -> dict[str, Any]:
|
||||
if not self._db_session:
|
||||
return {}
|
||||
try:
|
||||
from sqlalchemy import select
|
||||
from app.models.datasource_config import DataSourceConfig
|
||||
|
||||
result = await self._db_session.execute(
|
||||
select(DataSourceConfig)
|
||||
.where(DataSourceConfig.name == self.name)
|
||||
.where(DataSourceConfig.is_active.is_(True))
|
||||
)
|
||||
datasource_config = result.scalar_one_or_none()
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
if not datasource_config:
|
||||
return {}
|
||||
return {
|
||||
"auth_config": datasource_config.auth_config or {},
|
||||
"config": datasource_config.config or {},
|
||||
}
|
||||
|
||||
async def _get_access_token(self, client: httpx.AsyncClient) -> str | None:
|
||||
datasource_config = await self._load_datasource_config()
|
||||
auth_config = datasource_config.get("auth_config") or {}
|
||||
config = datasource_config.get("config") or {}
|
||||
client_id = (
|
||||
auth_config.get("client_id")
|
||||
or config.get("client_id")
|
||||
or os.getenv("BARENTSWATCH_CLIENT_ID")
|
||||
or os.getenv("BARRENTSWATCH_CLIENT_ID")
|
||||
)
|
||||
client_secret = (
|
||||
auth_config.get("client_secret")
|
||||
or config.get("client_secret")
|
||||
or os.getenv("BARENTSWATCH_CLIENT_SECRET")
|
||||
or os.getenv("BARRENTSWATCH_CLIENT_SECRET")
|
||||
)
|
||||
if not client_id or not client_secret:
|
||||
return None
|
||||
|
||||
response = await client.post(
|
||||
BARENTSWATCH_TOKEN_URL,
|
||||
data={
|
||||
"client_id": client_id,
|
||||
"client_secret": client_secret,
|
||||
"scope": "ais",
|
||||
"grant_type": "client_credentials",
|
||||
},
|
||||
headers={"Content-Type": "application/x-www-form-urlencoded"},
|
||||
)
|
||||
response.raise_for_status()
|
||||
payload = response.json()
|
||||
token = payload.get("access_token")
|
||||
return str(token) if token else None
|
||||
config = await resolve_barentswatch_config(self._db_session)
|
||||
return await fetch_barentswatch_access_token(client, config)
|
||||
|
||||
async def fetch(self) -> list[dict[str, Any]]:
|
||||
async with httpx.AsyncClient(timeout=60.0) as client:
|
||||
|
||||
165
backend/app/services/credential_guides.py
Normal file
165
backend/app/services/credential_guides.py
Normal file
@@ -0,0 +1,165 @@
|
||||
"""Credential setup guides for collector integrations."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.models.system_setting import SystemSetting
|
||||
from app.schemas.ai import SituationalAnalysisRequest
|
||||
from app.services.ai_client import AIProviderClient
|
||||
|
||||
|
||||
CREDENTIAL_GUIDES_CATEGORY = "collector_credential_guides"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class CredentialGuideDefault:
|
||||
provider: str
|
||||
title: str
|
||||
prompt: str
|
||||
markdown: str
|
||||
|
||||
|
||||
BARENTSWATCH_DEFAULT_GUIDE = CredentialGuideDefault(
|
||||
provider="barentswatch",
|
||||
title="BarentsWatch AIS 凭证获取教程",
|
||||
prompt=(
|
||||
"请生成一份中文教程,指导开发者获取 BarentsWatch Live AIS API 的 "
|
||||
"OAuth client credentials。教程要面向已经有本地开发环境的人,包含注册/登录、"
|
||||
"创建 client、申请或确认 ais scope、复制 client id 和 client secret、"
|
||||
"在系统设置中填写并验证连接、常见失败排查。不要编造具体页面按钮文案,"
|
||||
"必须参考官方 tutorial:https://developer.barentswatch.no/docs/tutorial 。"
|
||||
"必须强调 Live AIS 要选择 AIS-client / AIS - API,而不是普通 API-client。"
|
||||
"如果步骤可能变化,要提醒以 BarentsWatch developer portal 当前页面为准。"
|
||||
),
|
||||
markdown="""## BarentsWatch AIS 凭证获取
|
||||
|
||||
官方教程:https://developer.barentswatch.no/docs/tutorial
|
||||
|
||||
1. 先打开上面的 BarentsWatch 官方 tutorial,按官方流程登录或注册开发者账号。
|
||||
2. 在 Developer access 页面选择 `AIS - API`,不要选择普通的 `BarentsWatch - API`。
|
||||
3. 在 `AIS - API` 下创建用于 Planet 的 AIS client。
|
||||
4. 创建时记下你设置的 password / client secret。
|
||||
5. 回到 My Page 复制完整 `Client ID`。它通常长得像 `your.email@example.com:client-name`。
|
||||
6. 回到 Planet 的 `设置 -> 采集器设置 -> BarentsWatch AIS`,填入 `Client ID` 和 `Client Secret`。
|
||||
7. 点击 `连接` 验证 token 和 AIS endpoint 是否可访问。
|
||||
8. 连接成功后保存凭证。
|
||||
|
||||
### 请求规则
|
||||
|
||||
- Token 地址:`https://id.barentswatch.no/connect/token`
|
||||
- 请求方式:`POST`
|
||||
- Content-Type:`application/x-www-form-urlencoded`
|
||||
- Body 必须包含:`grant_type=client_credentials`、`client_id`、`client_secret`、`scope=ais`
|
||||
- `client_id`、`client_secret`、`scope`、`grant_type` 都要放在 body,不要放在 header。
|
||||
- AIS 数据请求使用 header:`Authorization: Bearer <access_token>`
|
||||
|
||||
### 常见排查
|
||||
|
||||
- `未找到凭证`:确认 `Client ID` 和 `Client Secret` 已填写,或已经写入 `~/.zshrc`。
|
||||
- `HTTP 401/403`:通常是选成了普通 `BarentsWatch - API` client、client secret 错误,或 token 请求没有使用 `scope=ais`。
|
||||
- `network` 错误:检查本机是否能访问 `id.barentswatch.no` 和 `live.ais.barentswatch.no`。
|
||||
- Endpoint 建议保持默认:`https://live.ais.barentswatch.no/v1/latest/combined`。
|
||||
""",
|
||||
)
|
||||
|
||||
|
||||
DEFAULT_CREDENTIAL_GUIDES = {
|
||||
BARENTSWATCH_DEFAULT_GUIDE.provider: BARENTSWATCH_DEFAULT_GUIDE,
|
||||
}
|
||||
|
||||
|
||||
async def _get_guide_store(db) -> tuple[SystemSetting | None, dict[str, Any]]:
|
||||
result = await db.execute(
|
||||
select(SystemSetting).where(SystemSetting.category == CREDENTIAL_GUIDES_CATEGORY)
|
||||
)
|
||||
record = result.scalar_one_or_none()
|
||||
payload = dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
|
||||
return record, payload
|
||||
|
||||
|
||||
async def get_credential_guide(db, provider: str) -> dict[str, Any]:
|
||||
default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
|
||||
if default is None:
|
||||
raise ValueError(f"Unsupported credential guide provider: {provider}")
|
||||
|
||||
_record, store = await _get_guide_store(db)
|
||||
custom = store.get(provider) if isinstance(store.get(provider), dict) else None
|
||||
return {
|
||||
"provider": provider,
|
||||
"title": custom.get("title") if custom else default.title,
|
||||
"markdown": custom.get("markdown") if custom else default.markdown,
|
||||
"prompt": default.prompt,
|
||||
"source": "ai" if custom else "default",
|
||||
}
|
||||
|
||||
|
||||
async def save_credential_guide(db, provider: str, title: str, markdown: str) -> dict[str, Any]:
|
||||
default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
|
||||
if default is None:
|
||||
raise ValueError(f"Unsupported credential guide provider: {provider}")
|
||||
|
||||
record, store = await _get_guide_store(db)
|
||||
store[provider] = {
|
||||
"title": title or default.title,
|
||||
"markdown": markdown,
|
||||
}
|
||||
if record is None:
|
||||
db.add(SystemSetting(category=CREDENTIAL_GUIDES_CATEGORY, payload=store))
|
||||
else:
|
||||
record.payload = store
|
||||
await db.commit()
|
||||
return await get_credential_guide(db, provider)
|
||||
|
||||
|
||||
async def reset_credential_guide(db, provider: str) -> dict[str, Any]:
|
||||
default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
|
||||
if default is None:
|
||||
raise ValueError(f"Unsupported credential guide provider: {provider}")
|
||||
|
||||
record, store = await _get_guide_store(db)
|
||||
if provider in store:
|
||||
store.pop(provider, None)
|
||||
if record is not None:
|
||||
record.payload = store
|
||||
await db.commit()
|
||||
return await get_credential_guide(db, provider)
|
||||
|
||||
|
||||
async def generate_credential_guide(
|
||||
db,
|
||||
provider: str,
|
||||
ai_client: AIProviderClient,
|
||||
) -> dict[str, Any]:
|
||||
default = DEFAULT_CREDENTIAL_GUIDES.get(provider)
|
||||
if default is None:
|
||||
raise ValueError(f"Unsupported credential guide provider: {provider}")
|
||||
|
||||
response = await ai_client.analyze(
|
||||
SituationalAnalysisRequest(
|
||||
title=f"Generate credential guide for {provider}",
|
||||
objective=default.prompt,
|
||||
context={
|
||||
"provider": provider,
|
||||
"current_default_guide": default.markdown,
|
||||
"product_context": "Planet collector credential settings",
|
||||
},
|
||||
observations=[
|
||||
"Use concise Chinese markdown.",
|
||||
"Prefer stable concepts over brittle UI labels.",
|
||||
"Include verification and troubleshooting steps.",
|
||||
],
|
||||
constraints=[
|
||||
"Do not ask the user for secrets.",
|
||||
"Do not include fabricated screenshots.",
|
||||
"Return markdown only.",
|
||||
],
|
||||
)
|
||||
)
|
||||
markdown = response.content.strip()
|
||||
if not markdown:
|
||||
markdown = default.markdown
|
||||
return await save_credential_guide(db, provider, default.title, markdown)
|
||||
384
backend/app/services/datasource_connectivity.py
Normal file
384
backend/app/services/datasource_connectivity.py
Normal file
@@ -0,0 +1,384 @@
|
||||
"""Connectivity validation helpers for built-in datasource overrides."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
from typing import Any
|
||||
|
||||
import httpx
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from app.core.data_sources import get_data_sources_config
|
||||
from app.core.datasource_defaults import DEFAULT_DATASOURCES
|
||||
from app.models.collected_data import CollectedData
|
||||
from app.models.datasource import DataSource
|
||||
from app.models.datasource_config import DataSourceConfig
|
||||
from app.models.system_setting import SystemSetting
|
||||
from app.services.barentswatch import (
|
||||
_read_zshrc_env,
|
||||
fetch_barentswatch_access_token,
|
||||
resolve_barentswatch_config,
|
||||
)
|
||||
|
||||
|
||||
CONNECTIVITY_VALIDATION_KEY = "connectivity_validation"
|
||||
CONNECTIVITY_STORE_CATEGORY = "datasource_connectivity_validations"
|
||||
|
||||
|
||||
def _sha256_json(payload: Any) -> str:
|
||||
encoded = json.dumps(payload, ensure_ascii=False, sort_keys=True, default=str).encode("utf-8")
|
||||
return hashlib.sha256(encoded).hexdigest()
|
||||
|
||||
|
||||
def _resolve_spacetrack_credentials() -> tuple[str, str, str]:
|
||||
zshrc_env = _read_zshrc_env()
|
||||
username = os.getenv("SPACETRACK_USERNAME") or zshrc_env.get("SPACETRACK_USERNAME") or ""
|
||||
password = os.getenv("SPACETRACK_PASSWORD") or zshrc_env.get("SPACETRACK_PASSWORD") or ""
|
||||
source = "environment" if os.getenv("SPACETRACK_USERNAME") or os.getenv("SPACETRACK_PASSWORD") else ""
|
||||
if not source and (username or password):
|
||||
source = "~/.zshrc"
|
||||
return username, password, source or "missing"
|
||||
|
||||
|
||||
def strip_connectivity_validation(config: dict | None) -> dict:
|
||||
cleaned = dict(config or {})
|
||||
cleaned.pop(CONNECTIVITY_VALIDATION_KEY, None)
|
||||
return cleaned
|
||||
|
||||
|
||||
def merge_connectivity_validation(existing_config: dict | None, next_config: dict | None) -> dict:
|
||||
merged = strip_connectivity_validation(next_config)
|
||||
validation = (existing_config or {}).get(CONNECTIVITY_VALIDATION_KEY)
|
||||
if validation:
|
||||
merged[CONNECTIVITY_VALIDATION_KEY] = validation
|
||||
return merged
|
||||
|
||||
|
||||
def get_connectivity_validation(config: DataSourceConfig | None) -> dict | None:
|
||||
validation = (config.config or {}).get(CONNECTIVITY_VALIDATION_KEY) if config else None
|
||||
return validation if isinstance(validation, dict) else None
|
||||
|
||||
|
||||
async def build_builtin_connectivity_checksum(
|
||||
source: str,
|
||||
endpoint: str,
|
||||
auth_type: str,
|
||||
headers: dict | None,
|
||||
config: dict | None,
|
||||
db=None,
|
||||
credential_override: dict[str, str] | None = None,
|
||||
) -> tuple[str, dict[str, Any]]:
|
||||
defaults = DEFAULT_DATASOURCES.get(source, {})
|
||||
credential_provider = defaults.get("credential_provider")
|
||||
credential_fingerprint = ""
|
||||
credential_source = "none"
|
||||
has_credentials = not defaults.get("requires_credentials", False)
|
||||
|
||||
if credential_provider == "barentswatch":
|
||||
if credential_override:
|
||||
client_id = credential_override.get("client_id", "")
|
||||
client_secret = credential_override.get("client_secret", "")
|
||||
credential_source = "draft"
|
||||
else:
|
||||
barentswatch_config = await resolve_barentswatch_config(db)
|
||||
client_id = barentswatch_config.client_id
|
||||
client_secret = barentswatch_config.client_secret
|
||||
credential_source = barentswatch_config.credential_source
|
||||
has_credentials = bool(client_id and client_secret)
|
||||
credential_fingerprint = _sha256_json(
|
||||
{
|
||||
"client_id": client_id,
|
||||
"client_secret": client_secret,
|
||||
}
|
||||
)
|
||||
elif credential_provider == "spacetrack":
|
||||
username, password, credential_source = _resolve_spacetrack_credentials()
|
||||
has_credentials = bool(username and password)
|
||||
credential_fingerprint = _sha256_json(
|
||||
{
|
||||
"username": username,
|
||||
"password": password,
|
||||
}
|
||||
)
|
||||
elif defaults.get("requires_credentials"):
|
||||
credential_source = str(credential_provider or "unsupported")
|
||||
|
||||
checksum_payload = {
|
||||
"source": source,
|
||||
"endpoint": endpoint,
|
||||
"auth_type": "none",
|
||||
"headers": headers or {},
|
||||
"config": strip_connectivity_validation(config),
|
||||
"credential_provider": credential_provider or "none",
|
||||
"credential_fingerprint": credential_fingerprint,
|
||||
}
|
||||
return _sha256_json(checksum_payload), {
|
||||
"requires_credentials": bool(defaults.get("requires_credentials", False)),
|
||||
"credential_provider": credential_provider,
|
||||
"credential_source": credential_source,
|
||||
"has_credentials": has_credentials,
|
||||
}
|
||||
|
||||
|
||||
async def test_builtin_connectivity(
|
||||
source: str,
|
||||
endpoint: str,
|
||||
auth_type: str,
|
||||
headers: dict | None,
|
||||
config: dict | None,
|
||||
db=None,
|
||||
) -> dict[str, Any]:
|
||||
defaults = DEFAULT_DATASOURCES.get(source)
|
||||
if not defaults:
|
||||
return {
|
||||
"success": False,
|
||||
"message": "未知内置采集器,无法执行连接校验。",
|
||||
}
|
||||
|
||||
checksum, credential_context = await build_builtin_connectivity_checksum(
|
||||
source,
|
||||
endpoint,
|
||||
auth_type,
|
||||
headers,
|
||||
config,
|
||||
db,
|
||||
)
|
||||
if credential_context["requires_credentials"] and not credential_context["has_credentials"]:
|
||||
return {
|
||||
"success": False,
|
||||
"checksum": checksum,
|
||||
"stage": "credentials",
|
||||
"message": "该采集器需要凭证,请先到采集器凭证设置中配置。",
|
||||
"settings_tab": "collector_credentials",
|
||||
**credential_context,
|
||||
}
|
||||
supported_credential_providers = {"barentswatch", "spacetrack"}
|
||||
if (
|
||||
credential_context["requires_credentials"]
|
||||
and credential_context["credential_provider"] not in supported_credential_providers
|
||||
):
|
||||
return {
|
||||
"success": False,
|
||||
"checksum": checksum,
|
||||
"stage": "credentials",
|
||||
"message": "该采集器的凭证链路尚未接入,暂时无法完成连接校验。",
|
||||
"settings_tab": "collector_credentials",
|
||||
**credential_context,
|
||||
}
|
||||
|
||||
request_headers = {str(key): str(value) for key, value in (headers or {}).items()}
|
||||
request_config = strip_connectivity_validation(config)
|
||||
timeout = float(request_config.get("timeout") or 30)
|
||||
request_endpoint = endpoint
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
|
||||
if credential_context["credential_provider"] == "barentswatch":
|
||||
barentswatch_config = await resolve_barentswatch_config(db)
|
||||
token = await fetch_barentswatch_access_token(client, barentswatch_config)
|
||||
if not token:
|
||||
return {
|
||||
"success": False,
|
||||
"checksum": checksum,
|
||||
"stage": "token",
|
||||
"message": "凭证可读取,但 token 响应中没有 access_token。",
|
||||
"settings_tab": "collector_credentials",
|
||||
**credential_context,
|
||||
}
|
||||
request_headers["Authorization"] = f"Bearer {token}"
|
||||
elif credential_context["credential_provider"] == "spacetrack":
|
||||
username, password, _source = _resolve_spacetrack_credentials()
|
||||
login_url = "https://www.space-track.org/ajaxauth/login"
|
||||
login_response = await client.post(
|
||||
login_url,
|
||||
data={
|
||||
"identity": username,
|
||||
"password": password,
|
||||
},
|
||||
)
|
||||
login_response.raise_for_status()
|
||||
|
||||
started = datetime.now(UTC)
|
||||
async with client.stream("GET", request_endpoint, headers=request_headers) as response:
|
||||
response.raise_for_status()
|
||||
status_code = response.status_code
|
||||
elapsed_ms = (datetime.now(UTC) - started).total_seconds() * 1000
|
||||
|
||||
return {
|
||||
"success": True,
|
||||
"checksum": checksum,
|
||||
"stage": "endpoint",
|
||||
"message": "连接验证成功。",
|
||||
"status_code": status_code,
|
||||
"response_time_ms": elapsed_ms,
|
||||
**credential_context,
|
||||
}
|
||||
except httpx.HTTPStatusError as exc:
|
||||
return {
|
||||
"success": False,
|
||||
"checksum": checksum,
|
||||
"stage": "endpoint",
|
||||
"message": f"连接验证失败:HTTP {exc.response.status_code}",
|
||||
"error": f"HTTP Error: {exc.response.status_code}",
|
||||
**credential_context,
|
||||
}
|
||||
except httpx.HTTPError as exc:
|
||||
return {
|
||||
"success": False,
|
||||
"checksum": checksum,
|
||||
"stage": "network",
|
||||
"message": f"连接验证失败:{exc.__class__.__name__}",
|
||||
"error": str(exc),
|
||||
**credential_context,
|
||||
}
|
||||
|
||||
|
||||
def make_success_validation(checksum: str, result: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"checksum": checksum,
|
||||
"status": "success",
|
||||
"validated_at": datetime.now(UTC).isoformat(),
|
||||
"status_code": result.get("status_code"),
|
||||
"credential_source": result.get("credential_source"),
|
||||
}
|
||||
|
||||
|
||||
def is_builtin_validation_current(config: DataSourceConfig | None, checksum: str) -> bool:
|
||||
validation = get_connectivity_validation(config)
|
||||
return bool(
|
||||
validation
|
||||
and validation.get("status") == "success"
|
||||
and validation.get("checksum") == checksum
|
||||
)
|
||||
|
||||
|
||||
async def get_connectivity_store(db) -> dict[str, Any]:
|
||||
result = await db.execute(
|
||||
select(SystemSetting).where(SystemSetting.category == CONNECTIVITY_STORE_CATEGORY)
|
||||
)
|
||||
record = result.scalar_one_or_none()
|
||||
return dict(record.payload or {}) if record and isinstance(record.payload, dict) else {}
|
||||
|
||||
|
||||
async def save_connectivity_success(
|
||||
db,
|
||||
source: str,
|
||||
checksum: str,
|
||||
result: dict[str, Any],
|
||||
*,
|
||||
connected_by: str,
|
||||
) -> dict[str, Any]:
|
||||
store = await get_connectivity_store(db)
|
||||
validation = {
|
||||
**make_success_validation(checksum, result),
|
||||
"connected_by": connected_by,
|
||||
}
|
||||
store[source] = validation
|
||||
|
||||
existing = await db.execute(
|
||||
select(SystemSetting).where(SystemSetting.category == CONNECTIVITY_STORE_CATEGORY)
|
||||
)
|
||||
record = existing.scalar_one_or_none()
|
||||
if record is None:
|
||||
db.add(SystemSetting(category=CONNECTIVITY_STORE_CATEGORY, payload=store))
|
||||
else:
|
||||
record.payload = store
|
||||
return validation
|
||||
|
||||
|
||||
async def load_builtin_override_config(db, source: str) -> DataSourceConfig | None:
|
||||
result = await db.execute(
|
||||
select(DataSourceConfig)
|
||||
.where(DataSourceConfig.name == source)
|
||||
.where(DataSourceConfig.is_active.is_(True))
|
||||
)
|
||||
return result.scalar_one_or_none()
|
||||
|
||||
|
||||
async def get_builtin_effective_candidate(db, source: str) -> dict[str, Any]:
|
||||
override = await load_builtin_override_config(db, source)
|
||||
default_endpoint = get_data_sources_config().get_yaml_url(source)
|
||||
return {
|
||||
"name": source,
|
||||
"endpoint": (override.endpoint if override and override.endpoint else default_endpoint) or "",
|
||||
"auth_type": override.auth_type if override else "none",
|
||||
"headers": override.headers if override else {},
|
||||
"config": strip_connectivity_validation(override.config if override else {}),
|
||||
}
|
||||
|
||||
|
||||
async def has_collected_data(db, source: str) -> bool:
|
||||
result = await db.execute(select(func.count(CollectedData.id)).where(CollectedData.source == source))
|
||||
if (result.scalar() or 0) > 0:
|
||||
return True
|
||||
|
||||
datasource_result = await db.execute(select(DataSource).where(DataSource.source == source))
|
||||
datasource = datasource_result.scalar_one_or_none()
|
||||
return bool(datasource and datasource.last_status == "success")
|
||||
|
||||
|
||||
async def get_builtin_connection_status(
|
||||
db,
|
||||
source: str,
|
||||
endpoint: str,
|
||||
auth_type: str,
|
||||
headers: dict | None,
|
||||
config: dict | None,
|
||||
) -> dict[str, Any]:
|
||||
checksum, credential_context = await build_builtin_connectivity_checksum(
|
||||
source,
|
||||
endpoint,
|
||||
auth_type,
|
||||
headers,
|
||||
config,
|
||||
db,
|
||||
)
|
||||
store = await get_connectivity_store(db)
|
||||
validation = store.get(source)
|
||||
if isinstance(validation, dict) and validation.get("status") == "success":
|
||||
if validation.get("checksum") == checksum:
|
||||
return {
|
||||
"connected": True,
|
||||
"checksum": checksum,
|
||||
"connected_by": validation.get("connected_by") or "connection_button",
|
||||
"message": "当前配置已完成连接验证。",
|
||||
**credential_context,
|
||||
}
|
||||
|
||||
effective = await get_builtin_effective_candidate(db, source)
|
||||
effective_checksum, _ = await build_builtin_connectivity_checksum(
|
||||
source,
|
||||
effective["endpoint"],
|
||||
effective["auth_type"],
|
||||
effective["headers"],
|
||||
effective["config"],
|
||||
db,
|
||||
)
|
||||
if checksum == effective_checksum and await has_collected_data(db, source):
|
||||
return {
|
||||
"connected": True,
|
||||
"checksum": checksum,
|
||||
"connected_by": "collection",
|
||||
"message": "当前配置已有成功采集数据,视为已连接。",
|
||||
**credential_context,
|
||||
}
|
||||
|
||||
if isinstance(validation, dict) and validation.get("status") == "success":
|
||||
return {
|
||||
"connected": False,
|
||||
"checksum": checksum,
|
||||
"connected_by": None,
|
||||
"message": "接口地址或凭证指纹已变化,请重新点击连接验证。",
|
||||
**credential_context,
|
||||
}
|
||||
|
||||
return {
|
||||
"connected": False,
|
||||
"checksum": checksum,
|
||||
"connected_by": None,
|
||||
"message": "当前配置尚未连接,请点击连接验证。",
|
||||
**credential_context,
|
||||
}
|
||||
@@ -14,6 +14,11 @@ from app.core.time import to_iso8601_utc
|
||||
from app.models.datasource import DataSource
|
||||
from app.models.task import CollectionTask
|
||||
from app.services.collectors.registry import collector_registry
|
||||
from app.services.datasource_connectivity import (
|
||||
build_builtin_connectivity_checksum,
|
||||
get_builtin_effective_candidate,
|
||||
save_connectivity_success,
|
||||
)
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
@@ -179,6 +184,23 @@ async def run_collector_task(collector_name: str):
|
||||
task_result = await collector.run(db)
|
||||
datasource.last_run_at = datetime.now(UTC)
|
||||
datasource.last_status = task_result.get("status")
|
||||
if datasource.last_status == "success":
|
||||
effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
|
||||
checksum, _credential_context = await build_builtin_connectivity_checksum(
|
||||
datasource.source,
|
||||
effective_candidate["endpoint"],
|
||||
effective_candidate["auth_type"],
|
||||
effective_candidate["headers"],
|
||||
effective_candidate["config"],
|
||||
db,
|
||||
)
|
||||
await save_connectivity_success(
|
||||
db,
|
||||
datasource.source,
|
||||
checksum,
|
||||
{"status_code": None},
|
||||
connected_by="collection",
|
||||
)
|
||||
await _update_next_run_at(datasource, db)
|
||||
logger.info_event(
|
||||
"Collector completed",
|
||||
|
||||
@@ -19,6 +19,10 @@ ALLOWED_ACTIONS: dict[str, dict[str, Any]] = {
|
||||
"command": ["./planet.sh", "restart", "-b"],
|
||||
"recovery_mode": "backend",
|
||||
},
|
||||
"restart-frontend": {
|
||||
"command": ["./planet.sh", "restart", "-f"],
|
||||
"recovery_mode": "frontend",
|
||||
},
|
||||
"restart-ai-provider": {
|
||||
"command": ["./planet.sh", "restart", "-a"],
|
||||
"recovery_mode": "ai-provider",
|
||||
|
||||
@@ -2,6 +2,7 @@ from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import os
|
||||
import shlex
|
||||
import subprocess
|
||||
import sys
|
||||
import time
|
||||
@@ -59,6 +60,8 @@ def wait_for_recovery(action: str) -> tuple[bool, str]:
|
||||
recovery_mode = get_action_recovery_mode(action)
|
||||
if recovery_mode == "backend":
|
||||
return wait_for_http("http://localhost:8000/health"), "backend health recovery"
|
||||
if recovery_mode == "frontend":
|
||||
return wait_for_http("http://localhost:3000"), "frontend entrypoint recovery"
|
||||
if recovery_mode == "ai-provider":
|
||||
return wait_for_http("http://localhost:8010/health"), "ai provider health recovery"
|
||||
if recovery_mode == "database":
|
||||
@@ -108,8 +111,9 @@ def main() -> int:
|
||||
)
|
||||
append_task_log(args.task_id, "restart command started")
|
||||
|
||||
shell_command = shlex.join(command)
|
||||
completed = subprocess.run(
|
||||
command,
|
||||
["zsh", "-ic", shell_command],
|
||||
cwd=str(ROOT_DIR),
|
||||
env=env,
|
||||
capture_output=True,
|
||||
|
||||
@@ -6,6 +6,7 @@ from unittest.mock import AsyncMock, MagicMock, patch
|
||||
|
||||
from app.services.collectors.top500 import TOP500Collector
|
||||
from app.services.collectors.base import BaseCollector, HTTPCollector
|
||||
from app.models.task import CollectionTask
|
||||
|
||||
|
||||
class TestBaseCollector:
|
||||
@@ -19,6 +20,31 @@ class TestBaseCollector:
|
||||
assert collector.module == "L1"
|
||||
assert collector.frequency_hours == 4
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_update_phase_progress_tracks_phase_fields(self, mock_db_session):
|
||||
"""Test phase-level progress updates independently from record totals"""
|
||||
collector = TOP500Collector()
|
||||
task = CollectionTask(datasource_id=1, status="running", phase="fetching")
|
||||
collector._current_task = task
|
||||
collector._db_session = mock_db_session
|
||||
|
||||
with patch.object(collector, "_publish_task_update", new=AsyncMock()) as publish:
|
||||
await collector.update_phase_progress(
|
||||
current=512,
|
||||
total=1024,
|
||||
unit="bytes",
|
||||
message="Downloading dataset",
|
||||
commit=True,
|
||||
)
|
||||
|
||||
assert task.phase_progress == 50.0
|
||||
assert task.phase_current == 512
|
||||
assert task.phase_total == 1024
|
||||
assert task.phase_unit == "bytes"
|
||||
assert task.phase_message == "Downloading dataset"
|
||||
mock_db_session.commit.assert_awaited_once()
|
||||
publish.assert_awaited_once()
|
||||
|
||||
|
||||
class TestTOP500Collector:
|
||||
"""Tests for TOP500Collector"""
|
||||
|
||||
@@ -121,6 +121,25 @@ class TestCollectionTaskModel:
|
||||
)
|
||||
assert task.records_processed == 100
|
||||
|
||||
def test_task_with_phase_progress(self):
|
||||
"""Test collection task phase-level progress fields"""
|
||||
task = CollectionTask(
|
||||
datasource_id=1,
|
||||
status="running",
|
||||
phase="fetching",
|
||||
phase_progress=42.5,
|
||||
phase_message="Downloading dataset",
|
||||
phase_current=1024,
|
||||
phase_total=4096,
|
||||
phase_unit="bytes",
|
||||
)
|
||||
assert task.phase == "fetching"
|
||||
assert task.phase_progress == 42.5
|
||||
assert task.phase_message == "Downloading dataset"
|
||||
assert task.phase_current == 1024
|
||||
assert task.phase_total == 4096
|
||||
assert task.phase_unit == "bytes"
|
||||
|
||||
def test_task_error_message(self):
|
||||
"""Test collection task with error message"""
|
||||
task = CollectionTask(
|
||||
|
||||
@@ -7,6 +7,7 @@ from app.api.v1.visualization import convert_vessels_to_geojson
|
||||
from app.db.session import get_db
|
||||
from app.main import app
|
||||
from app.models.vessel import VesselPosition, VesselStatic
|
||||
from app.services import barentswatch
|
||||
from app.services.collectors.vessel_ais import VesselAISCollector
|
||||
|
||||
|
||||
@@ -34,6 +35,49 @@ def test_vessel_collector_transforms_barentswatch_like_records():
|
||||
assert records[0]["lat"] == pytest.approx(59.91)
|
||||
|
||||
|
||||
def test_barentswatch_reads_credentials_from_zshrc(tmp_path):
|
||||
zshrc = tmp_path / ".zshrc"
|
||||
zshrc.write_text(
|
||||
"\n".join(
|
||||
[
|
||||
"export BARENTSWATCH_CLIENT_ID='client-from-zshrc'",
|
||||
'export BARENTSWATCH_CLIENT_SECRET="secret-from-zshrc" # local dev credential',
|
||||
]
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
values = barentswatch._read_zshrc_env(zshrc)
|
||||
|
||||
assert values["BARENTSWATCH_CLIENT_ID"] == "client-from-zshrc"
|
||||
assert values["BARENTSWATCH_CLIENT_SECRET"] == "secret-from-zshrc"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_barentswatch_resolves_config_from_zshrc(tmp_path, monkeypatch):
|
||||
zshrc = tmp_path / ".zshrc"
|
||||
zshrc.write_text(
|
||||
"\n".join(
|
||||
[
|
||||
"export BARENTSWATCH_CLIENT_ID=client-from-zshrc",
|
||||
"export BARENTSWATCH_CLIENT_SECRET=secret-from-zshrc",
|
||||
]
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
monkeypatch.delenv("BARENTSWATCH_CLIENT_ID", raising=False)
|
||||
monkeypatch.delenv("BARENTSWATCH_CLIENT_SECRET", raising=False)
|
||||
monkeypatch.delenv("BARRENTSWATCH_CLIENT_ID", raising=False)
|
||||
monkeypatch.delenv("BARRENTSWATCH_CLIENT_SECRET", raising=False)
|
||||
monkeypatch.setattr(barentswatch.Path, "home", lambda: tmp_path)
|
||||
|
||||
config = await barentswatch.resolve_barentswatch_config(None)
|
||||
|
||||
assert config.client_id == "client-from-zshrc"
|
||||
assert config.client_secret == "secret-from-zshrc"
|
||||
assert config.credential_source == "~/.zshrc"
|
||||
|
||||
|
||||
def test_convert_vessels_to_geojson():
|
||||
position = VesselPosition(
|
||||
mmsi=257123000,
|
||||
|
||||
@@ -86,6 +86,12 @@ CREATE TABLE collection_tasks (
|
||||
id BIGSERIAL PRIMARY KEY,
|
||||
datasource_id INTEGER NOT NULL REFERENCES data_sources(id) ON DELETE CASCADE,
|
||||
status task_status NOT NULL DEFAULT 'pending',
|
||||
phase VARCHAR(30) DEFAULT 'queued',
|
||||
phase_progress FLOAT,
|
||||
phase_message VARCHAR(255),
|
||||
phase_current BIGINT,
|
||||
phase_total BIGINT,
|
||||
phase_unit VARCHAR(30),
|
||||
started_at TIMESTAMP WITH TIME ZONE,
|
||||
completed_at TIMESTAMP WITH TIME ZONE,
|
||||
records_processed INTEGER DEFAULT 0,
|
||||
|
||||
@@ -8,6 +8,9 @@ services:
|
||||
args:
|
||||
PYTHON_IMAGE: ${PYTHON_IMAGE:-python:3.14-slim}
|
||||
UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
|
||||
env_file:
|
||||
- ./aiprovider/.env
|
||||
- ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
|
||||
container_name: planet_aiprovider
|
||||
ports:
|
||||
- "8010:8010"
|
||||
|
||||
@@ -10,6 +10,7 @@ services:
|
||||
UV_IMAGE: ${UV_IMAGE:-ghcr.io/astral-sh/uv:latest}
|
||||
env_file:
|
||||
- ./aiprovider/.env
|
||||
- ${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-./aiprovider/.env}
|
||||
container_name: planet_aiprovider
|
||||
ports:
|
||||
- "8010:8010"
|
||||
|
||||
@@ -8,6 +8,62 @@ This project follows the repository versioning rule:
|
||||
- `improvement` -> `+0.0.1`(bugfix + 小功能混合)
|
||||
- `bugfix` -> `+0.0.1`
|
||||
|
||||
## [0.45.0] — 2026-04-29
|
||||
|
||||
### ✨ Highlights
|
||||
- 采集任务新增阶段级量化进度,`fetching` 可展示百分比、阶段说明和字节下载量。
|
||||
- AI Provider 启动链路支持从 `aiprovider/.env` 与 `~/.zshrc` 注入运行期配置,并避免密钥/模型变化触发镜像重建。
|
||||
- AI Provider Docker build context 收敛到服务必需文件,`uv sync` 接入 BuildKit 缓存以减少重复下载。
|
||||
|
||||
### 🔧 Improvements
|
||||
- IPtoASN、OpenGeoFeed、NRO delegated 下载型采集器接入真实字节进度上报。
|
||||
- 数据源页、采集中任务弹窗和任务历史页展示阶段摘要,并在 tooltip 中保留完整进度细节。
|
||||
- 调整 Earth 船只默认高度偏移,进一步贴近地表展示。
|
||||
|
||||
---
|
||||
|
||||
## [0.44.2] — 2026-04-29
|
||||
|
||||
### 📝 Documentation
|
||||
- 补充 Earth 船只图层技术文档,记录分桶 `THREE.Points` 批量渲染、同尺寸交互 overlay 和屏幕空间 picking 的设计约束。
|
||||
- 同步 Earth 渲染图层顺序和样式参考,明确 AIS 船只 renderOrder、depthTest、图标尺寸、航向分桶与 hover 命中半径。
|
||||
- 更新船只渲染性能计划状态,标注 `0.44.1` 已落地的实现与后续全球 AIS / LOD 演进方向。
|
||||
|
||||
---
|
||||
|
||||
## [0.44.1] — 2026-04-29
|
||||
|
||||
### 🐛 Fixes
|
||||
- 修复 Earth 船只图层拖动不跟手的问题,将普通船只从独立 Sprite 切换为按航向分桶的批量 Points 渲染。
|
||||
- 修正船只 hover/click 拾取错位,改为屏幕空间命中检测并在拖拽/惯性期间跳过 hover 拾取。
|
||||
- 统一 AIS 船只普通态与交互态方向,并让 hover/locked glow 与普通图标保持同尺寸覆盖。
|
||||
- 恢复船只深度测试并收敛默认图标尺寸,避免北部岛屿/冰面附近出现明显压盖陆地的视觉问题。
|
||||
|
||||
---
|
||||
|
||||
## [0.44.0] — 2026-04-29
|
||||
|
||||
### ✨ Highlights
|
||||
- 重构数据源与采集器设置边界:数据源页回归目录和采集触发,采集器 endpoint、请求头、凭证与连接验证统一进入设置页
|
||||
- BarentsWatch AIS 完整接入凭证解析、连接检查、默认教程、AI 生成教程和船只采集/可视化链路
|
||||
- Earth 新增船只图例、缩放反馈胶囊、缩放感知拖拽灵敏度,并将船只渲染性能优化方案沉淀到 plans
|
||||
- 仪表盘重启服务新增前端重启 action,并让 runner 通过 `~/.zshrc` 继承本地环境变量
|
||||
|
||||
### 🔧 Improvements
|
||||
- 采集器连接状态改为基于成功采集或手动连接校验 checksum 判断,避免只依赖前端样式状态
|
||||
- 数据源页新增采集中任务标签和任务进度弹窗,内置与自定义数据源统一展示
|
||||
- Earth 国界线进一步贴近地表,并补充船只图层渲染顺序、样式和用户手册说明
|
||||
- docs skill 与 Claude/Codex 文档流程补齐技术文档和 plans 的职责边界
|
||||
|
||||
---
|
||||
|
||||
## [0.43.1] — 2026-04-28
|
||||
|
||||
### 🐛 Fixes
|
||||
- 修正 `planet.sh` 在全量 restart 后启动 AI Provider 时的提示语义,避免把预期内未就绪描述成异常不健康
|
||||
|
||||
---
|
||||
|
||||
## [0.43.0] — 2026-04-28
|
||||
|
||||
### ✨ Highlights
|
||||
|
||||
@@ -24,6 +24,8 @@
|
||||
- [earth-webgl-instancing-satellites-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
|
||||
- [earth-real-terrain-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
|
||||
- [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
|
||||
- [earth-news-cruise-summary-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
|
||||
- [earth-vessel-rendering-performance-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-vessel-rendering-performance-plan.md)
|
||||
- [frontend-public-docs-site-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-public-docs-site-plan.md)
|
||||
- [frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
|
||||
- [ue5-mvp-fused-plan.md](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)
|
||||
|
||||
@@ -235,7 +235,9 @@ Settings 中保留统一外部集成配置:
|
||||
|
||||
DataSources 不直接管理全局 secret,只引用 provider profile。
|
||||
|
||||
### Phase 2 — DataSources 自定义源向导
|
||||
### Phase 2 — Settings 自定义源向导
|
||||
|
||||
自定义数据源配置入口应放在 `/settings` 的“采集器设置”或后续专门的自定义采集器设置区。`/datasources` 保持数据源目录和采集触发职责,不再承载编辑入口。
|
||||
|
||||
自定义数据源配置改成向导或右侧 drawer:
|
||||
|
||||
|
||||
366
docs/plans/earth-news-cruise-summary-plan.md
Normal file
366
docs/plans/earth-news-cruise-summary-plan.md
Normal file
@@ -0,0 +1,366 @@
|
||||
# Earth 新闻巡航摘要增强计划
|
||||
|
||||
## 背景
|
||||
|
||||
`Earth` 的新闻巡航模式目前直接消费 `/api/v1/news/earth-feed` 返回的 `items[].summary`。这个字段主要来自 RSS/Atom 的 `description`、`summary` 或 `content`,再经过 HTML 清理与长度截断。
|
||||
|
||||
这个实现足够轻量,但在巡航展示里有三个问题:
|
||||
|
||||
- 不是所有新闻源都会提供摘要,部分源只返回标题和链接。
|
||||
- 聚合源的摘要质量不稳定,可能只是重复标题、来源署名或片段文本。
|
||||
- 巡航模式需要更稳定的“态势说明”,否则新闻卡片的 `SUMMARY` 区域会显得空或信息密度不足。
|
||||
|
||||
目标不是把所有新闻都交给大模型,而是建立一个分层摘要管线:能用新闻源自带内容时零成本处理,需要增强时优先用本地模型,云端 LLM 只作为可控兜底。
|
||||
|
||||
## 当前相关实现
|
||||
|
||||
| 文件 | 作用 |
|
||||
| --- | --- |
|
||||
| `backend/app/services/earth_news.py` | 拉取 RSS/Atom 新闻源、解析标题/摘要、按区域聚合并返回 Earth 新闻 payload |
|
||||
| `backend/app/api/v1/news.py` | 暴露 `/api/v1/news/earth-feed` |
|
||||
| `frontend/public/earth/js/news.js` | 拉取新闻 payload 并渲染媒体面板新闻列表 |
|
||||
| `frontend/public/earth/js/news-cruise-adapter.js` | 将新闻条目映射为巡航事件,并把 `summary` 传给信息卡 |
|
||||
| `frontend/public/earth/js/info-card.js` | 展示新闻巡航卡片中的 `SUMMARY` |
|
||||
|
||||
当前摘要生成逻辑集中在 `earth_news.py`:
|
||||
|
||||
```python
|
||||
summary = _extract_item_text(node, "description", "content")
|
||||
clean_summary = _truncate(_strip_html(summary), 180)
|
||||
```
|
||||
|
||||
这意味着后端还没有区分“摘要来自哪里”“质量是否足够”“是否需要异步增强”。
|
||||
|
||||
## 总体方案
|
||||
|
||||
采用四级摘要来源:
|
||||
|
||||
| 优先级 | 来源 | 成本 | 适用情况 | 风险 |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| 1 | 新闻源自带 summary / description | 低 | RSS/Atom 已提供可读摘要 | 字段可能为空或重复标题 |
|
||||
| 2 | 本地规则提取 | 低 | 有正文片段但没有可靠摘要 | 只能抽取,不能真正概括 |
|
||||
| 3 | 本地 Gemma/Ollama 摘要 | 中 | 巡航会展示且前两级质量不足 | 本地模型质量与机器性能相关 |
|
||||
| 4 | 云端 LLM 兜底 | 高 | 用户手动增强、重点新闻、失败补偿 | 成本与网络依赖 |
|
||||
|
||||
推荐默认策略:
|
||||
|
||||
```text
|
||||
provider summary -> extractive summary -> cached local model summary -> async local model summary -> optional cloud LLM
|
||||
```
|
||||
|
||||
巡航 UI 永远先展示已有摘要,不等待模型调用。模型摘要在后台补齐,写入缓存后下一轮巡航或刷新时使用。
|
||||
|
||||
## 数据结构
|
||||
|
||||
后端应把原来的 `summary: str` 升级为可追踪的摘要元信息,同时为了兼容前端保留顶层 `summary` 字段。
|
||||
|
||||
建议新增结构:
|
||||
|
||||
```json
|
||||
{
|
||||
"summary": "短摘要文本",
|
||||
"summary_meta": {
|
||||
"source": "provider",
|
||||
"quality": "good",
|
||||
"generated_at": "2026-04-29T00:00:00Z",
|
||||
"content_hash": "sha256:...",
|
||||
"model": null,
|
||||
"language": "zh-CN"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
字段说明:
|
||||
|
||||
| 字段 | 可选值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `source` | `provider` / `extractive` / `local_llm` / `cloud_llm` / `fallback` | 摘要来源 |
|
||||
| `quality` | `good` / `partial` / `poor` | 后端对摘要可用性的判断 |
|
||||
| `generated_at` | ISO 时间 | 模型或规则生成时间 |
|
||||
| `content_hash` | SHA-256 | 用于缓存命中和判断内容变化 |
|
||||
| `model` | 字符串或 `null` | 例如 `gemma3:4b` |
|
||||
| `language` | 语言代码 | 默认 `zh-CN`,也可跟随新闻语言 |
|
||||
|
||||
前端第一阶段不需要显示 `summary_meta`,但可以用于后续调试面板或质量标记。
|
||||
|
||||
## 后端设计
|
||||
|
||||
### NewsSummaryService
|
||||
|
||||
新增 `backend/app/services/news_summary.py`,提供统一入口:
|
||||
|
||||
```python
|
||||
async def resolve_news_summary(item: ParsedNewsItem, *, mode: str) -> NewsSummaryResult:
|
||||
...
|
||||
```
|
||||
|
||||
核心职责:
|
||||
|
||||
- 标准化新闻输入:标题、URL、来源、发布时间、摘要片段、正文片段。
|
||||
- 判断 provider summary 是否可用。
|
||||
- 生成本地规则摘要。
|
||||
- 查询模型摘要缓存。
|
||||
- 在允许时调用本地 Ollama/Gemma。
|
||||
- 在增强模式或手动触发时调用云端 LLM。
|
||||
- 返回摘要文本与 `summary_meta`。
|
||||
|
||||
### 摘要质量判断
|
||||
|
||||
第一版可以用轻量规则:
|
||||
|
||||
- 少于 30 个字符:`poor`
|
||||
- 与标题高度重复:`partial`
|
||||
- 包含明显来源署名或聚合噪声:`partial`
|
||||
- 60-180 个字符且不重复标题:`good`
|
||||
|
||||
伪代码:
|
||||
|
||||
```python
|
||||
def score_summary(title: str, summary: str) -> SummaryQuality:
|
||||
if len(summary.strip()) < 30:
|
||||
return "poor"
|
||||
if normalized_overlap(title, summary) > 0.75:
|
||||
return "partial"
|
||||
if looks_like_source_attribution(summary):
|
||||
return "partial"
|
||||
return "good"
|
||||
```
|
||||
|
||||
### 本地规则摘要
|
||||
|
||||
如果新闻源没有摘要,但有 `content`、`description`、`snippet` 或正文片段:
|
||||
|
||||
- 清理 HTML。
|
||||
- 去掉标题重复内容。
|
||||
- 去掉来源署名、发布时间、图片说明。
|
||||
- 优先取前 1-2 个完整句子。
|
||||
- 控制在 80-140 个中文字符或 40-80 个英文词。
|
||||
|
||||
### 本地 Gemma/Ollama Provider
|
||||
|
||||
不要在业务里写死 Gemma,抽象为 `LocalLLMSummaryProvider`,默认可以指向 Ollama:
|
||||
|
||||
```env
|
||||
NEWS_SUMMARY_PROVIDER=ollama
|
||||
OLLAMA_BASE_URL=http://localhost:11434
|
||||
NEWS_SUMMARY_MODEL=gemma3:4b
|
||||
NEWS_SUMMARY_TIMEOUT_SECONDS=20
|
||||
NEWS_SUMMARY_MAX_INPUT_CHARS=5000
|
||||
NEWS_SUMMARY_MAX_PER_HOUR=60
|
||||
```
|
||||
|
||||
Ollama 请求示例:
|
||||
|
||||
```http
|
||||
POST /api/generate
|
||||
Content-Type: application/json
|
||||
|
||||
{
|
||||
"model": "gemma3:4b",
|
||||
"prompt": "...",
|
||||
"stream": false,
|
||||
"options": {
|
||||
"temperature": 0.2,
|
||||
"num_predict": 180
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
摘要 prompt 要强调“只基于原文”,避免模型补事实:
|
||||
|
||||
```text
|
||||
你是新闻摘要器。只根据输入新闻内容生成摘要,不要添加原文没有的信息。
|
||||
输出中文,1-2 句话,80-140 字。
|
||||
如果原文信息不足,只概括已知事实,不要推测。
|
||||
|
||||
标题:{title}
|
||||
来源:{source}
|
||||
发布时间:{published_at}
|
||||
正文或片段:
|
||||
{content}
|
||||
```
|
||||
|
||||
### 缓存
|
||||
|
||||
需要缓存模型摘要,避免重复花时间和费用。
|
||||
|
||||
缓存 key:
|
||||
|
||||
```text
|
||||
sha256(url + title + published_at + normalized_content)
|
||||
```
|
||||
|
||||
建议新增表或复用系统设置缓存。若要可查询与清理,推荐独立表:
|
||||
|
||||
```text
|
||||
news_summary_cache
|
||||
- id
|
||||
- cache_key
|
||||
- url
|
||||
- title
|
||||
- content_hash
|
||||
- summary
|
||||
- source
|
||||
- quality
|
||||
- provider
|
||||
- model
|
||||
- generated_at
|
||||
- expires_at
|
||||
- failure_count
|
||||
- last_error
|
||||
```
|
||||
|
||||
缓存策略:
|
||||
|
||||
- 同一 `cache_key` 命中后直接返回。
|
||||
- `provider` / `extractive` 可以短期缓存。
|
||||
- `local_llm` / `cloud_llm` 可以长缓存,内容 hash 变化才重算。
|
||||
- LLM 失败后记录 `failure_count`,短时间内不重复调用。
|
||||
|
||||
## 前端与巡航行为
|
||||
|
||||
前端第一阶段只需要继续使用 `item.summary`,不阻塞现有逻辑。
|
||||
|
||||
后续可选增强:
|
||||
|
||||
- `news.js` 在渲染新闻列表时,如果 `summary_meta.quality === "poor"`,可以用更紧凑的标题卡样式。
|
||||
- `news-cruise-adapter.js` 选择巡航项时,可以优先选择 `summary_meta.quality !== "poor"` 的新闻。
|
||||
- `info-card.js` 不显示“AI 生成中”这类文案,避免把系统内部状态暴露给用户。
|
||||
|
||||
如果后端异步生成完成,可以通过下一次 `/api/v1/news/earth-feed` 刷新自然更新。第一版不需要 WebSocket。
|
||||
|
||||
## 调用策略
|
||||
|
||||
默认使用“省钱模式”:
|
||||
|
||||
- 只处理本次 payload 中即将进入巡航队列的前 N 条。
|
||||
- `provider` 或 `extractive` 达到 `good` 时不调用模型。
|
||||
- 本地模型失败时不影响新闻 payload。
|
||||
- 云端 LLM 默认关闭,只允许手动增强或后台配置开启。
|
||||
|
||||
推荐限制:
|
||||
|
||||
| 配置 | 默认值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `NEWS_SUMMARY_MODE` | `economy` | `off` / `economy` / `enhanced` / `manual` |
|
||||
| `NEWS_SUMMARY_CRUISE_PREFETCH_LIMIT` | `10` | 每次新闻 payload 预热多少条巡航摘要 |
|
||||
| `NEWS_SUMMARY_MAX_PER_HOUR` | `60` | 本地模型每小时最多处理数量 |
|
||||
| `NEWS_SUMMARY_CLOUD_MAX_PER_DAY` | `20` | 云端 LLM 每天最多处理数量 |
|
||||
| `NEWS_SUMMARY_TIMEOUT_SECONDS` | `20` | 单条模型摘要超时 |
|
||||
| `NEWS_SUMMARY_MAX_INPUT_CHARS` | `5000` | 输入截断上限 |
|
||||
|
||||
## Gemma 本地部署建议
|
||||
|
||||
Gemma 适合作为“本地省钱层”,但不应成为强绑定依赖。建议通过 Ollama 接入,未来可切换 Qwen、Llama 或其他本地模型。
|
||||
|
||||
开发环境:
|
||||
|
||||
```bash
|
||||
ollama pull gemma3:4b
|
||||
ollama serve
|
||||
```
|
||||
|
||||
集成原则:
|
||||
|
||||
- 后端只依赖 Ollama HTTP API,不直接依赖 Gemma SDK。
|
||||
- 模型名称来自配置,不写死在代码里。
|
||||
- 健康检查访问 `/api/tags` 或执行一条极短测试 prompt。
|
||||
- 如果 Ollama 不可用,摘要管线自动退回 `provider` / `extractive`。
|
||||
|
||||
中文新闻较多时,需要单独评估 Gemma 与 Qwen 系本地模型的中文摘要质量。不要只看单条效果,至少抽样 50 条新闻比较:
|
||||
|
||||
- 事实准确性
|
||||
- 中文自然度
|
||||
- 长度稳定性
|
||||
- 延迟
|
||||
- 是否会补充原文没有的信息
|
||||
|
||||
## 云端 LLM 兜底
|
||||
|
||||
云端 LLM 不作为默认路径,只用于:
|
||||
|
||||
- 用户点击“增强摘要”。
|
||||
- 管理员开启增强模式。
|
||||
- 本地模型连续失败且新闻进入重点巡航队列。
|
||||
|
||||
云端结果同样写入 `news_summary_cache`,并受每日限额控制。
|
||||
|
||||
## 分阶段实施
|
||||
|
||||
### 第一阶段:零成本摘要质量增强
|
||||
|
||||
- 在 `earth_news.py` 中引入 `summary_meta`。
|
||||
- 增加 provider summary 质量判断。
|
||||
- 增加本地规则摘要兜底。
|
||||
- `/api/v1/news/earth-feed` 保持兼容,继续返回顶层 `summary`。
|
||||
- 前端无需大改。
|
||||
|
||||
验收标准:
|
||||
|
||||
- 没有摘要的新闻也能尽量得到短摘要。
|
||||
- `summary_meta.source` 和 `summary_meta.quality` 可用于调试。
|
||||
- 现有新闻面板和巡航模式不破坏。
|
||||
|
||||
### 第二阶段:本地 Gemma/Ollama 摘要
|
||||
|
||||
- 新增 `LocalLLMSummaryProvider`。
|
||||
- 接入 Ollama `/api/generate`。
|
||||
- 添加超时、输入截断、错误退避。
|
||||
- 增加模型摘要缓存。
|
||||
- 巡航 payload 后台预热前 N 条摘要。
|
||||
|
||||
验收标准:
|
||||
|
||||
- Ollama 可用时,低质量摘要能被本地模型增强。
|
||||
- Ollama 不可用时,新闻接口仍然正常返回。
|
||||
- 同一新闻不会重复调用模型。
|
||||
|
||||
### 第三阶段:设置与可观测性
|
||||
|
||||
- 在设置中增加新闻摘要模式:
|
||||
- `关闭`
|
||||
- `省钱模式`
|
||||
- `增强模式`
|
||||
- `仅手动`
|
||||
- 增加本地模型连通性检查。
|
||||
- 暴露缓存命中率、模型调用次数、失败次数。
|
||||
- 日志记录摘要来源和失败原因。
|
||||
|
||||
验收标准:
|
||||
|
||||
- 用户可以不改环境变量就知道本地摘要服务是否可用。
|
||||
- 管理员能看出成本和失败情况。
|
||||
|
||||
### 第四阶段:云端 LLM 兜底
|
||||
|
||||
- 接入现有 AI Provider 或新增 cloud summary provider。
|
||||
- 增加每日限额与手动增强入口。
|
||||
- 对云端生成结果落缓存。
|
||||
|
||||
验收标准:
|
||||
|
||||
- 云端调用可控、可关闭、可限流。
|
||||
- 云端失败不影响巡航。
|
||||
|
||||
## 风险与防护
|
||||
|
||||
| 风险 | 防护 |
|
||||
| --- | --- |
|
||||
| 本地模型生成不存在的事实 | prompt 明确禁止扩写;摘要只作为原文概括;保留来源链接 |
|
||||
| 本地模型慢导致新闻接口卡住 | 模型摘要异步化;接口先返回已有摘要 |
|
||||
| 成本失控 | 默认不启用云端;按小时/天限流;缓存命中优先 |
|
||||
| 摘要语言不一致 | 配置目标语言,默认 `zh-CN` |
|
||||
| 新闻源正文不足 | 只概括标题和片段,不强行扩写 |
|
||||
| 模型服务不可用 | 自动回退,不影响巡航主流程 |
|
||||
|
||||
## 推荐优先级
|
||||
|
||||
先做第一阶段和第二阶段的最小闭环:
|
||||
|
||||
1. `summary_meta` + 质量判断。
|
||||
2. 本地规则摘要。
|
||||
3. Ollama provider。
|
||||
4. 缓存。
|
||||
5. 巡航前 N 条异步预热。
|
||||
|
||||
云端 LLM 和设置页可以后置。这样能先验证“摘要缺失比例、本地模型质量、实际延迟”三个关键问题,再决定是否投入更重的 UI 与云端增强。
|
||||
@@ -96,3 +96,11 @@ GEO 轨道点数高,采样率需要按轨道类型分层。
|
||||
2. 解锁后轨道立即清除
|
||||
3. 不同轨道类型下点数可控
|
||||
4. 页面切换回来不会闪出旧轨道残留
|
||||
|
||||
## Satellite Footprint Follow-Up Items
|
||||
|
||||
从技术文档迁出的 footprint 后续项,作为卫星覆盖能力的计划 backlog:
|
||||
|
||||
1. 为 `iridium-next` 新建独立 footprint adapter。
|
||||
2. 在 UI 上补一个只读提示,让用户知道当前卫星是否支持 footprint。
|
||||
3. 如果未来拿到 GEO beam contour / operator metadata,再为 GEO 开 operator-specific footprint。
|
||||
|
||||
@@ -25,6 +25,49 @@
|
||||
- 状态和渲染更新散落在多个模块
|
||||
- 后续再加新图层时容易复制旧逻辑
|
||||
|
||||
## Current High-Frequency Risks
|
||||
|
||||
### 1. Visual state and business state drift apart
|
||||
|
||||
Earth 里最常见的 bug 不是“没渲染”,而是状态没有一起收口:
|
||||
|
||||
- 图层关了,tooltip 还在
|
||||
- 锁定对象隐藏了,info card 还在
|
||||
- legend 没跟图层切换
|
||||
- loading 已结束,但按钮还像没开
|
||||
|
||||
后续架构治理需要把这类同步责任从临时 UI patch 转为统一状态流。
|
||||
|
||||
### 2. HUD layout fixes skip structure analysis
|
||||
|
||||
Earth HUD 历史上反复出现:
|
||||
|
||||
- 面板只剩一条缝
|
||||
- markdown 被裁掉
|
||||
- tabs / iframe 被 `overflow: hidden` 吃掉
|
||||
|
||||
这类问题应纳入布局治理计划,而不是散落在单个功能改动里临时修。
|
||||
|
||||
### 3. Transitional paths keep accumulating
|
||||
|
||||
Earth 已经经历过多轮 HUD、toolbar、media panel 重构,容易留下:
|
||||
|
||||
- 旧 helper
|
||||
- 旧 class
|
||||
- 旧 fallback 逻辑
|
||||
- 已废弃变体
|
||||
|
||||
架构分离阶段需要把 cleanup pass 作为计划项,而不是让技术上下文承担提醒职责。
|
||||
|
||||
### 4. Cruise logic and business events couple too deeply
|
||||
|
||||
巡航相关风险是通用巡航层继续混入业务事件细节,导致 BGP、新闻、卫星、海缆各自复制一套状态机。
|
||||
|
||||
架构目标应保持:
|
||||
|
||||
- 通用巡航层管理目标、队列、focus、停留、隐藏和切换
|
||||
- 业务模块只提供队列、坐标、卡片内容和高亮副作用
|
||||
|
||||
## Target Architecture
|
||||
|
||||
Earth 对每类对象都尽量拆成三层:
|
||||
|
||||
249
docs/plans/earth-vessel-rendering-performance-plan.md
Normal file
249
docs/plans/earth-vessel-rendering-performance-plan.md
Normal file
@@ -0,0 +1,249 @@
|
||||
# Earth Vessel Rendering Performance Plan
|
||||
|
||||
## 当前状态
|
||||
|
||||
该计划的前端核心部分已经在 `0.44.1` 落地,但最终实现不是原文设想的 `InstancedBufferGeometry` quad,而是更稳的分桶 `THREE.Points` 方案:
|
||||
|
||||
- 普通船只按 moving / anchored 和 `VESSEL_COURSE_BINS` 航向分桶,使用 `PointsMaterial` 批量绘制。
|
||||
- 航行船只仍是带方向的三角形,停泊或低速船只仍是圆点。
|
||||
- hover / locked 不再放大成世界尺寸 Sprite,而是在原点位叠加同尺寸单点 glow overlay。
|
||||
- picking 改为屏幕空间命中,拖动和惯性期间跳过 hover picking。
|
||||
- 普通态关闭 glow,交互态才显示 glow,降低 overdraw 并让默认地图更干净。
|
||||
|
||||
后续如果需要全球 AIS 或更高船只密度,再评估是否从分桶 `Points` 升级到真正 instanced quad 或视口 bbox / LOD。
|
||||
|
||||
## 背景
|
||||
|
||||
Earth 船只图层已经形成了一套较好的视觉语言:
|
||||
|
||||
- 航行船只使用三角形标记
|
||||
- 标记按航向旋转
|
||||
- 停泊或低速船只使用圆点
|
||||
- 不同船型使用不同颜色
|
||||
- hover / locked 状态有 glow、透明度和聚焦反馈
|
||||
- 标记带有轻微 glow / soft edge,和 Earth HUD 的观感一致
|
||||
|
||||
当前性能问题不应通过降级成无方向、无船型语义的普通小点来解决。目标是在保留现有观赏性的前提下,把底层从“每艘船一个 Sprite 对象”优化为批量绘制和轻量交互。
|
||||
|
||||
## 当前问题判断
|
||||
|
||||
卫星图层能承载几万个对象,是因为它主要走 `THREE.Points` / `BufferGeometry` / instanced trail 路径。船只图层目前每艘船创建一个 `THREE.Sprite` 和独立 `SpriteMaterial`,这会带来:
|
||||
|
||||
- draw call 随船只数量增长
|
||||
- 透明 sprite 排序和 overdraw 成本上升
|
||||
- 每帧遍历所有船只更新 opacity / scale / visible
|
||||
- pointer move 时对船只 sprite 做对象级 raycast
|
||||
- hover reset 时全量遍历 marker
|
||||
|
||||
因此,即使免费 BarentsWatch AIS 只开放挪威周边数据,前端仍可能因为对象级 sprite、raycast 和每帧全量更新出现地球拖动卡顿。
|
||||
|
||||
## 目标
|
||||
|
||||
1. 保留当前船只标记的视觉质量。
|
||||
2. 保留 hover tooltip、点击详情、lock、轨迹等交互。
|
||||
3. 显著降低 draw call、每帧 JS 遍历和 pointer picking 成本。
|
||||
4. 为后续全球 AIS 或更高船只数量预留扩展空间。
|
||||
|
||||
## 非目标
|
||||
|
||||
- 不把船只降级为普通无方向 `Points`。
|
||||
- 不取消船型颜色、航向三角和停泊圆点。
|
||||
- 不为了短期性能直接删除 hover / click 交互。
|
||||
|
||||
## Phase 1:交互路径止血
|
||||
|
||||
这一阶段不改视觉,只减少 pointer move 和 hover 状态开销。
|
||||
|
||||
### 1. 拖动和惯性期间跳过船只 picking
|
||||
|
||||
地球拖动时用户主要关注视角变化,不需要每个 pointer move 都命中船只。
|
||||
|
||||
处理方式:
|
||||
|
||||
- `isDragging === true` 时跳过船只 hover picking。
|
||||
- 惯性旋转期间也跳过船只 hover picking。
|
||||
- 拖动结束后再恢复 hover 检测。
|
||||
|
||||
### 2. vessel hover picking 节流
|
||||
|
||||
对船只 hover 命中增加节流,例如 `80ms ~ 120ms` 一次。鼠标高速移动时复用上一次 hover 状态,不在每个 pointer event 上都做 raycast。
|
||||
|
||||
### 3. hover reset 从全量遍历改为增量更新
|
||||
|
||||
当前 `resetTransientVesselStates()` 会遍历所有船只。改为记录:
|
||||
|
||||
- `hoveredVessel`
|
||||
- `lockedObject`
|
||||
|
||||
当 hover 目标变化时,只更新旧 hover 和新 hover。
|
||||
|
||||
### 4. 点击路径只在 click 时做一次精确 picking
|
||||
|
||||
点击仍保留精确命中,但只在 click 事件里执行,不参与拖动和高频 pointer move。
|
||||
|
||||
## Phase 2:每帧更新减负
|
||||
|
||||
这一阶段仍保留 `Sprite` 外观,但减少每帧对全部 marker 的写操作。
|
||||
|
||||
### 1. `updateVesselVisualState()` 增量化
|
||||
|
||||
当前每帧都会遍历船只并写:
|
||||
|
||||
- `marker.material.opacity`
|
||||
- `marker.scale`
|
||||
- `marker.visible`
|
||||
|
||||
优化方向:
|
||||
|
||||
- 图层关闭时直接 return。
|
||||
- 没有船只时直接 return。
|
||||
- 只有以下状态变化时才更新 marker:
|
||||
- show/hide 变化
|
||||
- hover 变化
|
||||
- locked 变化
|
||||
- camera zoom / distance scale 变化超过阈值
|
||||
- focus dim 状态变化
|
||||
|
||||
### 2. 缓存 distance scale
|
||||
|
||||
`getDistanceScale(camera)` 可以按 camera distance 或 zoom 阈值缓存。缩放没有明显变化时,不必每帧重设所有船只 scale。
|
||||
|
||||
### 3. 降低透明 overdraw
|
||||
|
||||
在不破坏视觉的前提下微调:
|
||||
|
||||
- marker 基础尺寸
|
||||
- glow blur 半径
|
||||
- 最大 size stabilization
|
||||
|
||||
目标是减少屏幕空间重叠面积,而不是改变符号设计。
|
||||
|
||||
## Phase 3:保留视觉的批量渲染(已落地为分桶 Points)
|
||||
|
||||
原设想是把每艘船的视觉从 `THREE.Sprite` 迁移为 instanced sprite batch。实际落地时选择了更稳的分桶 `THREE.Points`:
|
||||
|
||||
- 不依赖自定义 shader。
|
||||
- 不依赖 `Points` 自带 raycaster。
|
||||
- 用 canvas 纹理保留三角、圆点、船型颜色和航向。
|
||||
- 用 hover / locked 单点 overlay 保留交互 glow。
|
||||
|
||||
如果未来全球 AIS 导致分桶 `Points` 仍不够,再升级到 instanced quad。
|
||||
|
||||
### 1. 原候选方案:instanced quad
|
||||
|
||||
每艘船仍然显示为带贴图/软边的 billboard,但底层使用:
|
||||
|
||||
- `THREE.InstancedBufferGeometry`
|
||||
- 每类船只一个或少量 material
|
||||
- per-instance attributes
|
||||
|
||||
可按形状和船型拆 batch:
|
||||
|
||||
- moving cargo
|
||||
- moving tanker
|
||||
- moving passenger
|
||||
- moving fishing
|
||||
- moving military
|
||||
- moving other
|
||||
- anchored / slow dot
|
||||
|
||||
这样 draw call 从“每艘船一个”变为“每类船只一个”。
|
||||
|
||||
### 2. 原候选方案:per-instance attributes
|
||||
|
||||
每个 instance 存:
|
||||
|
||||
- position
|
||||
- color
|
||||
- rotation
|
||||
- scale
|
||||
- opacity
|
||||
- state
|
||||
- mmsi / data index
|
||||
|
||||
hover、locked、dimmed 可通过更新少量 instance attribute 实现,不再逐个修改 material。
|
||||
|
||||
### 3. 当前落地方案:分桶 `THREE.Points`
|
||||
|
||||
当前实现按以下方式复刻视觉:
|
||||
|
||||
- moving 船只按 `VESSEL_COURSE_BINS` 做航向分桶。
|
||||
- anchored / slow 船只使用圆点分桶。
|
||||
- 每个分桶生成一组 `THREE.Points`,共享 `PointsMaterial` 和 canvas 点纹理。
|
||||
- `VESSEL_CONFIG.colors` 仍通过 vertex colors 表示船型颜色。
|
||||
- hover / locked 在原位置叠加同尺寸单点 overlay,普通态不带 glow,交互态才带 glow。
|
||||
|
||||
这样 draw call 从“每艘船一个”变为“每个形状 / 航向分桶一组”,同时避免自定义 shader 的兼容风险。
|
||||
|
||||
### 4. 复刻当前视觉
|
||||
|
||||
视觉上继续使用当前 canvas texture 或等效 shader:
|
||||
|
||||
- moving 使用三角形纹理
|
||||
- anchored 使用圆点纹理
|
||||
- 保留 soft glow
|
||||
- 保留航向 rotation
|
||||
- 保留 hover / locked 放大
|
||||
|
||||
因此用户看到的效果应与当前船只图层基本一致。
|
||||
|
||||
## Phase 4:picking 改造
|
||||
|
||||
批量渲染后不再适合对所有 sprite object 做 `raycaster.intersectObjects()`。
|
||||
|
||||
### 1. 屏幕空间 picking
|
||||
|
||||
参考卫星 picking:
|
||||
|
||||
1. 过滤背面船只。
|
||||
2. 将候选船只世界坐标投影到屏幕。
|
||||
3. 用鼠标位置计算距离。
|
||||
4. 取距离最近且小于半径阈值的船只。
|
||||
|
||||
### 2. 可选空间索引
|
||||
|
||||
如果后续船只数量明显上升,可增加轻量空间索引:
|
||||
|
||||
- 经纬度网格 bucket
|
||||
- 屏幕空间 bucket
|
||||
- viewport bbox 过滤
|
||||
|
||||
第一阶段不必引入复杂索引。
|
||||
|
||||
## Phase 5:数据层和 LOD
|
||||
|
||||
当接入全球 AIS 或船只数量显著增加时,再做数据层优化。
|
||||
|
||||
### 1. 请求视口范围
|
||||
|
||||
前端请求 `/api/v1/visualization/geo/vessels` 时带上当前视口 `bbox`,减少无关船只。
|
||||
|
||||
### 2. 后端排序策略
|
||||
|
||||
从单纯 `received_at desc` 改为综合排序:
|
||||
|
||||
- 数据新鲜度
|
||||
- 船型优先级
|
||||
- 当前视口相关性
|
||||
- 是否正在航行
|
||||
|
||||
### 3. 远景聚合
|
||||
|
||||
远景可显示聚合或 top N,近景展开单船。
|
||||
|
||||
## 验收指标
|
||||
|
||||
1. 船只视觉效果保持当前质量:三角、圆点、颜色、航向、hover、lock 都保留。
|
||||
2. 开启船只图层后拖动地球不应明显掉帧。
|
||||
3. pointer move 不应因为船只 hover 导致卡顿。
|
||||
4. 船只数量达到 `1000` 级别时仍可顺畅旋转地球。
|
||||
5. `renderer.info.render.calls` 相比 Sprite 版本显著下降。
|
||||
6. hover / click 命中体验不低于当前版本。
|
||||
|
||||
## 建议落地顺序
|
||||
|
||||
1. 先做 Phase 1,快速恢复地球拖动手感。
|
||||
2. 再做 Phase 2,减少每帧 JS 写操作。
|
||||
3. Phase 3 和 Phase 4 已按分桶 `THREE.Points` + 屏幕空间 picking 落地。
|
||||
4. Phase 5 等全球船只数据或数量压力出现后再推进。
|
||||
5. 如果分桶 `THREE.Points` 达到瓶颈,再评估 instanced quad。
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
**状态**:规划中
|
||||
**创建日期**:2026-04-27
|
||||
**优先数据源**:BarentsWatch(免费)→ AISHub / MarineTraffic(TODO,付费)
|
||||
**优先数据源**:BarentsWatch AIS(免费但需要 OAuth client credentials)→ AISHub / MarineTraffic(TODO,付费)
|
||||
|
||||
## 已确认决策
|
||||
|
||||
@@ -22,17 +22,17 @@
|
||||
|
||||
| 来源类型 | 典型服务 | 覆盖范围 | 成本 | 状态 |
|
||||
|---------|---------|---------|------|------|
|
||||
| **BarentsWatch Open API** | live.ais.barentswatch.no | 挪威海域实时 | 完全免费 | **当前使用** |
|
||||
| **BarentsWatch AIS API** | live.ais.barentswatch.no | 挪威海域实时 | 免费,需要 AIS API client credentials | **当前使用** |
|
||||
| **AISHub** | aishub.net | 全球实时 | 免费/小额 | TODO:付费接入 |
|
||||
| **MarineTraffic API** | marinetraffic.com | 全球实时 | $50–$500/月 | TODO:评估 tier |
|
||||
| **VesselFinder API** | vesselfinder.com | 全球实时 | $50–$300/月 | TODO:备选 |
|
||||
| **自建 SDR 接收** | RTL-SDR + AIS-catcher | 仅本地 30–50km | 硬件 $30 | 不考虑 |
|
||||
| **NOAA 历史数据** | Marine Cadastre | 美国近海历史 | 免费 | 可用于冷启动 |
|
||||
|
||||
### BarentsWatch API
|
||||
### BarentsWatch AIS API
|
||||
|
||||
- 端点:`https://live.ais.barentswatch.no/v1/latest/combined`
|
||||
- 无需注册,直接 GET,返回挪威近海 2000–5000 艘船只 JSON
|
||||
- 需要在 BarentsWatch developer portal 创建 `AIS - API` client,通过 client credentials 获取 `scope=ais` 的 access token 后请求 AIS endpoint
|
||||
- 字段:mmsi, lat, lon, sog, cog, heading, nav_status, name, vessel_type, flag
|
||||
- 刷新频率:数据约 30–60s 更新一次,可随意轮询
|
||||
|
||||
@@ -49,7 +49,7 @@
|
||||
|
||||
### Phase 0 — 数据源验证与链路打通(1–2 天)
|
||||
|
||||
- 接入 BarentsWatch Open API,验证数据格式与字段
|
||||
- 接入 BarentsWatch AIS API,验证 OAuth token、数据格式与字段
|
||||
- 构建全球 mock 数据生成器(用于前端渲染压测,补充 BarentsWatch 的地域限制)
|
||||
- 确认前端可渲染船只点,整条链路走通
|
||||
|
||||
|
||||
@@ -103,7 +103,7 @@ Goals:
|
||||
|
||||
## Collector Configuration
|
||||
|
||||
`news_live_streams` does not need a separate new page; it reuses the existing data source configuration:
|
||||
`news_live_streams` does not need a separate new page; it reuses Collector Settings under `/settings`:
|
||||
|
||||
- `endpoint`
|
||||
- Channel directory JSON API URL
|
||||
|
||||
@@ -55,6 +55,30 @@ Parameters:
|
||||
| `--allow-lan` | Enable LAN access |
|
||||
| `--verbose` | Show more command output during execution |
|
||||
|
||||
### AI Provider Environment and Builds
|
||||
|
||||
AI Provider runtime configuration can live in `aiprovider/.env` or in matching variables in `~/.zshrc`. `planet.sh` reads simple `export AI_...=...` / `AI_...=...` lines and passes them to the container at startup.
|
||||
|
||||
Changing model, API key, or base URL does not rebuild the image. Restart only AI Provider to pick up runtime configuration changes:
|
||||
|
||||
```bash
|
||||
./planet.sh restart -a
|
||||
```
|
||||
|
||||
For complex shell expansion in `~/.zshrc`, opt in explicitly:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=source ./planet.sh start -a
|
||||
```
|
||||
|
||||
To ignore `~/.zshrc` during troubleshooting:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=0 ./planet.sh start -a
|
||||
```
|
||||
|
||||
The AI Provider Docker build context is intentionally limited to the files required by the service, and `uv sync` uses a BuildKit cache mount so dependency downloads are reused after the first build.
|
||||
|
||||
### Stop
|
||||
|
||||
```bash
|
||||
@@ -178,6 +202,7 @@ Earth is used to observe in a single globe view:
|
||||
- Satellites and orbital trails
|
||||
- Submarine cables and landing points
|
||||
- Compute centers
|
||||
- AIS vessels
|
||||
- Country borders, grid lines, HD texture, cloud layer, terrain
|
||||
- Live news streams and situational news
|
||||
- Search and focused object details
|
||||
@@ -195,6 +220,7 @@ Common layers include:
|
||||
- Submarine cables
|
||||
- Compute centers
|
||||
- BGP observation
|
||||
- AIS vessels
|
||||
- Satellites
|
||||
- Orbital trails
|
||||
- Terrain
|
||||
@@ -205,6 +231,21 @@ Some layers have dependencies:
|
||||
- Trails require Satellites
|
||||
- When HD texture is off, the globe shows the base map and edge glow effect
|
||||
|
||||
### Legend
|
||||
|
||||
The lower-left legend follows the currently focused or enabled layer.
|
||||
|
||||
Current legend modes include:
|
||||
|
||||
- Cables
|
||||
- Satellites
|
||||
- Country borders
|
||||
- Compute centers
|
||||
- BGP
|
||||
- AIS vessels
|
||||
|
||||
AIS vessel legend entries are grouped by vessel type: cargo, tanker, passenger, fishing, military, anchored/slow, and other. Triangle markers represent moving vessels; dots represent anchored or slow vessels.
|
||||
|
||||
### Search
|
||||
|
||||
Earth search finds current globe objects, such as:
|
||||
@@ -233,6 +274,25 @@ The settings panel contains:
|
||||
|
||||
These settings are stored in browser local storage. They revert to defaults if you switch browsers or clear site data.
|
||||
|
||||
### View Controls
|
||||
|
||||
Earth supports mouse, touchpad, and touchscreen interaction.
|
||||
|
||||
Common controls:
|
||||
|
||||
| Action | Result |
|
||||
| --- | --- |
|
||||
| Left-button drag | Rotates the globe |
|
||||
| One-finger drag | Rotates the globe on touch devices |
|
||||
| Mouse wheel | Zooms the view in or out |
|
||||
| Two-finger pinch | Zooms the view on touch devices |
|
||||
| Zoom buttons | Adjust zoom in fixed steps |
|
||||
| Click the zoom percent | Resets to the default zoom |
|
||||
|
||||
When zooming, the top capsule briefly shows the current zoom level, for example `Zoom 180%`. This indicates view zoom only, not data loading progress. Loading status takes priority and will not be interrupted by zoom feedback.
|
||||
|
||||
Drag sensitivity adjusts automatically based on the current zoom. Around the default view it keeps the normal rotation feel; when zoomed in, dragging becomes progressively finer for inspecting a region, vessel, satellite, or BGP event; when zoomed out, dragging is slightly faster for global browsing.
|
||||
|
||||
### Cruise Mode
|
||||
|
||||
Cruise mode makes Earth automatically cycle through focus targets.
|
||||
@@ -308,7 +368,7 @@ Common pages:
|
||||
| --- | --- | --- |
|
||||
| Dashboard | `/admin` | System overview |
|
||||
| Earth | `/earth` | Opens the public Earth page |
|
||||
| Data Sources | `/datasources` | Manage data sources and trigger collection |
|
||||
| Data Sources | `/datasources` | View data sources and trigger collection |
|
||||
| Collected Data | `/data` | View collected data |
|
||||
| BGP Observation | `/bgp` | BGP situational data |
|
||||
| System Alerts | `/alerts/system` | System-level alerts |
|
||||
@@ -321,17 +381,21 @@ Common pages:
|
||||
|
||||
### Data Sources
|
||||
|
||||
`/datasources` shows and manages collection sources.
|
||||
`/datasources` shows collection sources and triggers collection. It is now a data source directory that lists built-in and custom sources in one table.
|
||||
|
||||
Common operations:
|
||||
|
||||
- View data source status
|
||||
- Trigger collection
|
||||
- View recent collection tasks
|
||||
- Adjust configuration
|
||||
- Open the read-only detail drawer for endpoint, headers, runtime config, and built-in/custom source type
|
||||
|
||||
If a category of objects is missing on Earth, start here to confirm the data source is available.
|
||||
|
||||
The data source name opens an information drawer only. Endpoint, credentials, headers, and custom source configuration are maintained under `/settings` collector settings.
|
||||
|
||||
When collection tasks are running, the progress area shows a clickable `Collecting N` pill. Clicking it opens a modal with each running task's phase, progress, and processed count.
|
||||
|
||||
### Collected Data
|
||||
|
||||
`/data` shows the collected data table.
|
||||
@@ -369,10 +433,50 @@ Current common uses:
|
||||
|
||||
- System settings
|
||||
- TV live stream source configuration
|
||||
- Data source configuration entry points
|
||||
- Collector settings
|
||||
- External integrations and AI Provider configuration
|
||||
|
||||
Available configuration depends on the current user's role.
|
||||
|
||||
#### Collector Settings
|
||||
|
||||
`/settings?tab=collector_credentials` is currently displayed as Collector Settings. It manages connection settings for all collectors, not only credentials.
|
||||
|
||||
Use it to:
|
||||
|
||||
1. Select a collector from the dropdown.
|
||||
2. Review tags such as `Requires credentials`, module, enabled state, and `Unchecked` / `Available` / `Unavailable`.
|
||||
3. Click the plug icon next to the selector to run a health check.
|
||||
4. Edit endpoint, request headers, timeout, and retry settings.
|
||||
5. Save the collector settings.
|
||||
|
||||
Free collectors are checked by requesting their endpoint directly. Credentialed collectors use their credential provider. If endpoint or credential fingerprint changes after the last successful validation, the collector must be checked again.
|
||||
|
||||
The system treats a collector as connected when the current configuration has either collected data successfully or passed the manual connection check.
|
||||
|
||||
#### BarentsWatch AIS Credentials
|
||||
|
||||
`BarentsWatch AIS` is a credentialed built-in collector. Its credential card appears above the basic configuration card.
|
||||
|
||||
Configured fields:
|
||||
|
||||
- `Client ID`
|
||||
- `Client Secret`
|
||||
- `Endpoint`
|
||||
|
||||
If a secret is already configured, the input shows a masked preview. Keeping that preview unchanged preserves the stored secret; entering a new value replaces it.
|
||||
|
||||
BarentsWatch AIS credentials can be read from:
|
||||
|
||||
1. Collector settings saved in the console.
|
||||
2. Backend environment variables:
|
||||
- `BARENTSWATCH_CLIENT_ID`
|
||||
- `BARENTSWATCH_CLIENT_SECRET`
|
||||
- historical spellings: `BARRENTSWATCH_CLIENT_ID`, `BARRENTSWATCH_CLIENT_SECRET`
|
||||
3. matching `export` lines in `~/.zshrc`.
|
||||
|
||||
If connection fails, the page opens the credential guide. The guide can be regenerated through AI Provider or reset to the default guide. The default guide points users to the official BarentsWatch tutorial and emphasizes selecting `AIS - API`, not the regular `BarentsWatch - API`.
|
||||
|
||||
### System Logs
|
||||
|
||||
`/logs` views system logs. If the menu item is not visible, the current user likely lacks the required role.
|
||||
|
||||
@@ -24,6 +24,12 @@ This script checks and syncs common dependencies, and generates if missing:
|
||||
- `aiprovider/.env`
|
||||
- `frontend/.env.local`
|
||||
|
||||
Personal AI Provider configuration can also live in `~/.zshrc`. `planet.sh` reads simple `export AI_...=...` / `AI_...=...` lines and passes them to the AI Provider container. After changing model, key, or base URL, restart only AI Provider:
|
||||
|
||||
```bash
|
||||
./planet.sh restart -a
|
||||
```
|
||||
|
||||
## 1. Start Services
|
||||
|
||||
From the repository root:
|
||||
@@ -73,6 +79,7 @@ Once in, verify:
|
||||
- The globe renders correctly
|
||||
- The right-side layer panel can toggle layers on/off
|
||||
- Search can find cables, satellites, compute centers, BGP events
|
||||
- Mouse drag, wheel zoom, and zoom percent feedback work correctly
|
||||
- Settings panel can switch cruise mode, day/night mode, satellite display style
|
||||
|
||||
## 4. Open the Console
|
||||
@@ -87,7 +94,7 @@ The console manages data sources, collected data, situational observation, alert
|
||||
|
||||
First-time inspection checklist:
|
||||
|
||||
- `/datasources`: data source configuration and collection status
|
||||
- `/datasources`: data source directory and collection triggers; endpoint, headers, and credentials are configured under `/settings` collector settings
|
||||
- `/data`: collected data
|
||||
- `/bgp`: BGP situational view
|
||||
- `/alerts/system`: system alerts
|
||||
|
||||
@@ -5,7 +5,6 @@
|
||||
- 现在代码是怎么组织的
|
||||
- 当前入口在哪
|
||||
- 状态和组件如何工作
|
||||
- 后续改动应该沿着哪条实现边界继续走
|
||||
|
||||
适合放入这里的内容:
|
||||
|
||||
@@ -17,12 +16,14 @@
|
||||
- Earth 图层样式属性索引
|
||||
- 后端运行控制
|
||||
- 采集器现状
|
||||
- 采集器设置与连接验证
|
||||
- 采集格式约定
|
||||
|
||||
## 使用入口
|
||||
|
||||
- [quickstart.md](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md):从零启动 Planet 的最短路径
|
||||
- [manual.md](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md):控制台、`planet.sh`、Earth 和 Docs 的完整使用手册
|
||||
- [datasource-collector-settings-connectivity.md](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md):数据源目录、采集器设置、连接验证、BarentsWatch 凭证链路
|
||||
|
||||
不适合放入这里的内容:
|
||||
|
||||
|
||||
@@ -84,6 +84,8 @@ async def run(self, db):
|
||||
| HuggingFace Spaces | space | Demo应用 | 1天 |
|
||||
| PeeringDB | ixp/network/facility | 互联网交换点/网络/机房 | 1-2天 |
|
||||
| TeleGeography | submarine_cable | 海底光缆信息 | 7天 |
|
||||
| Space-Track TLE | satellite_tle | 卫星轨道 TLE 数据 | 依采集器配置 |
|
||||
| BarentsWatch AIS | vessel | 船只位置、航速、航向、MMSI 等 AIS 数据 | 依采集器配置 |
|
||||
|
||||
## 四、数据格式 (统一存储到 CollectedData 表)
|
||||
|
||||
@@ -200,6 +202,30 @@ def start_scheduler():
|
||||
|
||||
**核心文件**: `backend/app/services/scheduler.py`
|
||||
|
||||
### 成功采集与连接状态
|
||||
|
||||
内置采集器成功采集后,调度器会记录当前有效配置已通过连接验证:
|
||||
|
||||
```python
|
||||
if datasource.last_status == "success":
|
||||
effective_candidate = await get_builtin_effective_candidate(db, datasource.source)
|
||||
checksum, _ = await build_builtin_connectivity_checksum(...)
|
||||
await save_connectivity_success(
|
||||
db,
|
||||
datasource.source,
|
||||
checksum,
|
||||
{"status_code": None},
|
||||
connected_by="collection",
|
||||
)
|
||||
```
|
||||
|
||||
这个记录用于控制台“采集器设置”中的连接状态判断:如果当前配置和成功采集时的 checksum 一致,就视为已连接,不要求用户再手动点击连接按钮。只有 endpoint、请求头、基础配置或凭证指纹变化时,才需要重新验证。
|
||||
|
||||
相关实现见:
|
||||
|
||||
- [datasource_connectivity.py](/home/ray/dev/linkong/planet/backend/app/services/datasource_connectivity.py)
|
||||
- [datasource-collector-settings-connectivity.md](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)
|
||||
|
||||
## 八、相关代码文件
|
||||
|
||||
```
|
||||
@@ -211,13 +237,86 @@ backend/app/services/collectors/
|
||||
├── epoch_ai.py # Epoch AI采集器
|
||||
├── huggingface.py # HuggingFace采集器
|
||||
├── peeringdb.py # PeeringDB采集器
|
||||
└── telegeraphy.py # TeleGeography海底光缆采集器
|
||||
├── telegeraphy.py # TeleGeography海底光缆采集器
|
||||
└── vessel_ais.py # BarentsWatch AIS 船只采集器
|
||||
|
||||
backend/app/models/
|
||||
└── collected_data.py # 统一数据模型
|
||||
```
|
||||
|
||||
## 九、数据使用场景
|
||||
## 九、凭证型采集器
|
||||
|
||||
部分采集器需要外部服务凭证,例如:
|
||||
|
||||
| 采集器 | credential provider | 凭证来源 |
|
||||
| --- | --- | --- |
|
||||
| `barentswatch_vessels` | `barentswatch` | 控制台采集器设置、环境变量、`~/.zshrc` |
|
||||
| `spacetrack_tle` | `spacetrack` | 环境变量、`~/.zshrc` |
|
||||
|
||||
### BarentsWatch AIS
|
||||
|
||||
BarentsWatch AIS 的凭证解析统一在:
|
||||
|
||||
- [barentswatch.py](/home/ray/dev/linkong/planet/backend/app/services/barentswatch.py)
|
||||
|
||||
`VesselAISCollector` 只负责采集和转换 AIS 数据,不再自己读取环境变量或拼 token 请求。它通过:
|
||||
|
||||
- `resolve_barentswatch_config()`
|
||||
- `fetch_barentswatch_access_token()`
|
||||
|
||||
获取运行时配置。
|
||||
|
||||
解析优先级:
|
||||
|
||||
1. `DataSourceConfig.auth_config`
|
||||
2. `DataSourceConfig.config`
|
||||
3. 环境变量
|
||||
4. `~/.zshrc`
|
||||
|
||||
支持变量:
|
||||
|
||||
```bash
|
||||
export BARENTSWATCH_CLIENT_ID="..."
|
||||
export BARENTSWATCH_CLIENT_SECRET="..."
|
||||
```
|
||||
|
||||
并兼容历史拼写:
|
||||
|
||||
```bash
|
||||
export BARRENTSWATCH_CLIENT_ID="..."
|
||||
export BARRENTSWATCH_CLIENT_SECRET="..."
|
||||
```
|
||||
|
||||
连接验证会先请求 `https://id.barentswatch.no/connect/token` 获取 `scope=ais` 的 access token,再用 `Authorization: Bearer <token>` 请求 AIS endpoint。
|
||||
|
||||
## 十、采集器设置与连接验证
|
||||
|
||||
控制台的“采集器设置”页提供所有内置采集器的 endpoint、请求头、超时、重试和凭证配置。连接验证不是只看前端按钮状态,而是由后端计算 checksum:
|
||||
|
||||
- endpoint
|
||||
- auth type
|
||||
- headers
|
||||
- config
|
||||
- credential provider
|
||||
- 凭证指纹
|
||||
|
||||
相关 API:
|
||||
|
||||
```http
|
||||
GET /api/v1/datasources/configs/all
|
||||
POST /api/v1/datasources/configs/builtin/connection-status
|
||||
POST /api/v1/datasources/configs/builtin/connect
|
||||
POST /api/v1/settings/integrations/barentswatch/connect
|
||||
GET /api/v1/settings/credential-guides/{provider}
|
||||
POST /api/v1/settings/credential-guides/{provider}/generate
|
||||
POST /api/v1/settings/credential-guides/{provider}/reset
|
||||
```
|
||||
|
||||
更多细节见:
|
||||
|
||||
- [datasource-collector-settings-connectivity.md](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)
|
||||
|
||||
## 十一、数据使用场景
|
||||
|
||||
采集的数据最终会:
|
||||
|
||||
@@ -225,7 +324,7 @@ backend/app/models/
|
||||
2. **态势分析** - 统计全球算力分布、增长趋势
|
||||
3. **告警系统** - 检测重要节点变化
|
||||
|
||||
## 十、采集器注册机制
|
||||
## 十二、采集器注册机制
|
||||
|
||||
采集器在应用启动时自动注册:
|
||||
|
||||
@@ -247,7 +346,7 @@ collector_registry.register(TeleGeographyCableSystemCollector())
|
||||
|
||||
**核心文件**: `backend/app/services/collectors/registry.py`
|
||||
|
||||
## 十一、触发采集
|
||||
## 十三、触发采集
|
||||
|
||||
### 方式一:定时触发
|
||||
系统启动时,APScheduler会自动根据各采集器的`frequency_hours`设置定时任务。
|
||||
|
||||
@@ -69,7 +69,7 @@ last_status = datasource.last_status
|
||||
```
|
||||
datasources SELECT → 主数据,必须
|
||||
_load_latest_running_tasks → 必须(进行中状态 + stale check)
|
||||
_load_datasource_endpoint_overrides → 必须(endpoint 覆盖,编辑内置 collector 时需要默认值)
|
||||
_load_datasource_endpoint_overrides → 必须(endpoint 覆盖,列表详情和采集器设置需要显示当前有效地址)
|
||||
```
|
||||
|
||||
3 个查询(原来 5 个),后两个顺序执行(running tasks 先完成用于 stale check,endpoint overrides 轻量)。
|
||||
@@ -87,7 +87,6 @@ fetchData() // 总是立即刷 → 与上面的延迟刷重叠
|
||||
|
||||
// 修复后(二者互斥)
|
||||
if (res.data.task_id) {
|
||||
setTaskProgress(...)
|
||||
fetchData() // 有 task_id:立即刷一次
|
||||
} else {
|
||||
window.setTimeout(fetchData, 800) // 无 task_id:等 800ms 再刷一次
|
||||
|
||||
@@ -22,10 +22,10 @@
|
||||
|
||||
| Action 名称 | 用途 | `planet.sh` 命令 | 备注 |
|
||||
| --- | --- | --- | --- |
|
||||
| `restart-backend` | 只重启后端 API | `./planet.sh restart -b` | 推荐作为 UI 触发重启流程的第一阶段实现。 |
|
||||
| `restart-backend` | 只重启后端 API | `./planet.sh restart -b` | 页面通常短暂失联后由 `/health` 轮询恢复。 |
|
||||
| `restart-frontend` | 只重启前端开发服务器 | `./planet.sh restart -f` | 页面入口会短暂不可用;UI 通过前端入口探测恢复后刷新。 |
|
||||
| `restart-database` | 重启 PostgreSQL 和 Redis 容器 | `./planet.sh restart -d` | 适合数据库/缓存需要受控重启但不希望重启 UI 的场景。 |
|
||||
| `restart-system` | 重启整个应用栈 | `./planet.sh restart` | 前端会短暂中断;UI 应进入引导恢复模式。 |
|
||||
| `restart-frontend` | 只重启前端开发服务器 | `./planet.sh restart -f` | 谨慎使用;UI 连续性弱于只重启后端。 |
|
||||
| `restart-backend-port` | 在指定端口重启后端 | `./planet.sh restart -b <port>` | 执行前必须由后端校验端口。 |
|
||||
| `restart-frontend-port` | 在指定端口重启前端 | `./planet.sh restart -f <port>` | 执行前必须由后端校验端口。 |
|
||||
| `health-check` | 读取当前服务健康状态 | `./planet.sh health` | 安全的只读运维动作。 |
|
||||
@@ -269,21 +269,25 @@ health-check -> ["./planet.sh", "health"]
|
||||
- `服务已恢复,正在刷新页面`
|
||||
- `恢复超时,请手动检查服务状态`
|
||||
|
||||
## 第一阶段建议
|
||||
## 当前 Dashboard 实现
|
||||
|
||||
第一阶段只实现:
|
||||
Dashboard 当前已实现:
|
||||
|
||||
- `restart-backend`
|
||||
- `restart-frontend`
|
||||
- `restart-ai-provider`
|
||||
- `restart-database`
|
||||
- `restart-system`
|
||||
- `super_admin` 权限门禁
|
||||
- 任务创建接口
|
||||
- Redis 任务状态
|
||||
- 前端确认 modal
|
||||
- 前端 `/health` 轮询
|
||||
- 后端 `/health` 轮询
|
||||
- 前端入口轮询
|
||||
- 恢复后自动刷新页面
|
||||
|
||||
第一阶段不要实现:
|
||||
暂不实现:
|
||||
|
||||
- 完整 `./planet.sh restart`
|
||||
- 原始 shell 命令透传
|
||||
- 任意服务控制
|
||||
- 完整终端 stdout 流式输出
|
||||
@@ -305,17 +309,18 @@ health-check -> ["./planet.sh", "health"]
|
||||
|
||||
### 前端
|
||||
|
||||
1. 在 dashboard 为 `super_admin` 增加 `重启后端` 控件
|
||||
1. 在 dashboard 为 `super_admin` 增加 `重启服务` 控件
|
||||
2. 发送前展示确认 modal
|
||||
3. 提交后将 modal 切换为阻塞式重启状态
|
||||
4. 轮询 `/health` 直到确认后端恢复
|
||||
5. 连续健康检查成功后自动刷新页面
|
||||
6. 展示简短阶段日志,而不是原始终端流
|
||||
4. 后端重启使用 `/health` 轮询确认恢复
|
||||
5. 前端重启和完全重启使用前端入口探测确认恢复
|
||||
6. 连续健康检查成功后自动刷新页面
|
||||
7. 展示简短阶段日志,而不是原始终端流
|
||||
|
||||
### 运维说明
|
||||
|
||||
1. 第一阶段目标应限定为只重启后端
|
||||
2. 前端重启初期保持在范围外
|
||||
1. 优先使用局部重启,只有确实需要时才执行完全重启
|
||||
2. 前端重启会打断当前页面入口,必须进入恢复等待状态
|
||||
3. 命令执行必须始终从仓库根目录发起
|
||||
4. API 边界只能传递固定 action 名称
|
||||
|
||||
@@ -324,10 +329,5 @@ health-check -> ["./planet.sh", "health"]
|
||||
- 拒绝任何不在白名单中的 action。
|
||||
- 如果增加带端口 action,端口必须校验为 `1..65535` 的整数。
|
||||
- 从仓库根目录解析命令,确保 `planet.sh` 的工作目录稳定。
|
||||
- detached runner 使用 `zsh -ic` 执行白名单命令,确保 `~/.zshrc` 中的本地环境变量进入重启流程。
|
||||
- 记录请求 action、操作者身份、执行开始时间和结果。
|
||||
|
||||
## 实现建议
|
||||
|
||||
- UI 触发重启流程时,优先实现 `restart-backend`。
|
||||
- 不要依赖当前 API 请求进程在触发自身重启后继续输出完整日志。
|
||||
- 主 UX 使用任务记录加轮询/健康检查恢复流程,而不是原始终端流。
|
||||
|
||||
327
docs/technical/zh/datasource-collector-settings-connectivity.md
Normal file
327
docs/technical/zh/datasource-collector-settings-connectivity.md
Normal file
@@ -0,0 +1,327 @@
|
||||
# 采集器设置与连接验证
|
||||
|
||||
## 背景
|
||||
|
||||
控制台现在把“数据源目录”和“采集器配置”拆开:
|
||||
|
||||
- `/datasources`
|
||||
- 展示所有数据源,包括内置和自定义。
|
||||
- 点击名称只打开信息抽屉。
|
||||
- 负责查看状态、触发采集和查看采集中任务。
|
||||
- `/settings?tab=collector_credentials`
|
||||
- 显示为“采集器设置”。
|
||||
- 负责 endpoint、请求头、基础参数和凭证配置。
|
||||
- 所有采集器都提供连接按钮,用于健康检查。
|
||||
|
||||
这样做是为了减少首次使用时的认知分裂:接口地址、请求头、凭证和自定义源配置都属于“采集器设置”,而不是散落在数据源列表和系统设置多个入口里。
|
||||
|
||||
## 用户侧规则
|
||||
|
||||
连接状态不是前端样式状态,而是由后端根据配置 checksum 和已验证记录判断。
|
||||
|
||||
一个内置采集器被视为“已连接”需要满足任一条件:
|
||||
|
||||
- 当前配置已经成功采集过数据。
|
||||
- 当前配置点击过连接按钮,并且后端验证成功。
|
||||
|
||||
如果 endpoint、请求头、基础配置或凭证指纹相对上次验证成功时发生变化,状态会回到“需要重新连接”。
|
||||
|
||||
## 前端入口
|
||||
|
||||
### 数据源目录
|
||||
|
||||
文件:
|
||||
|
||||
- [DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx)
|
||||
- [index.css](/home/ray/dev/linkong/planet/frontend/src/index.css)
|
||||
|
||||
当前行为:
|
||||
|
||||
- 内置数据源和自定义数据源合并为 `UnifiedDataSource` 列表。
|
||||
- 表格只保留查看、采集和状态类操作。
|
||||
- 名称点击打开只读抽屉。
|
||||
- 抽屉中展示:
|
||||
- 是否内置
|
||||
- 是否启用
|
||||
- 模块、优先级、频率
|
||||
- endpoint
|
||||
- 请求头
|
||||
- 基础配置
|
||||
- 是否需要凭证
|
||||
- 有任务运行时,顶部进度区显示 `采集中 N` 可点击标签。
|
||||
- 点击 `采集中 N` 打开任务列表弹窗,显示各任务进度。
|
||||
|
||||
`data-source-bulk-toolbar__running-pill` 是“采集中”标签的样式入口。它和其他状态标签同排,但通过 hover、箭头和蓝色描边表达可交互性。
|
||||
|
||||
### 采集器设置
|
||||
|
||||
文件:
|
||||
|
||||
- [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx)
|
||||
|
||||
当前行为:
|
||||
|
||||
- `collector_credentials` tab 展示为“采集器设置”。
|
||||
- 下拉框列出所有内置采集器。
|
||||
- 下拉框右侧只有一个插头图标按钮,用于健康检查。
|
||||
- 下拉框下方用状态标签展示:
|
||||
- `需要凭证` / `无需凭证`
|
||||
- 模块
|
||||
- `启用` / `禁用`
|
||||
- `未检查` / `可用` / `不可用`
|
||||
- 是否覆盖 endpoint
|
||||
- 需要凭证的采集器把凭证卡片放在基础配置上方。
|
||||
- 不需要凭证的采集器只显示基础配置。
|
||||
|
||||
连接按钮使用内联 Tabler 风格插头图标,来源语义对应 `plug-connected`,避免继续使用刷新图标表达连接动作。
|
||||
|
||||
## 后端接口
|
||||
|
||||
### 数据源配置列表
|
||||
|
||||
```http
|
||||
GET /api/v1/datasources/configs/all
|
||||
```
|
||||
|
||||
返回 YAML 默认数据源和数据库覆盖配置的合并结果。该路由必须定义在 `/configs/{config_id}` 之前,否则 `all` 会被 FastAPI 当成路径参数并触发 422。
|
||||
|
||||
返回字段包括:
|
||||
|
||||
- `name`
|
||||
- `default_url`
|
||||
- `endpoint`
|
||||
- `is_overridden`
|
||||
- `is_active`
|
||||
- `source_type`
|
||||
- `auth_type`
|
||||
- `headers`
|
||||
- `config`
|
||||
- `config_id`
|
||||
- `description`
|
||||
|
||||
`config` 返回前会移除内部连接验证字段,避免前端把校验元数据当成用户配置展示。
|
||||
|
||||
### 内置采集器连接状态
|
||||
|
||||
```http
|
||||
POST /api/v1/datasources/configs/builtin/connection-status
|
||||
```
|
||||
|
||||
用途:
|
||||
|
||||
- 给定一份候选配置。
|
||||
- 计算 checksum。
|
||||
- 判断当前配置是否已经连接。
|
||||
|
||||
当前前端主要通过连接按钮即时检查,不强依赖这个接口,但它是后续保存按钮置灰、页面初始化状态恢复的后端依据。
|
||||
|
||||
### 内置采集器连接验证
|
||||
|
||||
```http
|
||||
POST /api/v1/datasources/configs/builtin/connect
|
||||
```
|
||||
|
||||
用途:
|
||||
|
||||
- 免费采集器直接请求 endpoint。
|
||||
- 需要凭证的采集器走对应 credential provider。
|
||||
- 验证成功后写入系统级连接记录。
|
||||
|
||||
成功返回中会带:
|
||||
|
||||
- `success`
|
||||
- `connected`
|
||||
- `checksum`
|
||||
- `stage`
|
||||
- `message`
|
||||
- `response_time_ms`
|
||||
- `credential_provider`
|
||||
- `credential_source`
|
||||
|
||||
### BarentsWatch AIS 连接验证
|
||||
|
||||
```http
|
||||
POST /api/v1/settings/integrations/barentswatch/connect
|
||||
GET /api/v1/settings/integrations/barentswatch/connectivity
|
||||
```
|
||||
|
||||
BarentsWatch 使用独立接口,是因为它需要在保存前验证草稿凭证:
|
||||
|
||||
- 使用草稿 `client_id` / `client_secret` 获取 token。
|
||||
- 使用 token 请求 AIS endpoint。
|
||||
- 连接成功后用草稿凭证指纹写入内置采集器连接记录。
|
||||
|
||||
## 连接校验服务
|
||||
|
||||
文件:
|
||||
|
||||
- [datasource_connectivity.py](/home/ray/dev/linkong/planet/backend/app/services/datasource_connectivity.py)
|
||||
|
||||
核心职责:
|
||||
|
||||
- 计算内置采集器配置 checksum。
|
||||
- 读取环境变量和 `~/.zshrc` 中的凭证。
|
||||
- 判断当前配置是否已连接。
|
||||
- 执行 endpoint 健康检查。
|
||||
- 保存连接成功记录。
|
||||
|
||||
### checksum 组成
|
||||
|
||||
checksum 包含:
|
||||
|
||||
- 采集器名称
|
||||
- endpoint
|
||||
- auth type
|
||||
- headers
|
||||
- 去掉内部校验字段后的 config
|
||||
- credential provider
|
||||
- 凭证指纹
|
||||
|
||||
凭证指纹使用凭证内容 hash,不把明文凭证写入连接记录。
|
||||
|
||||
### 连接记录
|
||||
|
||||
连接成功记录写入 `SystemSetting`:
|
||||
|
||||
```text
|
||||
category = datasource_connectivity_validations
|
||||
```
|
||||
|
||||
payload 以采集器 source 为 key:
|
||||
|
||||
```json
|
||||
{
|
||||
"barentswatch_vessels": {
|
||||
"checksum": "...",
|
||||
"status": "success",
|
||||
"validated_at": "2026-04-29T00:00:00+00:00",
|
||||
"status_code": 200,
|
||||
"credential_source": "datasource_config",
|
||||
"connected_by": "connection_button"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`connected_by` 当前有两个来源:
|
||||
|
||||
- `connection_button`
|
||||
- 用户手动点击连接按钮。
|
||||
- `collection`
|
||||
- 采集任务成功完成,系统自动记录当前有效配置已连通。
|
||||
|
||||
### 成功采集即连接
|
||||
|
||||
调度器在采集成功后会调用连接记录写入逻辑:
|
||||
|
||||
- [scheduler.py](/home/ray/dev/linkong/planet/backend/app/services/scheduler.py)
|
||||
|
||||
这样已有数据的采集器不会要求用户重复验证。只有当配置 checksum 变化时,才需要重新点击连接。
|
||||
|
||||
## BarentsWatch AIS 凭证链路
|
||||
|
||||
文件:
|
||||
|
||||
- [barentswatch.py](/home/ray/dev/linkong/planet/backend/app/services/barentswatch.py)
|
||||
- [vessel_ais.py](/home/ray/dev/linkong/planet/backend/app/services/collectors/vessel_ais.py)
|
||||
|
||||
解析优先级:
|
||||
|
||||
1. `DataSourceConfig.auth_config`
|
||||
2. `DataSourceConfig.config`
|
||||
3. 环境变量
|
||||
4. `~/.zshrc`
|
||||
|
||||
支持的环境变量:
|
||||
|
||||
```bash
|
||||
export BARENTSWATCH_CLIENT_ID="..."
|
||||
export BARENTSWATCH_CLIENT_SECRET="..."
|
||||
```
|
||||
|
||||
也兼容历史拼写:
|
||||
|
||||
```bash
|
||||
export BARRENTSWATCH_CLIENT_ID="..."
|
||||
export BARRENTSWATCH_CLIENT_SECRET="..."
|
||||
```
|
||||
|
||||
Token 请求规则:
|
||||
|
||||
- Token URL:`https://id.barentswatch.no/connect/token`
|
||||
- `Content-Type`: `application/x-www-form-urlencoded`
|
||||
- Body:
|
||||
- `grant_type=client_credentials`
|
||||
- `client_id`
|
||||
- `client_secret`
|
||||
- `scope=ais`
|
||||
|
||||
AIS 请求规则:
|
||||
|
||||
- Endpoint 默认:`https://live.ais.barentswatch.no/v1/latest/combined`
|
||||
- Header:`Authorization: Bearer <access_token>`
|
||||
|
||||
`VesselAISCollector` 不再自己读取环境变量,而是统一走 `resolve_barentswatch_config()` 和 `fetch_barentswatch_access_token()`,避免设置页、连接验证和采集器三套凭证逻辑分叉。
|
||||
|
||||
## 凭证教程
|
||||
|
||||
文件:
|
||||
|
||||
- [credential_guides.py](/home/ray/dev/linkong/planet/backend/app/services/credential_guides.py)
|
||||
|
||||
接口:
|
||||
|
||||
```http
|
||||
GET /api/v1/settings/credential-guides/{provider}
|
||||
POST /api/v1/settings/credential-guides/{provider}/generate
|
||||
POST /api/v1/settings/credential-guides/{provider}/reset
|
||||
```
|
||||
|
||||
当前支持:
|
||||
|
||||
- `barentswatch`
|
||||
|
||||
默认教程包含 BarentsWatch 官方 tutorial 地址:
|
||||
|
||||
```text
|
||||
https://developer.barentswatch.no/docs/tutorial
|
||||
```
|
||||
|
||||
如果用户点击“教程不好用”,后端会把默认 prompt 发给 AI Provider 生成新的中文教程,并保存到 `SystemSetting`:
|
||||
|
||||
```text
|
||||
category = collector_credential_guides
|
||||
```
|
||||
|
||||
“重置”会删除自定义教程,恢复默认教程。
|
||||
|
||||
## 保存规则
|
||||
|
||||
内置采集器配置保存时会移除内部 `connectivity_validation` 字段,避免校验状态跟用户配置混在一起。
|
||||
|
||||
BarentsWatch `client_secret` 保存时有特殊处理:
|
||||
|
||||
- 输入框显示脱敏预览。
|
||||
- 如果提交值仍等于脱敏预览,后端保留原 secret。
|
||||
- 如果输入新值,才替换 secret。
|
||||
- 不再提供单独“清除当前 secret”复选框。
|
||||
|
||||
## 测试覆盖
|
||||
|
||||
相关测试:
|
||||
|
||||
- [test_vessels.py](/home/ray/dev/linkong/planet/backend/tests/test_vessels.py)
|
||||
|
||||
新增覆盖:
|
||||
|
||||
- 能从 `~/.zshrc` 解析 BarentsWatch 凭证。
|
||||
- 环境变量为空时,`resolve_barentswatch_config()` 能回退到 `~/.zshrc`。
|
||||
- 船只数据转换和 GeoJSON 输出保持兼容。
|
||||
|
||||
## 当前 Provider 覆盖
|
||||
|
||||
当前已经支持的凭证 provider:
|
||||
|
||||
- `barentswatch`
|
||||
- `spacetrack`
|
||||
|
||||
其他 `requires_credentials=true` 的采集器如果还没有 provider,会返回“凭证链路尚未接入”,前端显示 `不可用`。
|
||||
@@ -81,6 +81,13 @@ React 路由入口:
|
||||
- status message
|
||||
- tooltip / error / 清理逻辑
|
||||
|
||||
当前 status message 有两类短提示:
|
||||
|
||||
- 普通业务提示:通过 `showStatusMessage()` 入队显示。
|
||||
- 手势提示:通过 `showGestureStatusMessage()` 直接短暂显示,用于缩放视角时的 `缩放 N%`。
|
||||
|
||||
手势提示不会抢占 loading 状态。对应样式是 [hud.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/hud.css) 中的 `.earth-status-message.gesture`。
|
||||
|
||||
### 5. 地球与地形
|
||||
|
||||
- [earth.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/earth.js)
|
||||
@@ -98,6 +105,7 @@ React 路由入口:
|
||||
- [cables.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cables.js)
|
||||
- [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js)
|
||||
- [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
|
||||
- [vessels.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/vessels.js)
|
||||
- [news.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/news.js)
|
||||
- [tv.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/tv.js)
|
||||
- [layer-startup-tasks.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/layer-startup-tasks.js)
|
||||
@@ -132,6 +140,10 @@ React 路由入口:
|
||||
|
||||
当前 BGP 巡航只是这套能力的一个调用方,不应再把“按队列巡航”和“BGP 事件展示”混写在同一个状态机里。
|
||||
|
||||
新闻巡航摘要计划见:
|
||||
|
||||
- [earth-news-cruise-summary-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-cruise-summary-plan.md)
|
||||
|
||||
## 当前样式分层
|
||||
|
||||
Earth 的 CSS 不是一份大样式表,而是分层管理:
|
||||
@@ -232,6 +244,79 @@ Earth 图层按钮现在不应再只有“开/关”两态,而应支持:
|
||||
|
||||
这样后续新增会参与启动加载的图层时,顺序、模式和提示文案都在同一处定义,不需要再去 `main.js` 里补第二套常量。
|
||||
|
||||
### 船只图层与图例
|
||||
|
||||
AIS 船只图层入口:
|
||||
|
||||
- [vessels.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/vessels.js)
|
||||
|
||||
船只图层当前负责:
|
||||
|
||||
- 请求 `/api/v1/visualization/geo/vessels`
|
||||
- 将 BarentsWatch AIS GeoJSON 转为地球局部坐标 marker 数据
|
||||
- 用按航向分桶的 `THREE.Points` 批量渲染普通船只 marker
|
||||
- 按船型映射颜色
|
||||
- 根据航行/停泊状态绘制三角形或圆点纹理
|
||||
- 用单点 `THREE.Points` overlay 承载 hover / locked glow
|
||||
- 支持 hover、lock、轨迹加载和视觉聚焦
|
||||
|
||||
船只图层不再是“每艘船一个 `THREE.Sprite`”。原始 Sprite 方案在拖动地球时会把透明对象排序、draw call 和对象级 raycast 成本全部放到主交互路径上;即使 BarentsWatch 免费 AIS 当前只覆盖挪威周边,也会让地球拖动明显不跟手。
|
||||
|
||||
当前设计把普通船只拆成少量批次:
|
||||
|
||||
- moving / anchored 分开。
|
||||
- moving 船只按 `VESSEL_COURSE_BINS` 做航向分桶。
|
||||
- 每个批次是一组 `THREE.PointsMaterial`,位置和颜色写入 `BufferGeometry` attribute。
|
||||
- 普通态不带 glow;hover / locked 时才在相同点位叠加带 glow 的单点 overlay。
|
||||
|
||||
方向标准以 AIS `course / cog` 为准:从正北开始顺时针。普通态和交互态都通过同一套 canvas 旋转规则生成纹理,避免 hover 后箭头方向和原 marker 不一致。
|
||||
|
||||
船只 hover / click 也不再对渲染对象做 `raycaster.intersectObjects()`。`main.js` 会:
|
||||
|
||||
1. 拖动地球或惯性旋转时跳过 hover picking。
|
||||
2. 对 hover picking 做轻量节流。
|
||||
3. 只保留正面船只作为候选。
|
||||
4. 将候选船只投影到屏幕坐标。
|
||||
5. 用 `VESSEL_POINTER_RADIUS_PX` 做像素距离命中,并取最近船只。
|
||||
|
||||
这样 picking 位置和用户看到的屏幕 marker 对齐,也避免 `Points` 自带 raycaster 在固定屏幕尺寸图标上的命中半径错位。
|
||||
|
||||
图例系统已经注册 `vessels` 模式:
|
||||
|
||||
- [legend.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/legend.js)
|
||||
- [legend.css](/home/ray/dev/linkong/planet/frontend/public/earth/css/legend.css)
|
||||
|
||||
`getVesselLegendItems()` 返回带 `shape` 的图例项:
|
||||
|
||||
- `shape: "vessel"`:三角形,表示航行船只。
|
||||
- `shape: "dot"`:圆点,表示停泊或低速状态。
|
||||
|
||||
图例项颜色来自 `VESSEL_CONFIG.colors`,不要在 `legend.css` 里重新定义业务颜色。新增船型时,应优先改 `vessels.js` 和 `constants.js` 的船型映射,再同步图例项。
|
||||
|
||||
### 视角控制反馈
|
||||
|
||||
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) 统一维护 Earth 缩放状态。滚轮缩放、缩放按钮和触屏双指捏合最终都会更新 `zoomLevel`,并通过 `showZoomStatusCapsule()` 显示当前缩放比例:
|
||||
|
||||
```javascript
|
||||
showGestureStatusMessage(`缩放 ${Math.round(zoomLevel * 100)}%`, "info");
|
||||
```
|
||||
|
||||
该提示每 90ms 最多更新一次,显示 760ms 后淡出。它是视角反馈,不是数据加载进度,也不应该写进图层 loading 状态。
|
||||
|
||||
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 只负责在双指捏合缩放时调用 `setZoomLevel()` 和 `showZoomStatusCapsule()`。鼠标滚轮与缩放按钮的胶囊提示应继续放在 `controls.js`,避免同一种缩放反馈散落在多个模块。
|
||||
|
||||
拖拽地球的旋转灵敏度会根据当前缩放连续衰减,而不是按某个缩放阈值分段:
|
||||
|
||||
```javascript
|
||||
const scale = THREE.MathUtils.clamp(
|
||||
Math.pow(zoom, -CONFIG.dragRotationZoomExponent),
|
||||
CONFIG.dragRotationScaleMin,
|
||||
CONFIG.dragRotationScaleMax,
|
||||
);
|
||||
```
|
||||
|
||||
调参入口在 [constants.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/constants.js):`dragRotationFactorBase` 控制基础速度,`dragRotationZoomExponent` 控制放大后的衰减曲线,`dragRotationScaleMin` / `dragRotationScaleMax` 控制上下限。
|
||||
|
||||
### `data-status-target`
|
||||
|
||||
图层按钮可以通过:
|
||||
@@ -284,98 +369,23 @@ Earth 设置面板当前由 [controls.js](/home/ray/dev/linkong/planet/frontend/
|
||||
1. 图层开关 loading 状态持续可见
|
||||
2. 页面空闲时会预热 `ensureTerrainReady()`
|
||||
|
||||
也就是说,后续再继续优化 terrain 时,优先顺序应该是:
|
||||
## 当前巡航链路
|
||||
|
||||
1. 先保证用户感知正确
|
||||
2. 再压缩首次等待
|
||||
3. 最后才做更激进的几何/瓦片优化
|
||||
当前巡航边界:
|
||||
|
||||
## 当前高频风险点
|
||||
|
||||
### 1. 视觉状态和业务状态不同步
|
||||
|
||||
Earth 里最常见的 bug 不是“没渲染”,而是:
|
||||
|
||||
- 图层关了,tooltip 还在
|
||||
- 锁定对象隐藏了,info card 还在
|
||||
- legend 没跟图层切换
|
||||
- loading 已结束,但按钮还像没开
|
||||
|
||||
后续改动必须优先检查状态同步。
|
||||
|
||||
### 2. HUD 布局问题先查结构,不要先打 CSS 补丁
|
||||
|
||||
Earth HUD 历史上反复出现:
|
||||
|
||||
- 面板只剩一条缝
|
||||
- markdown 被裁掉
|
||||
- tabs/iframe 被 `overflow: hidden` 吃掉
|
||||
|
||||
优先检查:
|
||||
|
||||
1. 谁负责高度
|
||||
2. 谁负责滚动
|
||||
3. 哪一层在裁剪
|
||||
|
||||
不要上来先加 `overflow: hidden` 或额外包装层。
|
||||
|
||||
### 3. Transitional path 必须收口
|
||||
|
||||
Earth 已经经历过多轮 HUD、toolbar、media panel 重构,所以最容易积累:
|
||||
|
||||
- 旧 helper
|
||||
- 旧 class
|
||||
- 旧 fallback 逻辑
|
||||
- 已废弃变体
|
||||
|
||||
每次大功能完成后,都要做一次 cleanup pass。
|
||||
|
||||
### 4. 巡航与业务事件不要再深度耦合
|
||||
|
||||
当前正确边界应该是:
|
||||
|
||||
- 通用巡航层只知道:
|
||||
- 通用巡航层包含:
|
||||
- 当前目标
|
||||
- 队列顺序
|
||||
- 相机 focus
|
||||
- 停留 / 隐藏 / 切换
|
||||
- 业务模块只负责:
|
||||
- 业务模块提供:
|
||||
- 提供目标队列
|
||||
- 提供 focus 坐标
|
||||
- 提供卡片内容
|
||||
- 提供高亮/图层副作用
|
||||
|
||||
如果以后再给海缆、卫星或新闻做巡航,不应复制一套新的 `main.js` 状态变量,而应复用:
|
||||
巡航模块的结构文件:
|
||||
|
||||
- [cruise-sequencer.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cruise-sequencer.js)
|
||||
- [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js)
|
||||
- [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js) 这种业务适配层模式
|
||||
|
||||
## 当前推荐改动方式
|
||||
|
||||
如果后续继续改 Earth,建议按这个顺序:
|
||||
|
||||
1. 先确认改的是:
|
||||
- Three.js 渲染层
|
||||
- HUD 结构层
|
||||
- 图层状态层
|
||||
- 面板内容层
|
||||
2. 如果涉及图层按钮,优先接入统一状态机
|
||||
3. 如果涉及可见性切换,检查 tooltip / legend / info-card / lock 是否一起收口
|
||||
4. 如果涉及面板布局,先查结构再动 CSS
|
||||
|
||||
## 当前与控制台前端的边界
|
||||
|
||||
Earth 前端和控制台前端不是同一套 UI 系统:
|
||||
|
||||
- 控制台前端:React + Ant Design 工作台
|
||||
- Earth 前端:`public/earth` 原生 HUD + Three.js 展示面
|
||||
|
||||
因此:
|
||||
|
||||
- Earth 不应该直接复用 Ant Table / AppLayout 语义
|
||||
- 控制台也不应该照搬 Earth HUD 动画和玻璃层语言
|
||||
|
||||
控制台相关结构见:
|
||||
|
||||
- [admin-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-admin-frontend-context.md)
|
||||
- [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
|
||||
|
||||
@@ -95,11 +95,11 @@
|
||||
| 国界线颜色 | `COUNTRY_BOUNDARY_CONFIG.lineColor` | `0x7fc7ff` | 普通国界线 |
|
||||
| 国界线透明度 | `COUNTRY_BOUNDARY_CONFIG.lineOpacity` | `0.58` | 普通国界线 opacity |
|
||||
| 国界线 hover 时压暗透明度 | `COUNTRY_BOUNDARY_CONFIG.dimmedLineOpacity` | `0.18` | hover 时普通国界线 opacity |
|
||||
| 国界线半径偏移 | `COUNTRY_BOUNDARY_CONFIG.lineAltitudeOffset` | `0.24` | 普通国界线半径 |
|
||||
| 国界线半径偏移 | `COUNTRY_BOUNDARY_CONFIG.lineAltitudeOffset` | `0.115` | 普通国界线半径;略高于高清材质 `0.10`,低于地形基准 `0.16`,减少悬浮感 |
|
||||
| 国界线 renderOrder | `COUNTRY_BOUNDARY_CONFIG.lineRenderOrder` | `2.2` | 普通国界线层级 |
|
||||
| 国界 hover 颜色 | `COUNTRY_BOUNDARY_CONFIG.hoverLineColor` | `0xff3b1f` | 霓虹红橘 |
|
||||
| 国界 hover 透明度 | `COUNTRY_BOUNDARY_CONFIG.hoverLineOpacity` | `1.0` | hover 实线 opacity |
|
||||
| 国界 hover 半径偏移 | `COUNTRY_BOUNDARY_CONFIG.hoverAltitudeOffset` | `0.32` | hover 实线半径 |
|
||||
| 国界 hover 半径偏移 | `COUNTRY_BOUNDARY_CONFIG.hoverAltitudeOffset` | `0.14` | hover 实线半径;贴近地表但高于普通国界线 |
|
||||
| 国界 hover renderOrder | `COUNTRY_BOUNDARY_CONFIG.hoverLineRenderOrder` | `2.3` | hover 实线层级 |
|
||||
| 国界 hover glow 透明度 | `COUNTRY_BOUNDARY_CONFIG.hoverGlowOpacity` | `0.38` | glow 线 opacity |
|
||||
| 国界 hover glow 线宽 | `COUNTRY_BOUNDARY_CONFIG.hoverGlowLineWidth` | `3` | glow `LineBasicMaterial.linewidth` |
|
||||
@@ -186,6 +186,27 @@
|
||||
| footprint renderOrder | local `GROUND_FOOTPRINT_RENDER_ORDER` | `3` | footprint fill |
|
||||
| footprint group renderOrder | inline | `0` | 避免 Group 排序盖过卫星点 |
|
||||
|
||||
## AIS 船只
|
||||
|
||||
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 船只半径偏移 | `VESSEL_CONFIG.altitudeOffset` | `0.2` | 普通 marker 位置,贴近真实地形基础层 |
|
||||
| 船只轨迹半径偏移 | `VESSEL_CONFIG.track.altitudeOffset` | `0.22` | 选中船只轨迹线,略高于船只 marker 以保持可见 |
|
||||
| 船只 renderOrder | local `VESSEL_RENDER_ORDER` | `4.4` | 普通 marker 和交互 overlay |
|
||||
| 船只轨迹 renderOrder | `VESSEL_RENDER_ORDER - 0.1` | `4.3` | 低于船只 marker |
|
||||
| 船只点像素尺寸 | local `VESSEL_POINT_SIZE` | `34` | 普通 marker 与 hover / locked overlay 共享尺寸 |
|
||||
| 船只纹理画布尺寸 | local `VESSEL_ATLAS_CELL_SIZE` | `128` | canvas 点纹理 |
|
||||
| 航向分桶数 | local `VESSEL_COURSE_BINS` | `32` | moving 船只按 COG 分桶,降低 draw call 同时保留方向 |
|
||||
| 船只 hover 拾取节流 | local `VESSEL_HOVER_PICK_INTERVAL_MS` | `100` | `main.js` hover picking |
|
||||
| 船只屏幕命中半径 | local `VESSEL_POINTER_RADIUS_PX` | `22` | `main.js` 屏幕空间 picking |
|
||||
| 普通船只透明度 | `VESSEL_CONFIG.marker.baseOpacity` | `0.88` | 普通 `PointsMaterial.opacity` |
|
||||
| dimmed 船只透明度 | `VESSEL_CONFIG.marker.dimmedOpacity` | `0.26` | 锁定某艘船后其他批次透明度 |
|
||||
| hover 船只透明度 | inline | `0.98` | hover overlay |
|
||||
| locked 船只透明度 | inline | `1` | locked overlay |
|
||||
| 船型颜色 | `VESSEL_CONFIG.colors.*` | cargo / tanker / passenger / fishing / military / other | `PointsMaterial.vertexColors` 和 overlay texture |
|
||||
|
||||
AIS 船只普通态使用批量 `THREE.Points`,不是逐船 `THREE.Sprite`。航行船只保持三角形,停泊或低速船只保持圆点;普通态不带 glow,hover / locked 时在同一屏幕尺寸上叠加带 glow 的单点 overlay。AIS 航向按 `course / cog` 从正北顺时针解释,普通态和交互态必须使用同一套 canvas 旋转规则。
|
||||
|
||||
## 算力中心
|
||||
|
||||
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
|
||||
|
||||
@@ -103,7 +103,7 @@
|
||||
|
||||
## 采集器配置方式
|
||||
|
||||
`news_live_streams` 不需要单独新页面,直接复用现有数据源配置:
|
||||
`news_live_streams` 不需要单独新页面,直接复用控制台 `/settings` 的“采集器设置”:
|
||||
|
||||
- `endpoint`
|
||||
- 频道目录 JSON API 地址
|
||||
|
||||
@@ -7,8 +7,8 @@
|
||||
|
||||
| 顺序类型 | 当前顺序 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| 控制面板顺序 | 海缆 → 轨迹 → 卫星 → 算力中心 → BGP → 地形 → 高清材质 → 大气云图 → 国界 → 经纬线 | 由 `displayOrder` 控制,按操作关注度排列。 |
|
||||
| 注册 / 启动加载顺序 | 经纬线 → 国界 → 高清材质 → 大气云图 → 海缆 → 算力中心 → BGP → 卫星 | 由注册顺序和 `startupPriority` 控制,按地表到天空排列;轨迹和地形是依赖/可选显示层,不参与常规启动数据加载。 |
|
||||
| 控制面板顺序 | 海缆 → 轨迹 → 卫星 → 算力中心 → 船只 → BGP → 地形 → 高清材质 → 大气云图 → 国界 → 经纬线 | 由 `displayOrder` 控制,按操作关注度排列。 |
|
||||
| 注册 / 启动加载顺序 | 经纬线 → 国界 → 高清材质 → 大气云图 → 海缆 → 算力中心 → 船只 → BGP → 卫星 | 由注册顺序和 `startupPriority` 控制,按地表到天空排列;船只和卫星默认关闭,只有可见时参与启动加载;轨迹和地形是依赖/可选显示层,不参与常规启动数据加载。 |
|
||||
|
||||
## 地表图层栈
|
||||
|
||||
@@ -24,11 +24,13 @@
|
||||
| 1 | 海缆 | `cables.js` | `CABLE_CONFIG.line.renderOrder` | 海缆拾取路径 | 保持现有海缆层级。 |
|
||||
| 1.2 | 真实地形 | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` 加地形位移 | 禁用 raycast | 地形压过高清材质;高清材质关闭时临时隐藏,重新开启后恢复原状态。 |
|
||||
| 2.05 | 经纬线 | `earth.js` | `CONFIG.earthRadius + 0.14` | 禁用 raycast | 低透明度显示在高清材质上。 |
|
||||
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset` | 禁用 raycast | 只保证压过高清材质。 |
|
||||
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset = 0.115` | `depthTest: true`,禁用 raycast | 略高于高清材质 `0.10`,低于地形基准 `0.16`,减少悬浮感;地形 `depthWrite: false`,所以地形开启时仍可见。 |
|
||||
| 2.29 | 国界 hover 光晕 | `country-boundaries.js` | hover 半径加 glow 偏移 | `depthTest: false`,禁用 raycast | 用 additive 光晕增强交界边和地形开启时的 hover 可见性。 |
|
||||
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;中国和中国(台湾)共享高亮组。 |
|
||||
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset = 0.14` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;中国和中国(台湾)共享高亮组。 |
|
||||
| 3 | 卫星 footprint 填充 | `satellites.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-tested,Group renderOrder 保持 0 | Footprint 在国界线之上,但在算力中心和卫星之下。 |
|
||||
| 3-5 | BGP 标记和覆盖层 | `bgp.js` | 各 marker 自身 renderOrder | BGP 拾取路径 | 保持现有 BGP 视觉层级。 |
|
||||
| 4.3 | AIS 船只轨迹线 | `vessels.js` | `VESSEL_RENDER_ORDER - 0.1`;`CONFIG.earthRadius + VESSEL_CONFIG.track.altitudeOffset` | 跟随船只显隐,不单独参与拾取 | 选中船只后显示最近轨迹,低于船只 marker。 |
|
||||
| 4.4 | AIS 船只 marker | `vessels.js` | `VESSEL_RENDER_ORDER`;`CONFIG.earthRadius + VESSEL_CONFIG.altitudeOffset`;普通 marker 为分桶 `THREE.Points`,hover / locked 为单点 `THREE.Points` overlay | `depthTest: true`;`main.js` 使用屏幕空间 picking,只取正面 marker | 航行船只用三角点纹理,停泊/低速用圆点;普通态无 glow,交互态叠加同尺寸 glow;低于算力中心 `4.5`。 |
|
||||
| 4.5 | 算力中心 | `compute-centers.js` | `COMPUTE_CENTER_RENDER_ORDER` | 算力中心拾取路径 | 地表设施,保持在卫星下方。 |
|
||||
| 5 | 卫星背景点 | `satellites.js` | 固定 renderOrder | 屏幕空间卫星拾取 | 位于卫星点下方。 |
|
||||
| 6 | 卫星点 | `satellites.js` | 固定 renderOrder | 屏幕空间卫星拾取 | 卫星点压过 footprint 和算力中心。 |
|
||||
@@ -55,3 +57,4 @@
|
||||
| 中国 / 台湾 hover | `CHN` 和 `TWN` 被归到同一个 hover 高亮组;tooltip 仍显示鼠标实际命中的 feature。 |
|
||||
| 地形 | 只作为视觉层参与,`terrain.raycast` 已禁用。 |
|
||||
| 卫星 | 使用屏幕空间卫星拾取,避免 footprint 或地表层挡住卫星点击。 |
|
||||
| 船只 | 不使用对象级 sprite raycast。`main.js` 会在拖动 / 惯性期间跳过 hover picking,平时将正面船只投影到屏幕坐标,用像素半径命中最近船只;点击后可加载轨迹线。 |
|
||||
|
||||
@@ -188,11 +188,3 @@
|
||||
- 非 Starlink 的能力判断属于“策略层 / 适配层”
|
||||
- 不要把不同星座的覆盖模型再混写进同一套参数里
|
||||
- `iridium-next` 已经切成独立 adapter,应继续沿这条边界演进,而不是给现有 Starlink bowtie 增加更多 if/else
|
||||
|
||||
## 后续建议
|
||||
|
||||
如果继续往前做,推荐顺序是:
|
||||
|
||||
1. 为 `iridium-next` 新建独立 footprint adapter
|
||||
2. 在 UI 上补一个只读提示,让用户知道当前卫星是否支持 footprint
|
||||
3. 如果未来拿到 GEO beam contour / operator metadata,再为 GEO 开 operator-specific footprint
|
||||
|
||||
@@ -208,7 +208,7 @@
|
||||
- 先通过 port/types 定义边界
|
||||
- 再由 http/mock gateway 实现
|
||||
|
||||
## 当前页面分层建议
|
||||
## 当前页面分层
|
||||
|
||||
### 1. 仪表盘和摘要型页面
|
||||
|
||||
@@ -237,6 +237,63 @@
|
||||
- 不要让表格撑爆整页
|
||||
- 新表格区域优先复用 `TableScrollRegion` / `ScrollbarOverlay`
|
||||
|
||||
### 数据源目录页
|
||||
|
||||
[DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx) 当前不再承担配置编辑职责,而是数据源目录和采集操作页。
|
||||
|
||||
当前页面边界:
|
||||
|
||||
- 内置数据源和自定义数据源合并为 `UnifiedDataSource`。
|
||||
- 列表展示类型、状态、最近运行、采集进度和操作。
|
||||
- 点击名称打开只读抽屉。
|
||||
- 抽屉中明确显示“内置数据源”或“自定义数据源”。
|
||||
- endpoint、headers、config 只展示,不在这里编辑。
|
||||
- 需要凭证的采集器提示用户到“设置 -> 采集器设置”维护。
|
||||
|
||||
这个边界很重要:后续不要把自定义数据源编辑、内置 endpoint 覆盖或凭证表单再塞回 `/datasources`。这些配置入口统一放在 `/settings?tab=collector_credentials`。
|
||||
|
||||
页面顶部的总进度区域新增 `采集中 N` 标签:
|
||||
|
||||
- 仅在存在运行中采集任务时显示。
|
||||
- 样式定义在 [index.css](/home/ray/dev/linkong/planet/frontend/src/index.css) 的 `data-source-bulk-toolbar__running-pill`。
|
||||
- 点击后打开 `采集中任务` Modal。
|
||||
- Modal 内展示每个运行任务的阶段、进度、已处理/总数。
|
||||
|
||||
这个标签和其他状态标签同排,但通过 hover、蓝色描边和箭头表示可交互,不应改成普通 Tag。
|
||||
|
||||
### 采集器设置页
|
||||
|
||||
[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx) 中的 `collector_credentials` tab 当前显示为“采集器设置”。
|
||||
|
||||
当前页面边界:
|
||||
|
||||
- 下拉框选择所有内置采集器。
|
||||
- 下拉框右侧只有一个插头图标按钮,用于健康检查。
|
||||
- 状态标签在选择器下方展示 `未检查` / `可用` / `不可用`。
|
||||
- 需要凭证的采集器将凭证卡片放在基础配置上方。
|
||||
- 不需要凭证的采集器只显示基础配置:endpoint、默认 endpoint、请求头、timeout、retry。
|
||||
- `BarentsWatch AIS` 使用专用凭证表单。
|
||||
|
||||
连接图标使用内联 `PlugConnectIcon`,视觉语义来自 Tabler `plug-connected`。后续如果控制台重写图标体系,应迁移到 Tabler Icons,而不是继续使用 Ant Design 刷新图标表达连接。
|
||||
|
||||
`Client Secret` 的表单语义:
|
||||
|
||||
- 已配置时,输入框显示脱敏 preview。
|
||||
- 聚焦且当前值等于 preview 时清空,方便输入新 secret。
|
||||
- 保存时如果值仍等于 preview,提交空值表示保留原 secret。
|
||||
- 不再提供单独的“清除当前 secret”复选框。
|
||||
|
||||
凭证教程 Modal:
|
||||
|
||||
- `GET /api/v1/settings/credential-guides/{provider}` 读取教程。
|
||||
- `POST /generate` 调用 AI Provider 重新生成教程。
|
||||
- `POST /reset` 恢复默认教程。
|
||||
- Modal 使用 `MarkdownRenderer` 渲染教程正文。
|
||||
|
||||
相关后端设计见:
|
||||
|
||||
- [datasource-collector-settings-connectivity.md](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)
|
||||
|
||||
### 3. 复杂工作区页面
|
||||
|
||||
例如:
|
||||
@@ -264,30 +321,3 @@
|
||||
详细经验见:
|
||||
|
||||
- [frontend-layout-guidelines.md](/home/ray/dev/linkong/planet/docs/technical/zh/frontend-layout-guidelines.md)
|
||||
|
||||
## 当前推荐改动方式
|
||||
|
||||
如果后续继续改后台页面,建议按这个顺序:
|
||||
|
||||
1. 先确认页面属于摘要页、表格页还是复杂工作区
|
||||
2. 先接入现有壳层和滚动语义
|
||||
3. 优先复用共享滚动组件
|
||||
4. 最后再改视觉和细节交互
|
||||
|
||||
不要先写局部 CSS 补丁,再回头补结构。
|
||||
|
||||
## 当前明显边界
|
||||
|
||||
控制台前端和 Earth 前端不是一套系统:
|
||||
|
||||
- 控制台前端是 React + Ant Design 工作台
|
||||
- Earth 前端是 `public/earth` 下的独立原生 HUD 系统
|
||||
|
||||
因此:
|
||||
|
||||
- 不要把 Earth 的 HUD/动画/状态机直接挪进控制台
|
||||
- 不要把控制台表格/滚动策略硬套到 Earth HUD
|
||||
|
||||
Earth 相关结构见:
|
||||
|
||||
- [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-frontend-context.md)
|
||||
|
||||
@@ -55,6 +55,52 @@
|
||||
| `--allow-lan` | 允许局域网访问 |
|
||||
| `--verbose` | 在执行过程中显示更多命令输出 |
|
||||
|
||||
### AI Provider 环境变量和构建
|
||||
|
||||
AI Provider 的运行期配置可以放在两处:
|
||||
|
||||
| 位置 | 适合内容 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `aiprovider/.env` | 团队约定的本地默认配置 | Docker Compose 会作为 `env_file` 读取 |
|
||||
| `~/.zshrc` | 个人机器上的 provider、模型、密钥和代理变量 | `planet.sh` 启动时会读取常见的 `AI_*`、`SERVICE_*`、`PYTHON_IMAGE`、`UV_IMAGE`、代理变量 |
|
||||
|
||||
推荐写法:
|
||||
|
||||
```bash
|
||||
export AI_PROVIDER=minimax
|
||||
export AI_PROVIDER_API=anthropic-messages
|
||||
export AI_BASE_URL=https://api.example.com/anthropic
|
||||
export AI_API_KEY=sk-change-me
|
||||
export AI_MODEL=MiniMax-M2.7
|
||||
export AI_PROVIDER_SERVICE_TOKEN=change_me
|
||||
```
|
||||
|
||||
默认情况下,`planet.sh` 只静态解析 `~/.zshrc` 中简单的 `export KEY=value` 或 `KEY=value` 行,避免 shell 主题、插件或交互初始化拖慢启动。如果变量依赖复杂 shell 展开,可以显式启用 source 模式:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=source ./planet.sh start -a
|
||||
```
|
||||
|
||||
如需完全忽略 `~/.zshrc`:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=0 ./planet.sh start -a
|
||||
```
|
||||
|
||||
AI Provider 镜像只在代码、Dockerfile、Compose 配置或相关 Python 依赖变化时重建。修改 `aiprovider/.env` 或 `~/.zshrc` 中的模型、密钥、Base URL 不会触发镜像重建;重启 AI Provider 即可让容器读取新配置:
|
||||
|
||||
```bash
|
||||
./planet.sh restart -a
|
||||
```
|
||||
|
||||
构建较慢时,优先判断当前卡在哪一层:
|
||||
|
||||
| 现象 | 常见原因 | 处理方式 |
|
||||
| --- | --- | --- |
|
||||
| `transferring context` 很大 | Docker build context 包含前端资源、PDF、数据目录等无关文件 | 当前仓库通过 `.dockerignore` 只发送 AI Provider 必需文件 |
|
||||
| `uv sync` 下载依赖较慢 | 首次构建或缓存为空,网络访问 Python 包较慢 | 等待首次构建完成;后续会复用 BuildKit 的 uv 下载缓存 |
|
||||
| 改密钥后仍显示旧配置 | 容器尚未重启 | 执行 `./planet.sh restart -a` |
|
||||
|
||||
### 停止
|
||||
|
||||
```bash
|
||||
@@ -196,6 +242,7 @@ Earth 用于在一个地球视图中观察:
|
||||
- 算力中心
|
||||
- BGP 观测
|
||||
- 卫星
|
||||
- AIS 船只
|
||||
- 轨迹
|
||||
- 地形
|
||||
|
||||
@@ -205,6 +252,31 @@ Earth 用于在一个地球视图中观察:
|
||||
- 轨迹依赖卫星
|
||||
- 高清材质关闭时,地球会显示基座地图和边缘识别效果
|
||||
|
||||
### 图例
|
||||
|
||||
左下角图例会跟随当前聚焦或启用的图层切换。
|
||||
|
||||
当前已覆盖:
|
||||
|
||||
- 海缆
|
||||
- 卫星
|
||||
- 国界
|
||||
- 算力中心
|
||||
- BGP
|
||||
- AIS 船只
|
||||
|
||||
AIS 船只图例按船型显示颜色:
|
||||
|
||||
- 货轮
|
||||
- 油轮
|
||||
- 客船
|
||||
- 渔船
|
||||
- 军舰
|
||||
- 停泊/低速
|
||||
- 其他船只
|
||||
|
||||
船只图例中的三角形对应地图上的航行船只标记,圆点对应停泊或低速状态。
|
||||
|
||||
### 搜索
|
||||
|
||||
Earth 搜索支持查找当前地球对象,例如:
|
||||
@@ -233,6 +305,25 @@ Earth 搜索支持查找当前地球对象,例如:
|
||||
|
||||
这些设置会保存在浏览器本地存储中。换浏览器或清理站点数据后会恢复默认值。
|
||||
|
||||
### 视角控制
|
||||
|
||||
Earth 支持鼠标、触控板和触屏操作。
|
||||
|
||||
常用控制方式:
|
||||
|
||||
| 操作 | 作用 |
|
||||
| --- | --- |
|
||||
| 鼠标左键拖动 | 旋转地球 |
|
||||
| 手指单指拖动 | 在触屏设备上旋转地球 |
|
||||
| 鼠标滚轮 | 放大或缩小视角 |
|
||||
| 双指捏合 | 在触屏设备上放大或缩小视角 |
|
||||
| 缩放按钮 | 按固定步长调整缩放 |
|
||||
| 点击缩放百分比 | 重置到默认缩放 |
|
||||
|
||||
缩放时,顶部胶囊会短暂显示当前缩放比例,例如 `缩放 180%`。这个提示只表示当前视角缩放,不代表数据加载进度;如果页面正在加载数据,加载提示优先显示,缩放提示不会打断加载状态。
|
||||
|
||||
拖动灵敏度会根据当前缩放自动调整。默认视角附近保持常规旋转速度;放大后拖动会逐步变细,适合检查某个区域、船只、卫星或 BGP 事件;缩小后拖动会略快,方便快速浏览全球态势。
|
||||
|
||||
### 巡航模式
|
||||
|
||||
巡航模式会让 Earth 自动轮播聚焦目标。
|
||||
@@ -308,7 +399,7 @@ http://localhost:3000/admin
|
||||
| --- | --- | --- |
|
||||
| 仪表盘 | `/admin` | 系统概览 |
|
||||
| Earth | `/earth` | 打开公开 Earth 页面 |
|
||||
| 数据源 | `/datasources` | 管理数据源和触发采集 |
|
||||
| 数据源 | `/datasources` | 查看数据源和触发采集 |
|
||||
| 采集数据 | `/data` | 查看采集后的数据 |
|
||||
| BGP 观测 | `/bgp` | 查看 BGP 专题数据 |
|
||||
| 系统告警 | `/alerts/system` | 系统级告警 |
|
||||
@@ -321,17 +412,21 @@ http://localhost:3000/admin
|
||||
|
||||
### 数据源
|
||||
|
||||
`/datasources` 用于查看和管理采集来源。
|
||||
`/datasources` 用于查看采集来源和触发采集。当前页面是“数据源目录”,会把内置数据源和自定义数据源放在同一张列表里展示。
|
||||
|
||||
常见操作:
|
||||
|
||||
- 查看数据源状态
|
||||
- 触发采集
|
||||
- 查看最近采集任务
|
||||
- 调整配置项
|
||||
- 打开详情抽屉查看 endpoint、请求头、基础配置和是否为内置数据源
|
||||
|
||||
如果 Earth 上某类对象缺失,通常先到这里确认数据源是否可用。
|
||||
|
||||
数据源列表中的名称点击后只打开信息抽屉,不再承担编辑入口。接口地址、凭证、请求头和自定义数据源配置统一到 `/settings` 的“采集器设置”里维护。
|
||||
|
||||
当有采集任务正在运行时,总体进度下方会出现 `采集中 N` 标签。这个标签和其他状态标签放在同一排,但带有可点击样式;点击后会弹出当前采集中任务列表,显示每个任务的阶段、进度和处理数量。
|
||||
|
||||
### 采集数据
|
||||
|
||||
`/data` 用于查看采集后的数据表。
|
||||
@@ -369,10 +464,63 @@ http://localhost:3000/admin
|
||||
|
||||
- 系统设置
|
||||
- 电视直播源配置
|
||||
- 数据源相关配置入口
|
||||
- 采集器设置
|
||||
- 外部集成和 AI Provider 配置
|
||||
|
||||
具体可用配置取决于当前登录用户权限。
|
||||
|
||||
#### 采集器设置
|
||||
|
||||
`/settings?tab=collector_credentials` 当前显示为“采集器设置”。这里统一维护所有采集器的连接配置,而不是只维护凭证。
|
||||
|
||||
使用方式:
|
||||
|
||||
1. 在下拉框选择采集器。
|
||||
2. 查看状态标签:
|
||||
- `无需凭证` / `需要凭证`
|
||||
- 所属模块
|
||||
- `启用` / `禁用`
|
||||
- `未检查` / `可用` / `不可用`
|
||||
3. 点击下拉框右侧的插头图标执行健康检查。
|
||||
4. 如果检查通过,状态会变为 `可用`。
|
||||
5. 修改 endpoint、请求头、超时或重试次数后保存。
|
||||
|
||||
对于免费且不需要凭证的采集器,连接检查会直接请求对应 endpoint。对于需要凭证的采集器,连接检查会走对应凭证链路;如果凭证或 endpoint 相比上次验证成功时发生变化,需要重新点击连接。
|
||||
|
||||
系统判断“已连接”的条件是:
|
||||
|
||||
- 当前配置已经成功采集过数据;或
|
||||
- 当前配置已经点击过连接按钮并验证成功。
|
||||
|
||||
#### BarentsWatch AIS 凭证
|
||||
|
||||
`BarentsWatch AIS` 是需要凭证的内置采集器。选择该采集器后,凭证区域会显示在基础配置上方。
|
||||
|
||||
配置项:
|
||||
|
||||
- `Client ID`
|
||||
- `Client Secret`
|
||||
- `Endpoint`
|
||||
|
||||
如果已经配置过 secret,输入框会显示脱敏预览。保存时如果保持这个脱敏预览不变,系统会保留原 secret;只有输入新的 secret 才会替换。
|
||||
|
||||
BarentsWatch AIS 支持从以下位置读取凭证:
|
||||
|
||||
1. 控制台采集器设置中保存的凭证。
|
||||
2. 后端环境变量:
|
||||
- `BARENTSWATCH_CLIENT_ID`
|
||||
- `BARENTSWATCH_CLIENT_SECRET`
|
||||
- 兼容历史拼写:`BARRENTSWATCH_CLIENT_ID`、`BARRENTSWATCH_CLIENT_SECRET`
|
||||
3. `~/.zshrc` 中的同名 `export`。
|
||||
|
||||
如果连接失败,页面会弹出凭证获取教程。教程支持:
|
||||
|
||||
- 查看默认教程。
|
||||
- 点击“教程不好用”让 AI Provider 根据默认 prompt 重新生成教程。
|
||||
- 点击“重置”恢复默认教程。
|
||||
|
||||
默认教程以 BarentsWatch 官方 tutorial 为准,并提醒 Live AIS 应选择 `AIS - API`,不是普通 `BarentsWatch - API`。
|
||||
|
||||
### 系统日志
|
||||
|
||||
`/logs` 用于查看系统日志。若菜单中不可见,通常是当前用户角色没有权限。
|
||||
@@ -397,7 +545,8 @@ http://localhost:3000/docs
|
||||
当前公开内容来自:
|
||||
|
||||
```text
|
||||
docs/technical/*.md
|
||||
docs/technical/zh/*.md
|
||||
docs/technical/en/*.md
|
||||
```
|
||||
|
||||
Docs 支持:
|
||||
@@ -486,3 +635,4 @@ source ~/.zshrc && bun run build
|
||||
- [earth-layer-style-reference.md](/home/ray/dev/linkong/planet/docs/technical/zh/earth-layer-style-reference.md)
|
||||
- [backend-system-service-control.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-system-service-control.md)
|
||||
- [backend-collectors.md](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)
|
||||
- [datasource-collector-settings-connectivity.md](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)
|
||||
|
||||
@@ -46,6 +46,8 @@ compute_ai_provider_build_fingerprint() {
|
||||
find aiprovider \
|
||||
-type f \
|
||||
! -path '*/__pycache__/*' \
|
||||
! -name '.env' \
|
||||
! -name '.env.*' \
|
||||
! -name '*.pyc' \
|
||||
! -name '*.pyo' \
|
||||
| LC_ALL=C sort \
|
||||
@@ -57,6 +59,58 @@ compute_ai_provider_build_fingerprint() {
|
||||
|
||||
速度提升约 10 倍(大量小文件场景),误报率相同(mtime+size 变化 ≡ 文件被修改)。
|
||||
|
||||
`.env` 和 `.env.*` 被排除在 fingerprint 外。它们属于运行期配置,不应该因为修改模型、密钥或 Base URL 触发镜像重建。
|
||||
|
||||
### Docker build context 收敛
|
||||
|
||||
AI Provider 镜像只需要根目录的 `pyproject.toml`、`uv.lock` 和 `aiprovider/` 代码。仓库中还包含前端静态大图、PDF、历史数据和 Unreal 资料,如果 build context 使用整个仓库,`transferring context` 会浪费大量时间。
|
||||
|
||||
当前通过根目录 `.dockerignore` 收敛上下文:
|
||||
|
||||
```dockerignore
|
||||
**
|
||||
|
||||
!pyproject.toml
|
||||
!uv.lock
|
||||
!aiprovider/
|
||||
!aiprovider/**
|
||||
|
||||
aiprovider/.env
|
||||
aiprovider/.env.*
|
||||
!aiprovider/.env.example
|
||||
```
|
||||
|
||||
Dockerfile 也从全仓复制改为只复制 AI Provider 代码:
|
||||
|
||||
```dockerfile
|
||||
COPY pyproject.toml uv.lock /app/
|
||||
RUN --mount=type=cache,target=/root/.cache/uv \
|
||||
uv sync --frozen --no-dev
|
||||
|
||||
COPY aiprovider /app/aiprovider
|
||||
```
|
||||
|
||||
`uv sync` 使用 BuildKit cache mount 后,首次构建仍可能受网络影响;后续构建会复用 `/root/.cache/uv`,依赖下载不再重复从零开始。
|
||||
|
||||
### 运行期配置来源
|
||||
|
||||
`planet.sh` 启动 AI Provider 前会生成临时 env-file,并把它传给 Compose 或手动 `docker run` fallback。配置优先来自:
|
||||
|
||||
1. `aiprovider/.env`
|
||||
2. `~/.zshrc` 中简单的 `export AI_...=...` 或 `AI_...=...` 行
|
||||
|
||||
默认解析是静态的,只覆盖 AI Provider、镜像、代理相关变量,避免执行交互 shell 初始化。如果确实需要复杂 shell 展开,可以显式启用:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=source ./planet.sh start -a
|
||||
```
|
||||
|
||||
如果排查时需要忽略个人 shell 配置:
|
||||
|
||||
```bash
|
||||
PLANET_LOAD_ZSHRC_ENV=0 ./planet.sh start -a
|
||||
```
|
||||
|
||||
### 跳过重建的原理
|
||||
|
||||
fingerprint 一致时不执行 `docker compose build`,而是:
|
||||
@@ -151,4 +205,7 @@ PY
|
||||
## 相关文件
|
||||
|
||||
- `planet.sh` — 全量修改
|
||||
- `.dockerignore` — 收敛 AI Provider Docker build context
|
||||
- `aiprovider/Dockerfile` — 只复制 AI Provider 代码,并为 `uv sync` 启用 BuildKit cache mount
|
||||
- `docker-compose.yml` / `docker-compose.simple.yml` — 读取 `planet.sh` 生成的运行期 env-file
|
||||
- `scripts/compute_aiprovider_dependency_fingerprint.py` — 依赖 fingerprint(未改动)
|
||||
|
||||
@@ -24,6 +24,12 @@
|
||||
- `aiprovider/.env`
|
||||
- `frontend/.env.local`
|
||||
|
||||
AI Provider 的个人配置也可以放在 `~/.zshrc`。`planet.sh` 会读取简单的 `export AI_...=...` 或 `AI_...=...` 行,并在启动 AI Provider 时传给容器。修改模型、密钥或 Base URL 后,通常只需要重启 AI Provider:
|
||||
|
||||
```bash
|
||||
./planet.sh restart -a
|
||||
```
|
||||
|
||||
## 1. 启动服务
|
||||
|
||||
在仓库根目录执行:
|
||||
@@ -73,6 +79,7 @@ Earth 是公开页面,不需要登录。
|
||||
- 地球正常显示
|
||||
- 右侧图层控制可打开/关闭图层
|
||||
- 搜索可以查找海缆、卫星、算力中心、BGP 事件
|
||||
- 鼠标拖动、滚轮缩放和缩放百分比提示正常工作
|
||||
- 设置面板可以切换巡航模式、日夜模式、卫星显示风格
|
||||
|
||||
## 4. 打开控制台
|
||||
@@ -87,7 +94,7 @@ http://localhost:3000/admin
|
||||
|
||||
首次排查建议查看:
|
||||
|
||||
- `/datasources`:数据源配置和采集状态
|
||||
- `/datasources`:数据源目录和采集触发;接口、请求头和凭证配置在 `/settings` 的“采集器设置”
|
||||
- `/data`:已采集数据
|
||||
- `/bgp`:BGP 专题观测
|
||||
- `/alerts/system`:系统告警
|
||||
|
||||
@@ -16,12 +16,17 @@
|
||||
## Current Version
|
||||
|
||||
- `main` 当前主线历史推导到:`0.16.5`
|
||||
- `dev` 当前开发分支历史推导到:`0.43.0`
|
||||
- `dev` 当前开发分支历史推导到:`0.45.0`
|
||||
|
||||
## Timeline
|
||||
|
||||
| Version | Type | Branch | Commit | Summary |
|
||||
| --- | --- | --- | --- | --- |
|
||||
| `0.45.0` | feature | `dev` | `pending` | 新增采集任务 fetching 阶段量化进度,收敛 AI Provider 运行期环境注入和 Docker build context |
|
||||
| `0.44.2` | bugfix | `dev` | `pending` | 补充 Earth 船只批量渲染、屏幕拾取、图层顺序、样式参考和性能计划状态文档 |
|
||||
| `0.44.1` | bugfix | `dev` | `pending` | 优化 Earth 船只批量渲染性能,修复拖动卡顿、拾取错位、交互态方向/尺寸和地表压盖问题 |
|
||||
| `0.44.0` | feature | `dev` | `pending` | 重构数据源目录与采集器设置,新增 BarentsWatch AIS 连接教程、Earth 船只/缩放体验优化和仪表盘前端重启 |
|
||||
| `0.43.1` | bugfix | `dev` | `pending` | 修正全量 restart 后 AI Provider 启动提示语义,避免把预期未就绪描述成异常 |
|
||||
| `0.43.0` | feature | `dev` | `pending` | 新增 Earth 船舶追踪、自定义数据源映射、外部集成配置中心、Markdown 渲染器增强,并整理规则/技能文档加载约束 |
|
||||
| `0.42.2` | bugfix | `dev` | `pending` | Docs 中文模式补齐分组和文档标题翻译,并更新文档站品牌文案 |
|
||||
| `0.42.1` | bugfix | `dev` | `pending` | 修正 release skill 的 feature 版本计算规则,minor 进位时重置 patch 为 0 |
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "planet-frontend",
|
||||
"version": "0.43.0",
|
||||
"version": "0.45.0",
|
||||
"private": true,
|
||||
"packageManager": "bun@1",
|
||||
"dependencies": {
|
||||
|
||||
@@ -1668,6 +1668,12 @@ label.is-disabled.earth-mobile-settings-card {
|
||||
opacity: 1;
|
||||
}
|
||||
|
||||
.earth-status-message.gesture {
|
||||
min-width: 0;
|
||||
padding-right: calc(16px * var(--hud-scale));
|
||||
color: #dcecff;
|
||||
}
|
||||
|
||||
.earth-error-message {
|
||||
top: calc(62px * var(--hud-scale));
|
||||
z-index: 211;
|
||||
|
||||
@@ -121,6 +121,19 @@
|
||||
box-shadow: 0 0 4px currentColor;
|
||||
}
|
||||
|
||||
.legend-dot--vessel {
|
||||
width: 0;
|
||||
height: 0;
|
||||
border-radius: 0;
|
||||
background: transparent !important;
|
||||
border-left: calc(5px * var(--hud-scale)) solid transparent;
|
||||
border-right: calc(5px * var(--hud-scale)) solid transparent;
|
||||
border-bottom: calc(13px * var(--hud-scale)) solid currentColor;
|
||||
color: inherit;
|
||||
box-shadow: none;
|
||||
transform: rotate(45deg);
|
||||
}
|
||||
|
||||
.legend-label {
|
||||
color: var(--hud-text);
|
||||
font-size: calc(0.78rem * var(--hud-scale));
|
||||
|
||||
@@ -9,8 +9,9 @@ export const CONFIG = {
|
||||
earthRadius: 100,
|
||||
rotationSpeed: 0.0005,
|
||||
dragRotationFactorBase: 0.005,
|
||||
dragRotationScaleMin: 0.28,
|
||||
dragRotationScaleMax: 2.0,
|
||||
dragRotationZoomExponent: 1.8,
|
||||
dragRotationScaleMin: 0.12,
|
||||
dragRotationScaleMax: 1.65,
|
||||
};
|
||||
|
||||
export const ROTATION_MODE = {
|
||||
@@ -170,8 +171,8 @@ export const TERRAIN_CONFIG = {
|
||||
|
||||
export const COUNTRY_BOUNDARY_CONFIG = {
|
||||
dataPath: "/earth/data/countries-admin0.min.geojson",
|
||||
lineAltitudeOffset: 0.24,
|
||||
hoverAltitudeOffset: 0.32,
|
||||
lineAltitudeOffset: 0.115,
|
||||
hoverAltitudeOffset: 0.14,
|
||||
lineColor: 0x7fc7ff,
|
||||
lineOpacity: 0.58,
|
||||
lineRenderOrder: 2.2,
|
||||
@@ -208,7 +209,7 @@ export const PATHS = {
|
||||
};
|
||||
|
||||
export const VESSEL_CONFIG = {
|
||||
altitudeOffset: 0.56,
|
||||
altitudeOffset: 0.2,
|
||||
maxRenderedMarkers: 5000,
|
||||
marker: {
|
||||
baseScale: 7.5,
|
||||
@@ -231,7 +232,7 @@ export const VESSEL_CONFIG = {
|
||||
max: 2.4,
|
||||
},
|
||||
track: {
|
||||
altitudeOffset: 0.7,
|
||||
altitudeOffset: 0.22,
|
||||
color: 0x7dd3fc,
|
||||
opacity: 0.82,
|
||||
},
|
||||
|
||||
21
frontend/public/earth/js/controls.js
vendored
21
frontend/public/earth/js/controls.js
vendored
@@ -10,7 +10,12 @@ import {
|
||||
ROTATION_MODE,
|
||||
SATELLITE_DISPLAY_STYLES,
|
||||
} from "./constants.js";
|
||||
import { setEarthStatValue, updateZoomDisplay, showStatusMessage } from "./ui.js";
|
||||
import {
|
||||
setEarthStatValue,
|
||||
showGestureStatusMessage,
|
||||
showStatusMessage,
|
||||
updateZoomDisplay,
|
||||
} from "./ui.js";
|
||||
import {
|
||||
toggleTerrain,
|
||||
setDayNightEnabled,
|
||||
@@ -121,6 +126,7 @@ const GRID_LINES_DEFAULT_VERSION = 3;
|
||||
const SATELLITE_DISPLAY_DEFAULT_VERSION = 4;
|
||||
const MEDIA_PANEL_DEFAULT_VERSION = 5;
|
||||
const DEFAULT_EARTH_ZOOM_STEP = 0.01;
|
||||
const ZOOM_STATUS_UPDATE_INTERVAL_MS = 90;
|
||||
let settingsModalTimer = null;
|
||||
let settingsSheetAnimation = null;
|
||||
let terrainToggleToken = 0;
|
||||
@@ -128,6 +134,7 @@ let terrainPrefetchStarted = false;
|
||||
let terrainPrefetchScheduled = false;
|
||||
let focusViewAnimationToken = 0;
|
||||
let earthSettingsDefaults = null;
|
||||
let lastZoomStatusUpdateTime = 0;
|
||||
let earthSettingsState = null;
|
||||
let layerRegistry = new Map();
|
||||
let layerPanelInitialized = false;
|
||||
@@ -1806,6 +1813,15 @@ export function setZoomLevel(nextZoom, camera = activeCamera) {
|
||||
return zoomLevel;
|
||||
}
|
||||
|
||||
export function showZoomStatusCapsule({ force = false } = {}) {
|
||||
const now = Date.now();
|
||||
if (!force && now - lastZoomStatusUpdateTime < ZOOM_STATUS_UPDATE_INTERVAL_MS) {
|
||||
return;
|
||||
}
|
||||
lastZoomStatusUpdateTime = now;
|
||||
showGestureStatusMessage(`缩放 ${Math.round(zoomLevel * 100)}%`, "info");
|
||||
}
|
||||
|
||||
function cancelSettingsSheetAnimation() {
|
||||
if (settingsSheetAnimation) {
|
||||
settingsSheetAnimation.cancel();
|
||||
@@ -2627,6 +2643,7 @@ function setupZoomControls(camera) {
|
||||
|
||||
zoomLevel = newPercent / 100;
|
||||
applyZoom(camera);
|
||||
showZoomStatusCapsule({ force: true });
|
||||
}
|
||||
|
||||
function doContinuousZoom(direction) {
|
||||
@@ -2638,6 +2655,7 @@ function setupZoomControls(camera) {
|
||||
|
||||
zoomLevel = newPercent / 100;
|
||||
applyZoom(camera);
|
||||
showZoomStatusCapsule();
|
||||
}
|
||||
|
||||
function startContinuousZoom(direction) {
|
||||
@@ -2736,6 +2754,7 @@ function setupWheelZoom(camera, renderer) {
|
||||
zoomLevel = Math.max(zoomLevel - 0.1, CONFIG.minZoom);
|
||||
}
|
||||
applyZoom(camera);
|
||||
showZoomStatusCapsule({ force: true });
|
||||
},
|
||||
{ passive: false },
|
||||
);
|
||||
|
||||
@@ -5,6 +5,7 @@ const LEGEND_MODES = {
|
||||
satellites: { title: "卫星" },
|
||||
countryBoundaries: { title: "国界" },
|
||||
computeCenters: { title: "算力" },
|
||||
vessels: { title: "船只" },
|
||||
bgp: { title: "BGP" },
|
||||
};
|
||||
|
||||
@@ -15,6 +16,7 @@ let legendItemsByMode = {
|
||||
satellites: [],
|
||||
countryBoundaries: [],
|
||||
computeCenters: [],
|
||||
vessels: [],
|
||||
bgp: [],
|
||||
};
|
||||
|
||||
@@ -81,7 +83,7 @@ function renderLegend(mode) {
|
||||
.map(
|
||||
(item) => `
|
||||
<div class="legend-item">
|
||||
<span class="legend-dot" style="background:${item.color}"></span>
|
||||
<span class="legend-dot legend-dot--${item.shape || "dot"}" style="background:${item.color}; color:${item.color}"></span>
|
||||
<span class="legend-label">${item.label}</span>
|
||||
</div>`,
|
||||
)
|
||||
|
||||
@@ -193,6 +193,7 @@ import {
|
||||
focusEarthView,
|
||||
getZoomLevel,
|
||||
setZoomLevel,
|
||||
showZoomStatusCapsule,
|
||||
teardownControls,
|
||||
getDayNightEnabled,
|
||||
setDayNightEnabledExternal,
|
||||
@@ -257,6 +258,7 @@ let lastBGPClickTime = 0;
|
||||
let lastBGPClickCollector = null;
|
||||
let lastBGPClickType = null;
|
||||
let lastBGPClickPos = { x: 0, y: 0 };
|
||||
let lastVesselHoverPickTime = 0;
|
||||
let earthTexture = null;
|
||||
let animationFrameId = null;
|
||||
let initialized = false;
|
||||
@@ -294,7 +296,9 @@ const scratchBGPWorldPosition = new THREE.Vector3();
|
||||
const scratchComputeCenterDirection = new THREE.Vector3();
|
||||
const scratchComputeCenterWorldPosition = new THREE.Vector3();
|
||||
const scratchVesselDirection = new THREE.Vector3();
|
||||
const scratchVesselCameraLocal = new THREE.Vector3();
|
||||
const scratchVesselWorldPosition = new THREE.Vector3();
|
||||
const scratchVesselScreenPosition = new THREE.Vector3();
|
||||
const scratchSatelliteWorldPosition = new THREE.Vector3();
|
||||
const scratchSatelliteScreenPosition = new THREE.Vector3();
|
||||
const scratchViewCenterWorld = new THREE.Vector3();
|
||||
@@ -309,6 +313,8 @@ const TOOLTIP_CURSOR_OFFSET = 14; // px offset from cursor for hover tooltips
|
||||
const TOOLTIP_COORDS_OFFSET = 10; // px offset for earth-coordinate tooltip
|
||||
const RELATED_SATELLITE_HIGHLIGHT_COLOR = "#7dd3fc";
|
||||
const DRAG_POINTER_THRESHOLD_PX = 8;
|
||||
const VESSEL_HOVER_PICK_INTERVAL_MS = 100;
|
||||
const VESSEL_POINTER_RADIUS_PX = 22;
|
||||
const GLOBE_DRAGGING_CLASS = "is-globe-dragging";
|
||||
const HUD_INTERACTIVE_SELECTORS = [
|
||||
".earth-left-column",
|
||||
@@ -363,10 +369,17 @@ function setGlobeDraggingUiState(active) {
|
||||
document.documentElement.classList.toggle(GLOBE_DRAGGING_CLASS, active);
|
||||
}
|
||||
|
||||
function hasActiveGlobeInertia() {
|
||||
return (
|
||||
Math.abs(inertialVelocity.x) > INERTIA_MIN_VELOCITY ||
|
||||
Math.abs(inertialVelocity.y) > INERTIA_MIN_VELOCITY
|
||||
);
|
||||
}
|
||||
|
||||
function getDragRotationFactor() {
|
||||
const zoom = Math.max(getZoomLevel(), 0.01);
|
||||
const scale = THREE.MathUtils.clamp(
|
||||
1 / zoom,
|
||||
Math.pow(zoom, -CONFIG.dragRotationZoomExponent),
|
||||
CONFIG.dragRotationScaleMin,
|
||||
CONFIG.dragRotationScaleMax,
|
||||
);
|
||||
@@ -539,21 +552,92 @@ function getFrontFacingVesselMarkers(markers) {
|
||||
const earth = getEarth();
|
||||
if (!earth) return markers;
|
||||
|
||||
scratchCameraToEarth.subVectors(camera.position, earth.position).normalize();
|
||||
scratchVesselCameraLocal.copy(camera.position);
|
||||
earth.worldToLocal(scratchVesselCameraLocal);
|
||||
scratchVesselCameraLocal.normalize();
|
||||
|
||||
return markers.filter((marker) => {
|
||||
scratchVesselWorldPosition.copy(marker.position);
|
||||
marker.parent?.localToWorld(scratchVesselWorldPosition);
|
||||
scratchVesselDirection
|
||||
.subVectors(scratchVesselWorldPosition, earth.position)
|
||||
.normalize();
|
||||
scratchVesselDirection.copy(marker.position).normalize();
|
||||
return (
|
||||
scratchCameraToEarth.dot(scratchVesselDirection) >
|
||||
scratchVesselCameraLocal.dot(scratchVesselDirection) >
|
||||
SATELLITE_CONFIG.frontFacingDotThreshold
|
||||
);
|
||||
});
|
||||
}
|
||||
|
||||
function getVesselPointerIntersections() {
|
||||
const earth = getEarth();
|
||||
if (!earth) return [];
|
||||
scratchVesselCameraLocal.copy(camera.position);
|
||||
earth.worldToLocal(scratchVesselCameraLocal);
|
||||
scratchVesselCameraLocal.normalize();
|
||||
|
||||
const pointerX = ((interactionMouse.x + 1) / 2) * window.innerWidth;
|
||||
const pointerY = ((1 - interactionMouse.y) / 2) * window.innerHeight;
|
||||
const radiusSq = VESSEL_POINTER_RADIUS_PX * VESSEL_POINTER_RADIUS_PX;
|
||||
const intersections = [];
|
||||
|
||||
getVesselMarkers().forEach((marker) => {
|
||||
scratchVesselDirection.copy(marker.position).normalize();
|
||||
if (
|
||||
scratchVesselCameraLocal.dot(scratchVesselDirection) <=
|
||||
SATELLITE_CONFIG.frontFacingDotThreshold
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
scratchVesselWorldPosition.copy(marker.position);
|
||||
earth.localToWorld(scratchVesselWorldPosition);
|
||||
scratchVesselScreenPosition.copy(scratchVesselWorldPosition).project(camera);
|
||||
if (
|
||||
scratchVesselScreenPosition.z < -1 ||
|
||||
scratchVesselScreenPosition.z > 1
|
||||
) {
|
||||
return;
|
||||
}
|
||||
|
||||
const screenX = (scratchVesselScreenPosition.x * 0.5 + 0.5) * window.innerWidth;
|
||||
const screenY = (-scratchVesselScreenPosition.y * 0.5 + 0.5) * window.innerHeight;
|
||||
const deltaX = screenX - pointerX;
|
||||
const deltaY = screenY - pointerY;
|
||||
const distancePxSq = deltaX * deltaX + deltaY * deltaY;
|
||||
if (distancePxSq > radiusSq) return;
|
||||
|
||||
intersections.push({
|
||||
object: marker,
|
||||
point: scratchVesselWorldPosition.clone(),
|
||||
distance: camera.position.distanceTo(scratchVesselWorldPosition),
|
||||
distancePxSq,
|
||||
});
|
||||
});
|
||||
|
||||
return intersections.sort((a, b) => a.distancePxSq - b.distancePxSq);
|
||||
}
|
||||
|
||||
function shouldSkipVesselHoverPicking() {
|
||||
return isDragging || hasActiveGlobeInertia();
|
||||
}
|
||||
|
||||
function getVesselHoverIntersections() {
|
||||
if (!getShowVessels()) {
|
||||
return { checked: true, intersects: [] };
|
||||
}
|
||||
if (shouldSkipVesselHoverPicking()) {
|
||||
return { checked: false, intersects: [] };
|
||||
}
|
||||
|
||||
const now = performance.now();
|
||||
if (now - lastVesselHoverPickTime < VESSEL_HOVER_PICK_INTERVAL_MS) {
|
||||
return { checked: false, intersects: [] };
|
||||
}
|
||||
lastVesselHoverPickTime = now;
|
||||
|
||||
return {
|
||||
checked: true,
|
||||
intersects: getVesselPointerIntersections(),
|
||||
};
|
||||
}
|
||||
|
||||
function applyBGPHoverState(marker) {
|
||||
resetTransientBGPStates();
|
||||
if (!marker) {
|
||||
@@ -589,14 +673,14 @@ function applyComputeCenterHoverState(marker) {
|
||||
}
|
||||
|
||||
function resetTransientVesselStates() {
|
||||
getVesselMarkers().forEach((marker) => {
|
||||
if (marker !== lockedObject) {
|
||||
setVesselMarkerState(marker, "normal");
|
||||
}
|
||||
});
|
||||
if (hoveredVessel && hoveredVessel !== lockedObject) {
|
||||
setVesselMarkerState(hoveredVessel, "normal");
|
||||
}
|
||||
hoveredVessel = null;
|
||||
}
|
||||
|
||||
function applyVesselHoverState(marker) {
|
||||
if (isSameVessel(hoveredVessel, marker)) return;
|
||||
resetTransientVesselStates();
|
||||
if (!marker) {
|
||||
hoveredVessel = null;
|
||||
@@ -2365,6 +2449,7 @@ export function init() {
|
||||
setLegendItems("satellites", getSatelliteLegendItems());
|
||||
setLegendItems("countryBoundaries", getCountryBoundaryLegendItems());
|
||||
setLegendItems("computeCenters", getComputeCenterLegendItems());
|
||||
setLegendItems("vessels", getVesselLegendItems());
|
||||
setLegendItems("bgp", getBGPLegendItems());
|
||||
const earthObj = createEarth(scene);
|
||||
applyImmediateView(earthObj, camera);
|
||||
@@ -3132,11 +3217,8 @@ function onMouseMove(event) {
|
||||
const computeCenterIntersects = getShowComputeCenters()
|
||||
? interactionRaycaster.intersectObjects(frontFacingComputeCenterMarkers)
|
||||
: [];
|
||||
const vesselIntersects = getShowVessels()
|
||||
? interactionRaycaster.intersectObjects(
|
||||
getFrontFacingVesselMarkers(getVesselMarkers()),
|
||||
)
|
||||
: [];
|
||||
const vesselPick = getVesselHoverIntersections();
|
||||
const vesselIntersects = vesselPick.intersects;
|
||||
|
||||
let hoveredSat = null;
|
||||
let hoveredSatIndexFromIntersect = null;
|
||||
@@ -3159,7 +3241,7 @@ function onMouseMove(event) {
|
||||
const hoveredComputeCenterMarker =
|
||||
computeCenterIntersects.length > 0 ? computeCenterIntersects[0].object : null;
|
||||
const hoveredVesselMarker =
|
||||
vesselIntersects.length > 0 ? vesselIntersects[0].object : null;
|
||||
vesselPick.checked && vesselIntersects.length > 0 ? vesselIntersects[0].object : null;
|
||||
|
||||
if (
|
||||
hoveredComputeCenter &&
|
||||
@@ -3167,7 +3249,11 @@ function onMouseMove(event) {
|
||||
) {
|
||||
clearTransientHoverState();
|
||||
}
|
||||
if (hoveredVessel && !isSameVessel(hoveredVessel, hoveredVesselMarker)) {
|
||||
if (
|
||||
vesselPick.checked &&
|
||||
hoveredVessel &&
|
||||
!isSameVessel(hoveredVessel, hoveredVesselMarker)
|
||||
) {
|
||||
clearTransientHoverState();
|
||||
}
|
||||
|
||||
@@ -3214,6 +3300,7 @@ function onMouseMove(event) {
|
||||
);
|
||||
objectTooltipShown = true;
|
||||
} else if (
|
||||
vesselPick.checked &&
|
||||
hoveredVesselMarker &&
|
||||
getShowVessels() &&
|
||||
lockedObjectType !== "vessel"
|
||||
@@ -3260,7 +3347,9 @@ function onMouseMove(event) {
|
||||
} else if (!lockedObjectType && !isCruisePresentationPinned()) {
|
||||
resetTransientBGPStates();
|
||||
resetTransientComputeCenterStates();
|
||||
resetTransientVesselStates();
|
||||
if (vesselPick.checked) {
|
||||
resetTransientVesselStates();
|
||||
}
|
||||
hideInfoCard();
|
||||
}
|
||||
|
||||
@@ -3358,6 +3447,7 @@ function onPointerDown(event) {
|
||||
pinchGesture = {
|
||||
distance: getTouchDistance(firstPoint, secondPoint),
|
||||
startZoom: getZoomLevel(),
|
||||
statusShown: false,
|
||||
};
|
||||
activeDragPointerId = null;
|
||||
onMouseUp();
|
||||
@@ -3390,6 +3480,8 @@ function onPointerMove(event) {
|
||||
if (pinchGesture.distance > 0) {
|
||||
const scale = nextDistance / pinchGesture.distance;
|
||||
setZoomLevel(pinchGesture.startZoom * scale, camera);
|
||||
showZoomStatusCapsule({ force: !pinchGesture.statusShown });
|
||||
pinchGesture.statusShown = true;
|
||||
suppressNextClick = true;
|
||||
hideTooltip();
|
||||
}
|
||||
@@ -3478,9 +3570,7 @@ function onClick(event) {
|
||||
)
|
||||
: [];
|
||||
const vesselIntersects = getShowVessels()
|
||||
? interactionRaycaster.intersectObjects(
|
||||
getFrontFacingVesselMarkers(getVesselMarkers()),
|
||||
)
|
||||
? getVesselPointerIntersections()
|
||||
: [];
|
||||
const satIntersects = getSatellitePointerIntersections(event);
|
||||
|
||||
@@ -3676,9 +3766,7 @@ function animate() {
|
||||
|
||||
const earth = getEarth();
|
||||
const deltaTime = clock.getDelta() * 1000;
|
||||
const hasInertia =
|
||||
Math.abs(inertialVelocity.x) > INERTIA_MIN_VELOCITY ||
|
||||
Math.abs(inertialVelocity.y) > INERTIA_MIN_VELOCITY;
|
||||
const hasInertia = hasActiveGlobeInertia();
|
||||
|
||||
if (getAutoRotate() && getRotationMode() === ROTATION_MODE.ROTATE && earth) {
|
||||
earth.rotation.y += CONFIG.rotationSpeed * (deltaTime / 16);
|
||||
|
||||
@@ -5,6 +5,7 @@ let statusHideTimeoutId = null;
|
||||
const STATUS_BASE_CLASS = "earth-status-message";
|
||||
const STATUS_DISPLAY_MS = 3000;
|
||||
const STATUS_FADE_MS = 280;
|
||||
const GESTURE_STATUS_DISPLAY_MS = 760;
|
||||
let statusQueue = [];
|
||||
let statusBusy = false;
|
||||
let loadingActive = false;
|
||||
@@ -148,6 +149,28 @@ export function queueStatusMessage(message, type = "info") {
|
||||
processStatusQueue();
|
||||
}
|
||||
|
||||
export function showGestureStatusMessage(message, type = "info") {
|
||||
if (loadingActive) return;
|
||||
const statusEl = getElement("status-message");
|
||||
if (!statusEl) return;
|
||||
|
||||
clearStatusTimers();
|
||||
statusBusy = true;
|
||||
buildStatusContent(statusEl, message, type);
|
||||
statusEl.className = `${STATUS_BASE_CLASS} ${type} gesture`;
|
||||
setElementDisplay(statusEl, true, "inline-flex");
|
||||
statusEl.offsetHeight;
|
||||
statusEl.classList.add("visible");
|
||||
|
||||
statusTimeoutId = setTimeout(() => {
|
||||
hideStatusElement(statusEl, () => {
|
||||
statusBusy = false;
|
||||
processStatusQueue();
|
||||
});
|
||||
statusTimeoutId = null;
|
||||
}, GESTURE_STATUS_DISPLAY_MS);
|
||||
}
|
||||
|
||||
// Update coordinates display
|
||||
export function updateCoordinatesDisplay(lat, lon, alt = 0) {
|
||||
const longitudeEl = getElement("longitude-value");
|
||||
|
||||
@@ -1,15 +1,29 @@
|
||||
import * as THREE from "three";
|
||||
|
||||
import { CONFIG, PATHS, VESSEL_CONFIG } from "./constants.js";
|
||||
import { getSurfaceMarkerCameraScale, latLonToVector3 } from "./utils.js";
|
||||
import { latLonToVector3 } from "./utils.js";
|
||||
|
||||
const vesselGroup = new THREE.Group();
|
||||
const vesselMarkers = [];
|
||||
const textureCache = new Map();
|
||||
let vesselPoints = null;
|
||||
let vesselPointObjects = [];
|
||||
let hoverOverlaySprite = null;
|
||||
let lockedOverlaySprite = null;
|
||||
let showVessels = false;
|
||||
let activeTrackLine = null;
|
||||
let lastVisualStateKey = "";
|
||||
let visualStateVersion = 0;
|
||||
|
||||
const VESSEL_RENDER_ORDER = 4.4;
|
||||
const VESSEL_POINT_SIZE = 34;
|
||||
const VESSEL_ATLAS_CELL_SIZE = 128;
|
||||
const VESSEL_COURSE_BINS = 32;
|
||||
|
||||
function invalidateVesselVisualState() {
|
||||
visualStateVersion += 1;
|
||||
lastVisualStateKey = "";
|
||||
}
|
||||
|
||||
function normalizeVesselType(value, code) {
|
||||
const type = String(value || "").trim().toLowerCase();
|
||||
@@ -22,36 +36,87 @@ function normalizeVesselType(value, code) {
|
||||
return "other";
|
||||
}
|
||||
|
||||
function createVesselTexture(type, anchored) {
|
||||
const textureKey = `${type}:${anchored ? "anchored" : "moving"}`;
|
||||
if (textureCache.has(textureKey)) return textureCache.get(textureKey);
|
||||
|
||||
const color = VESSEL_CONFIG.colors[type] || VESSEL_CONFIG.colors.other;
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = 96;
|
||||
canvas.height = 96;
|
||||
const context = canvas.getContext("2d");
|
||||
context.clearRect(0, 0, 96, 96);
|
||||
context.save();
|
||||
context.translate(48, 48);
|
||||
function drawVesselShape(context, anchored, glow, color = "#ffffff") {
|
||||
context.fillStyle = color;
|
||||
context.globalAlpha = anchored ? 0.55 : 0.96;
|
||||
context.shadowColor = color;
|
||||
context.shadowBlur = anchored ? 8 : 14;
|
||||
if (glow) {
|
||||
context.shadowColor = color;
|
||||
context.shadowBlur = anchored ? 8 : 14;
|
||||
}
|
||||
context.beginPath();
|
||||
if (anchored) {
|
||||
context.arc(0, 0, 18, 0, Math.PI * 2);
|
||||
context.arc(0, 0, 24, 0, Math.PI * 2);
|
||||
} else {
|
||||
context.moveTo(0, -28);
|
||||
context.lineTo(21, 24);
|
||||
context.lineTo(0, 13);
|
||||
context.lineTo(-21, 24);
|
||||
context.moveTo(0, -37);
|
||||
context.lineTo(28, 32);
|
||||
context.lineTo(0, 17);
|
||||
context.lineTo(-28, 32);
|
||||
context.closePath();
|
||||
}
|
||||
context.fill();
|
||||
}
|
||||
|
||||
function createVesselPointTexture(anchored, courseBin = 0) {
|
||||
const textureKey = `vessel-point:${anchored ? "anchored" : "moving"}:${courseBin}`;
|
||||
if (textureCache.has(textureKey)) return textureCache.get(textureKey);
|
||||
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = VESSEL_ATLAS_CELL_SIZE;
|
||||
canvas.height = VESSEL_ATLAS_CELL_SIZE;
|
||||
const context = canvas.getContext("2d");
|
||||
context.clearRect(0, 0, canvas.width, canvas.height);
|
||||
context.save();
|
||||
context.translate(canvas.width / 2, canvas.height / 2);
|
||||
if (!anchored) {
|
||||
context.rotate((courseBin / VESSEL_COURSE_BINS) * Math.PI * 2);
|
||||
}
|
||||
drawVesselShape(context, anchored, false);
|
||||
context.restore();
|
||||
|
||||
const texture = new THREE.CanvasTexture(canvas);
|
||||
texture.generateMipmaps = false;
|
||||
texture.minFilter = THREE.LinearFilter;
|
||||
texture.magFilter = THREE.LinearFilter;
|
||||
texture.needsUpdate = true;
|
||||
textureCache.set(textureKey, texture);
|
||||
return texture;
|
||||
}
|
||||
|
||||
function createVesselOverlayTexture(marker, glow = true) {
|
||||
const kind = marker?.userData?.vessel_kind || "other";
|
||||
const anchored = Boolean(marker?.userData?.anchored);
|
||||
const courseBin = marker ? getCourseBin(marker) : 0;
|
||||
const textureKey = [
|
||||
"vessel-overlay",
|
||||
kind,
|
||||
anchored ? "anchored" : "moving",
|
||||
courseBin,
|
||||
glow ? "glow" : "plain",
|
||||
].join(":");
|
||||
if (textureCache.has(textureKey)) return textureCache.get(textureKey);
|
||||
|
||||
const canvas = document.createElement("canvas");
|
||||
canvas.width = VESSEL_ATLAS_CELL_SIZE;
|
||||
canvas.height = VESSEL_ATLAS_CELL_SIZE;
|
||||
const context = canvas.getContext("2d");
|
||||
context.clearRect(0, 0, canvas.width, canvas.height);
|
||||
context.save();
|
||||
context.translate(canvas.width / 2, canvas.height / 2);
|
||||
if (!anchored) {
|
||||
context.rotate((courseBin / VESSEL_COURSE_BINS) * Math.PI * 2);
|
||||
}
|
||||
drawVesselShape(
|
||||
context,
|
||||
anchored,
|
||||
glow,
|
||||
VESSEL_CONFIG.colors[kind] || VESSEL_CONFIG.colors.other,
|
||||
);
|
||||
context.restore();
|
||||
|
||||
const texture = new THREE.CanvasTexture(canvas);
|
||||
texture.generateMipmaps = false;
|
||||
texture.minFilter = THREE.LinearFilter;
|
||||
texture.magFilter = THREE.LinearFilter;
|
||||
texture.needsUpdate = true;
|
||||
textureCache.set(textureKey, texture);
|
||||
return texture;
|
||||
@@ -80,16 +145,7 @@ function buildVesselMarkerData(feature) {
|
||||
}
|
||||
|
||||
function createVesselMarker(markerData) {
|
||||
const material = new THREE.SpriteMaterial({
|
||||
map: createVesselTexture(markerData.type, markerData.anchored),
|
||||
transparent: true,
|
||||
depthWrite: false,
|
||||
opacity: VESSEL_CONFIG.marker.baseOpacity,
|
||||
rotation: markerData.anchored
|
||||
? 0
|
||||
: THREE.MathUtils.degToRad(-markerData.course),
|
||||
});
|
||||
const marker = new THREE.Sprite(material);
|
||||
const marker = new THREE.Object3D();
|
||||
marker.position.copy(
|
||||
latLonToVector3(
|
||||
markerData.latitude,
|
||||
@@ -97,9 +153,6 @@ function createVesselMarker(markerData) {
|
||||
CONFIG.earthRadius + VESSEL_CONFIG.altitudeOffset,
|
||||
),
|
||||
);
|
||||
marker.scale.setScalar(VESSEL_CONFIG.marker.baseScale);
|
||||
marker.renderOrder = VESSEL_RENDER_ORDER;
|
||||
marker.visible = showVessels;
|
||||
marker.userData = {
|
||||
...markerData,
|
||||
type: "vessel",
|
||||
@@ -107,10 +160,88 @@ function createVesselMarker(markerData) {
|
||||
baseScale: VESSEL_CONFIG.marker.baseScale,
|
||||
state: "normal",
|
||||
};
|
||||
vesselGroup.add(marker);
|
||||
vesselMarkers.push(marker);
|
||||
}
|
||||
|
||||
function colorToRgbArray(colorValue) {
|
||||
const color = new THREE.Color(colorValue || VESSEL_CONFIG.colors.other);
|
||||
return [color.r, color.g, color.b];
|
||||
}
|
||||
|
||||
function getCourseBin(marker) {
|
||||
if (marker.userData.anchored) return 0;
|
||||
const course = Number(marker.userData.course || 0);
|
||||
const normalized = ((course % 360) + 360) % 360;
|
||||
return Math.round((normalized / 360) * VESSEL_COURSE_BINS) % VESSEL_COURSE_BINS;
|
||||
}
|
||||
|
||||
function buildVesselPoints() {
|
||||
vesselPoints = new THREE.Group();
|
||||
vesselPoints.visible = showVessels;
|
||||
vesselPoints.renderOrder = VESSEL_RENDER_ORDER;
|
||||
vesselPoints.userData = { type: "vessel_points" };
|
||||
vesselPointObjects = [];
|
||||
|
||||
const groups = new Map();
|
||||
vesselMarkers.forEach((marker, index) => {
|
||||
const anchored = Boolean(marker.userData.anchored);
|
||||
const courseBin = getCourseBin(marker);
|
||||
const key = `${anchored ? "anchored" : "moving"}:${courseBin}`;
|
||||
if (!groups.has(key)) {
|
||||
groups.set(key, {
|
||||
anchored,
|
||||
courseBin,
|
||||
markers: [],
|
||||
markerIndexes: [],
|
||||
});
|
||||
}
|
||||
const group = groups.get(key);
|
||||
group.markers.push(marker);
|
||||
group.markerIndexes.push(index);
|
||||
});
|
||||
|
||||
groups.forEach((group) => {
|
||||
const count = group.markers.length;
|
||||
const positions = new Float32Array(count * 3);
|
||||
const colors = new Float32Array(count * 3);
|
||||
group.markers.forEach((marker, index) => {
|
||||
positions[index * 3] = marker.position.x;
|
||||
positions[index * 3 + 1] = marker.position.y;
|
||||
positions[index * 3 + 2] = marker.position.z;
|
||||
const [r, g, b] = colorToRgbArray(VESSEL_CONFIG.colors[marker.userData.vessel_kind]);
|
||||
colors[index * 3] = r;
|
||||
colors[index * 3 + 1] = g;
|
||||
colors[index * 3 + 2] = b;
|
||||
});
|
||||
|
||||
const geometry = new THREE.BufferGeometry();
|
||||
geometry.setAttribute("position", new THREE.BufferAttribute(positions, 3));
|
||||
geometry.setAttribute("color", new THREE.BufferAttribute(colors, 3));
|
||||
geometry.computeBoundingSphere();
|
||||
const material = new THREE.PointsMaterial({
|
||||
map: createVesselPointTexture(group.anchored, group.courseBin),
|
||||
size: VESSEL_POINT_SIZE,
|
||||
sizeAttenuation: false,
|
||||
vertexColors: true,
|
||||
transparent: true,
|
||||
opacity: VESSEL_CONFIG.marker.baseOpacity,
|
||||
depthWrite: false,
|
||||
depthTest: true,
|
||||
alphaTest: 0.01,
|
||||
});
|
||||
const points = new THREE.Points(geometry, material);
|
||||
points.renderOrder = VESSEL_RENDER_ORDER;
|
||||
points.frustumCulled = false;
|
||||
points.userData = {
|
||||
type: "vessel_points",
|
||||
};
|
||||
vesselPointObjects.push(points);
|
||||
vesselPoints.add(points);
|
||||
});
|
||||
|
||||
vesselGroup.add(vesselPoints);
|
||||
}
|
||||
|
||||
function clearGroup(group) {
|
||||
for (let index = group.children.length - 1; index >= 0; index -= 1) {
|
||||
const child = group.children[index];
|
||||
@@ -120,13 +251,24 @@ function clearGroup(group) {
|
||||
}
|
||||
}
|
||||
|
||||
function getDistanceScale(camera) {
|
||||
return getSurfaceMarkerCameraScale(camera, {
|
||||
altitudeOffset: VESSEL_CONFIG.altitudeOffset,
|
||||
referenceFov: 75,
|
||||
min: VESSEL_CONFIG.sizeStabilization.min,
|
||||
max: VESSEL_CONFIG.sizeStabilization.max,
|
||||
function disposeVesselRenderObjects() {
|
||||
if (vesselPoints?.parent) {
|
||||
vesselPoints.parent.remove(vesselPoints);
|
||||
}
|
||||
vesselPointObjects.forEach((points) => {
|
||||
points.geometry?.dispose?.();
|
||||
points.material?.dispose?.();
|
||||
});
|
||||
hoverOverlaySprite?.geometry?.dispose?.();
|
||||
hoverOverlaySprite?.material?.dispose?.();
|
||||
hoverOverlaySprite?.parent?.remove?.(hoverOverlaySprite);
|
||||
lockedOverlaySprite?.geometry?.dispose?.();
|
||||
lockedOverlaySprite?.material?.dispose?.();
|
||||
lockedOverlaySprite?.parent?.remove?.(lockedOverlaySprite);
|
||||
vesselPoints = null;
|
||||
vesselPointObjects = [];
|
||||
hoverOverlaySprite = null;
|
||||
lockedOverlaySprite = null;
|
||||
}
|
||||
|
||||
export function getVesselMarkers() {
|
||||
@@ -143,10 +285,11 @@ export function getShowVessels() {
|
||||
|
||||
export function toggleVessels(show) {
|
||||
showVessels = Boolean(show);
|
||||
invalidateVesselVisualState();
|
||||
vesselGroup.visible = showVessels;
|
||||
vesselMarkers.forEach((marker) => {
|
||||
marker.visible = showVessels;
|
||||
});
|
||||
if (vesselPoints) {
|
||||
vesselPoints.visible = showVessels;
|
||||
}
|
||||
if (activeTrackLine) {
|
||||
activeTrackLine.visible = showVessels;
|
||||
}
|
||||
@@ -168,12 +311,16 @@ function clearVesselTrack() {
|
||||
|
||||
export function setVesselMarkerState(marker, state = "normal") {
|
||||
if (!marker || marker.userData?.type !== "vessel") return;
|
||||
if (marker.userData.state === state) return;
|
||||
marker.userData.state = state;
|
||||
invalidateVesselVisualState();
|
||||
}
|
||||
|
||||
export function clearVesselData(earth) {
|
||||
vesselMarkers.length = 0;
|
||||
invalidateVesselVisualState();
|
||||
clearVesselSelection();
|
||||
vesselMarkers.length = 0;
|
||||
disposeVesselRenderObjects();
|
||||
clearGroup(vesselGroup);
|
||||
if (earth && vesselGroup.parent === earth) {
|
||||
earth.remove(vesselGroup);
|
||||
@@ -196,6 +343,7 @@ export async function loadVessels(_scene, earth, options = {}) {
|
||||
.filter(Boolean)
|
||||
.slice(0, VESSEL_CONFIG.maxRenderedMarkers)
|
||||
.forEach((markerData) => createVesselMarker(markerData));
|
||||
buildVesselPoints();
|
||||
|
||||
if (earth && !vesselGroup.parent) {
|
||||
earth.add(vesselGroup);
|
||||
@@ -246,35 +394,112 @@ export async function showVesselTrack(marker, earth) {
|
||||
|
||||
export function getVesselLegendItems() {
|
||||
return [
|
||||
{ label: "货轮", color: VESSEL_CONFIG.colors.cargo },
|
||||
{ label: "油轮", color: VESSEL_CONFIG.colors.tanker },
|
||||
{ label: "客船", color: VESSEL_CONFIG.colors.passenger },
|
||||
{ label: "渔船", color: VESSEL_CONFIG.colors.fishing },
|
||||
{ label: "军舰", color: VESSEL_CONFIG.colors.military },
|
||||
{ label: "其他", color: VESSEL_CONFIG.colors.other },
|
||||
{ label: "货轮", color: VESSEL_CONFIG.colors.cargo, shape: "vessel" },
|
||||
{ label: "油轮", color: VESSEL_CONFIG.colors.tanker, shape: "vessel" },
|
||||
{ label: "客船", color: VESSEL_CONFIG.colors.passenger, shape: "vessel" },
|
||||
{ label: "渔船", color: VESSEL_CONFIG.colors.fishing, shape: "vessel" },
|
||||
{ label: "军舰", color: VESSEL_CONFIG.colors.military, shape: "vessel" },
|
||||
{ label: "停泊/低速", color: VESSEL_CONFIG.colors.other, shape: "dot" },
|
||||
{ label: "其他船只", color: VESSEL_CONFIG.colors.other, shape: "vessel" },
|
||||
];
|
||||
}
|
||||
|
||||
export function updateVesselVisualState(lockedObjectType, lockedObject, camera) {
|
||||
const hasFocus = lockedObjectType === "vessel" && lockedObject;
|
||||
const distanceScale = getDistanceScale(camera);
|
||||
vesselMarkers.forEach((marker) => {
|
||||
const isLocked = lockedObjectType === "vessel" && lockedObject === marker;
|
||||
const state = marker.userData?.state || "normal";
|
||||
let opacity = VESSEL_CONFIG.marker.baseOpacity;
|
||||
let scaleMultiplier = 1;
|
||||
if (isLocked) {
|
||||
opacity = 1;
|
||||
scaleMultiplier = VESSEL_CONFIG.marker.lockedScale;
|
||||
} else if (state === "hover") {
|
||||
opacity = 0.98;
|
||||
scaleMultiplier = VESSEL_CONFIG.marker.hoverScale;
|
||||
} else if (hasFocus) {
|
||||
opacity = VESSEL_CONFIG.marker.dimmedOpacity;
|
||||
scaleMultiplier = VESSEL_CONFIG.marker.dimmedScale;
|
||||
}
|
||||
marker.material.opacity = showVessels ? opacity : 0;
|
||||
marker.scale.setScalar(marker.userData.baseScale * scaleMultiplier * distanceScale);
|
||||
marker.visible = showVessels;
|
||||
function ensureOverlaySprite(kind) {
|
||||
const existing = kind === "locked" ? lockedOverlaySprite : hoverOverlaySprite;
|
||||
if (existing) return existing;
|
||||
|
||||
const geometry = new THREE.BufferGeometry();
|
||||
geometry.setAttribute(
|
||||
"position",
|
||||
new THREE.BufferAttribute(new Float32Array(3), 3),
|
||||
);
|
||||
const material = new THREE.PointsMaterial({
|
||||
size: VESSEL_POINT_SIZE,
|
||||
sizeAttenuation: false,
|
||||
transparent: true,
|
||||
depthWrite: false,
|
||||
depthTest: true,
|
||||
opacity: 1,
|
||||
alphaTest: 0.01,
|
||||
});
|
||||
const overlay = new THREE.Points(geometry, material);
|
||||
overlay.renderOrder = VESSEL_RENDER_ORDER + (kind === "locked" ? 0.2 : 0.1);
|
||||
overlay.frustumCulled = false;
|
||||
overlay.visible = false;
|
||||
vesselGroup.add(overlay);
|
||||
if (kind === "locked") {
|
||||
lockedOverlaySprite = overlay;
|
||||
} else {
|
||||
hoverOverlaySprite = overlay;
|
||||
}
|
||||
return overlay;
|
||||
}
|
||||
|
||||
function updateOverlaySprite(overlay, marker, opacity) {
|
||||
if (!overlay) return;
|
||||
if (!marker) {
|
||||
overlay.visible = false;
|
||||
return;
|
||||
}
|
||||
|
||||
const texture = createVesselOverlayTexture(marker, true);
|
||||
if (overlay.material.map !== texture) {
|
||||
overlay.material.map = texture;
|
||||
overlay.material.needsUpdate = true;
|
||||
}
|
||||
overlay.material.opacity = opacity;
|
||||
overlay.material.size = VESSEL_POINT_SIZE;
|
||||
const positionAttribute = overlay.geometry.getAttribute("position");
|
||||
positionAttribute.setXYZ(0, marker.position.x, marker.position.y, marker.position.z);
|
||||
positionAttribute.needsUpdate = true;
|
||||
overlay.visible = showVessels;
|
||||
}
|
||||
|
||||
export function updateVesselVisualState(lockedObjectType, lockedObject, camera) {
|
||||
if (!showVessels || vesselMarkers.length === 0 || !vesselPoints) {
|
||||
if (lastVisualStateKey !== "hidden") {
|
||||
if (vesselPoints) vesselPoints.visible = false;
|
||||
if (hoverOverlaySprite) hoverOverlaySprite.visible = false;
|
||||
if (lockedOverlaySprite) lockedOverlaySprite.visible = false;
|
||||
lastVisualStateKey = "hidden";
|
||||
}
|
||||
return;
|
||||
}
|
||||
|
||||
vesselPoints.visible = true;
|
||||
const hasFocus = lockedObjectType === "vessel" && lockedObject;
|
||||
const lockedKey = hasFocus
|
||||
? lockedObject?.userData?.mmsi || lockedObject?.uuid || "locked"
|
||||
: "none";
|
||||
const stateKey = [
|
||||
"visible",
|
||||
lockedObjectType || "none",
|
||||
lockedKey,
|
||||
visualStateVersion,
|
||||
].join(":");
|
||||
|
||||
if (stateKey === lastVisualStateKey) return;
|
||||
lastVisualStateKey = stateKey;
|
||||
|
||||
vesselPointObjects.forEach((points) => {
|
||||
points.visible = showVessels;
|
||||
points.material.opacity = hasFocus
|
||||
? VESSEL_CONFIG.marker.dimmedOpacity
|
||||
: VESSEL_CONFIG.marker.baseOpacity;
|
||||
points.material.size = VESSEL_POINT_SIZE;
|
||||
});
|
||||
|
||||
const hoverMarker = vesselMarkers.find(
|
||||
(marker) => marker.userData?.state === "hover" && marker !== lockedObject,
|
||||
);
|
||||
updateOverlaySprite(
|
||||
ensureOverlaySprite("hover"),
|
||||
hoverMarker,
|
||||
0.98,
|
||||
);
|
||||
updateOverlaySprite(
|
||||
ensureOverlaySprite("locked"),
|
||||
hasFocus ? lockedObject : null,
|
||||
1,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -1487,6 +1487,49 @@ body {
|
||||
color: #cf1322;
|
||||
}
|
||||
|
||||
.data-source-bulk-toolbar__running-pill {
|
||||
width: fit-content;
|
||||
display: inline-flex;
|
||||
align-items: center;
|
||||
gap: 8px;
|
||||
padding: 6px 10px;
|
||||
border: 1px solid #91caff;
|
||||
border-radius: 999px;
|
||||
background: #ffffff;
|
||||
color: #475467;
|
||||
line-height: 1;
|
||||
cursor: pointer;
|
||||
box-shadow: 0 1px 0 rgba(22, 119, 255, 0.08);
|
||||
transition: border-color 0.16s ease, box-shadow 0.16s ease, transform 0.16s ease;
|
||||
}
|
||||
|
||||
.data-source-bulk-toolbar__running-pill:hover {
|
||||
border-color: #1677ff;
|
||||
box-shadow: 0 6px 14px rgba(22, 119, 255, 0.16);
|
||||
transform: translateY(-1px);
|
||||
}
|
||||
|
||||
.data-source-bulk-toolbar__running-pill strong {
|
||||
color: #0958d9;
|
||||
font-size: 13px;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.data-source-bulk-toolbar__running-dot {
|
||||
width: 7px;
|
||||
height: 7px;
|
||||
border-radius: 999px;
|
||||
background: #1677ff;
|
||||
box-shadow: 0 0 0 4px rgba(22, 119, 255, 0.12);
|
||||
}
|
||||
|
||||
.data-source-bulk-toolbar__running-arrow {
|
||||
color: #1677ff;
|
||||
font-size: 16px;
|
||||
line-height: 1;
|
||||
margin-left: 2px;
|
||||
}
|
||||
|
||||
.data-source-bulk-toolbar__progress {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
|
||||
@@ -48,7 +48,7 @@ interface RestartTaskLogs {
|
||||
lines: string[]
|
||||
}
|
||||
|
||||
type RestartAction = 'restart-backend' | 'restart-ai-provider' | 'restart-database' | 'restart-system'
|
||||
type RestartAction = 'restart-backend' | 'restart-frontend' | 'restart-ai-provider' | 'restart-database' | 'restart-system'
|
||||
type RestartStage = 'confirming' | 'waiting_for_shutdown' | 'waiting_for_recovery' | 'recovered' | 'failed' | 'timeout'
|
||||
|
||||
const RESTART_ACTION_OPTIONS: Array<{ value: RestartAction; label: string; description: string; command: string }> = [
|
||||
@@ -58,6 +58,12 @@ const RESTART_ACTION_OPTIONS: Array<{ value: RestartAction; label: string; descr
|
||||
description: '只重启后端服务,页面通常会短暂失联后自动恢复。',
|
||||
command: './planet.sh restart -b',
|
||||
},
|
||||
{
|
||||
value: 'restart-frontend',
|
||||
label: '重启前端',
|
||||
description: '只重启前端开发服务,页面会短暂不可用,恢复后自动刷新。',
|
||||
command: './planet.sh restart -f',
|
||||
},
|
||||
{
|
||||
value: 'restart-ai-provider',
|
||||
label: '重启 AI Provider',
|
||||
@@ -84,6 +90,11 @@ const RESTART_GUIDE_LINES: Record<RestartAction, string[]> = {
|
||||
'[ctl] handing restart to detached runner',
|
||||
'[ctl] waiting for backend health recovery',
|
||||
],
|
||||
'restart-frontend': [
|
||||
'[ctl] preparing frontend restart task',
|
||||
'[ctl] handing restart to detached runner',
|
||||
'[ctl] waiting for frontend entrypoint recovery',
|
||||
],
|
||||
'restart-ai-provider': [
|
||||
'[ctl] preparing ai provider restart task',
|
||||
'[ctl] handing restart to detached runner',
|
||||
@@ -106,6 +117,9 @@ const RESTART_GUIDE_LINES: Record<RestartAction, string[]> = {
|
||||
let cachedDashboardStats: Stats | null = null
|
||||
|
||||
function getRestartConfirmMessage(action: RestartAction): string {
|
||||
if (action === 'restart-frontend') {
|
||||
return '将重启前端开发服务,页面会短暂不可用,恢复后会自动刷新。'
|
||||
}
|
||||
if (action === 'restart-ai-provider') {
|
||||
return '将重启 AI Provider 适配服务,页面通常保持在线,但 AI 分析请求会短暂不可用。'
|
||||
}
|
||||
@@ -215,6 +229,8 @@ function Dashboard() {
|
||||
setRestartMessage(
|
||||
restartAction === 'restart-system'
|
||||
? '已发送完全重启指令,页面可能暂时失联,恢复后会自动刷新。'
|
||||
: restartAction === 'restart-frontend'
|
||||
? '已发送前端重启指令,正在等待页面入口恢复。'
|
||||
: restartAction === 'restart-ai-provider'
|
||||
? '已发送 AI Provider 重启指令,正在等待 AI 服务恢复。'
|
||||
: '已发送重启指令,正在等待服务进入重启流程。'
|
||||
@@ -290,14 +306,18 @@ function Dashboard() {
|
||||
// Backend may be temporarily down during restart; handled by health polling below.
|
||||
}
|
||||
|
||||
if (restartAction === 'restart-system') {
|
||||
if (restartAction === 'restart-system' || restartAction === 'restart-frontend') {
|
||||
try {
|
||||
const rootRes = await fetch(`/?restart_probe=${Date.now()}`, { cache: 'no-store' })
|
||||
if (rootRes.ok) {
|
||||
frontendHealthyStreak += 1
|
||||
if (sawUnhealthy && frontendHealthyStreak >= 2) {
|
||||
setRestartStage('recovered')
|
||||
setRestartMessage('系统已恢复,正在刷新页面。')
|
||||
setRestartMessage(
|
||||
restartAction === 'restart-frontend'
|
||||
? '前端已恢复,正在刷新页面。'
|
||||
: '系统已恢复,正在刷新页面。'
|
||||
)
|
||||
appendLog('[ctl] frontend entrypoint reachable again')
|
||||
window.setTimeout(() => window.location.reload(), 600)
|
||||
return
|
||||
@@ -306,14 +326,22 @@ function Dashboard() {
|
||||
sawUnhealthy = true
|
||||
frontendHealthyStreak = 0
|
||||
setRestartStage('waiting_for_recovery')
|
||||
setRestartMessage('系统正在完全重启,正在等待前端恢复访问。')
|
||||
setRestartMessage(
|
||||
restartAction === 'restart-frontend'
|
||||
? '前端正在重启,正在等待页面入口恢复访问。'
|
||||
: '系统正在完全重启,正在等待前端恢复访问。'
|
||||
)
|
||||
appendLog('[ctl] frontend is temporarily unavailable')
|
||||
}
|
||||
} catch {
|
||||
sawUnhealthy = true
|
||||
frontendHealthyStreak = 0
|
||||
setRestartStage('waiting_for_recovery')
|
||||
setRestartMessage('系统正在完全重启,正在等待前端恢复访问。')
|
||||
setRestartMessage(
|
||||
restartAction === 'restart-frontend'
|
||||
? '前端正在重启,正在等待页面入口恢复访问。'
|
||||
: '系统正在完全重启,正在等待前端恢复访问。'
|
||||
)
|
||||
appendLog('[ctl] frontend is temporarily unavailable')
|
||||
}
|
||||
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,8 +1,19 @@
|
||||
import { useEffect, useRef, useState, type ReactNode } from 'react'
|
||||
import { useCollapsedActions } from '../../hooks'
|
||||
import { TableActions, actionCellProps } from '../../components/TableActions/TableActions'
|
||||
import { ApiOutlined, CheckCircleOutlined, DeleteOutlined, EditOutlined, PlusOutlined, SyncOutlined } from '@ant-design/icons'
|
||||
import {
|
||||
ApiOutlined,
|
||||
BookOutlined,
|
||||
CheckCircleOutlined,
|
||||
DeleteOutlined,
|
||||
EditOutlined,
|
||||
PlusOutlined,
|
||||
ReloadOutlined,
|
||||
RobotOutlined,
|
||||
SyncOutlined,
|
||||
} from '@ant-design/icons'
|
||||
import {
|
||||
Alert,
|
||||
Button,
|
||||
Card,
|
||||
Checkbox,
|
||||
@@ -24,6 +35,7 @@ import axios from 'axios'
|
||||
import AppLayout from '../../components/AppLayout/AppLayout'
|
||||
import Scrollbar from '../../components/Scrollbar/Scrollbar'
|
||||
import TableScrollRegion from '../../components/Scrollbar/TableScrollRegion'
|
||||
import MarkdownRenderer from '../../components/MarkdownRenderer/MarkdownRenderer'
|
||||
import { formatDateTimeZhCN } from '../../utils/datetime'
|
||||
import { useSearchParams } from 'react-router-dom'
|
||||
|
||||
@@ -138,6 +150,84 @@ interface AIProviderPreset {
|
||||
refresh_error?: string
|
||||
}
|
||||
|
||||
interface CredentialGuide {
|
||||
provider: string
|
||||
title: string
|
||||
markdown: string
|
||||
prompt: string
|
||||
source: string
|
||||
}
|
||||
|
||||
interface CollectorConfigOption {
|
||||
name: string
|
||||
default_url: string
|
||||
endpoint: string
|
||||
is_overridden: boolean
|
||||
is_active: boolean
|
||||
source_type: string
|
||||
auth_type: string
|
||||
headers: Record<string, string>
|
||||
config: Record<string, any>
|
||||
config_id: number | null
|
||||
description: string
|
||||
}
|
||||
|
||||
function PlugConnectIcon() {
|
||||
return (
|
||||
<svg width="1em" height="1em" viewBox="0 0 24 24" aria-hidden="true" focusable="false">
|
||||
<path stroke="none" d="M0 0h24v24H0z" fill="none" />
|
||||
<path
|
||||
d="M7 12l5 5l-1.5 1.5a3.536 3.536 0 1 1 -5 -5l1.5 -1.5"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M17 12l-5 -5l1.5 -1.5a3.536 3.536 0 1 1 5 5l-1.5 1.5"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M3 21l2.5 -2.5"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M18.5 5.5l2.5 -2.5"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M10 11l-2 2"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
<path
|
||||
d="M13 14l-2 2"
|
||||
fill="none"
|
||||
stroke="currentColor"
|
||||
strokeWidth="2"
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
/>
|
||||
</svg>
|
||||
)
|
||||
}
|
||||
|
||||
function SettingsPanel({
|
||||
loading,
|
||||
children,
|
||||
@@ -166,9 +256,19 @@ function Settings() {
|
||||
const [tvSettings, setTvSettings] = useState<TVSettings | null>(null)
|
||||
const [integrations, setIntegrations] = useState<ExternalIntegrations | null>(null)
|
||||
const [aiProviderPresets, setAiProviderPresets] = useState<AIProviderPreset[]>([])
|
||||
const [collectorConfigs, setCollectorConfigs] = useState<CollectorConfigOption[]>([])
|
||||
const [selectedCollectorSource, setSelectedCollectorSource] = useState<string>('barentswatch_vessels')
|
||||
const [savingCollectorConfig, setSavingCollectorConfig] = useState(false)
|
||||
const [collectorHealthStatus, setCollectorHealthStatus] = useState<Record<string, { ok: boolean; message: string }>>({})
|
||||
const [refreshingAiPreset, setRefreshingAiPreset] = useState(false)
|
||||
const [savingTvSettings, setSavingTvSettings] = useState(false)
|
||||
const [savingIntegrations, setSavingIntegrations] = useState(false)
|
||||
const [testingCredentialProvider, setTestingCredentialProvider] = useState<string | null>(null)
|
||||
const [credentialGuide, setCredentialGuide] = useState<CredentialGuide | null>(null)
|
||||
const [credentialGuideOpen, setCredentialGuideOpen] = useState(false)
|
||||
const [credentialGuideLoading, setCredentialGuideLoading] = useState(false)
|
||||
const [credentialGuideGenerating, setCredentialGuideGenerating] = useState(false)
|
||||
const [credentialGuideResetting, setCredentialGuideResetting] = useState(false)
|
||||
const [editingSource, setEditingSource] = useState<TVStreamSource | null>(null)
|
||||
const [tvActionsCollapsed, tvTableRef] = useCollapsedActions(780)
|
||||
const collectorTableRegionRef = useRef<HTMLDivElement | null>(null)
|
||||
@@ -177,9 +277,14 @@ function Settings() {
|
||||
const [notificationForm] = Form.useForm<NotificationSettings>()
|
||||
const [securityForm] = Form.useForm<SecuritySettings>()
|
||||
const [integrationForm] = Form.useForm()
|
||||
const [collectorConfigForm] = Form.useForm()
|
||||
const [tvEditForm] = Form.useForm<TVStreamSource>()
|
||||
const selectedAiProvider = Form.useWatch(['ai_provider', 'provider'], integrationForm)
|
||||
const credentialCollectors = collectors.filter((collector) => collector.requires_credentials)
|
||||
const selectedCollector = collectors.find((collector) => collector.source === selectedCollectorSource)
|
||||
const selectedCollectorConfig = collectorConfigs.find((config) => config.name === selectedCollectorSource)
|
||||
const selectedCollectorHealth = selectedCollector
|
||||
? collectorHealthStatus[selectedCollector.source]
|
||||
: undefined
|
||||
const settingsTabKeys = new Set([
|
||||
'display',
|
||||
'notifications',
|
||||
@@ -208,9 +313,10 @@ function Settings() {
|
||||
const fetchSettings = async () => {
|
||||
try {
|
||||
setLoading(true)
|
||||
const [response, presetsResponse] = await Promise.all([
|
||||
const [response, presetsResponse, collectorConfigsResponse] = await Promise.all([
|
||||
axios.get('/api/v1/settings'),
|
||||
axios.get('/api/v1/settings/integrations/ai-provider/presets'),
|
||||
axios.get('/api/v1/datasources/configs/all'),
|
||||
])
|
||||
setSystemSettings(response.data.system)
|
||||
setNotificationSettings(response.data.notifications)
|
||||
@@ -219,6 +325,7 @@ function Settings() {
|
||||
setIntegrations(response.data.integrations || null)
|
||||
setCollectors(response.data.collectors || [])
|
||||
setAiProviderPresets(presetsResponse.data.data || [])
|
||||
setCollectorConfigs(collectorConfigsResponse.data.data || [])
|
||||
} catch (error) {
|
||||
message.error('获取系统配置失败')
|
||||
console.error(error)
|
||||
@@ -270,12 +377,25 @@ function Settings() {
|
||||
barentswatch: {
|
||||
endpoint: integrations.barentswatch.endpoint,
|
||||
client_id: integrations.barentswatch.client_id,
|
||||
client_secret: '',
|
||||
clear_client_secret: false,
|
||||
client_secret: integrations.barentswatch.client_secret.configured
|
||||
? integrations.barentswatch.client_secret.preview
|
||||
: '',
|
||||
},
|
||||
})
|
||||
}, [integrationForm, integrations, loading])
|
||||
|
||||
useEffect(() => {
|
||||
if (loading || !selectedCollectorConfig) return
|
||||
collectorConfigForm.setFieldsValue({
|
||||
endpoint: selectedCollectorConfig.endpoint,
|
||||
headers: Object.entries(selectedCollectorConfig.headers || {}).map(([key, value]) => ({ key, value })),
|
||||
config: {
|
||||
timeout: selectedCollectorConfig.config?.timeout ?? 30,
|
||||
retry: selectedCollectorConfig.config?.retry ?? 3,
|
||||
},
|
||||
})
|
||||
}, [collectorConfigForm, loading, selectedCollectorConfig])
|
||||
|
||||
useEffect(() => {
|
||||
const updateTableHeight = () => {
|
||||
const regionHeight = collectorTableRegionRef.current?.offsetHeight || 0
|
||||
@@ -311,6 +431,15 @@ function Settings() {
|
||||
)
|
||||
}
|
||||
|
||||
const headersListToMap = (headers?: Array<{ key?: string; value?: string }>) => (
|
||||
(headers || []).reduce<Record<string, string>>((acc, item) => {
|
||||
const key = item?.key?.trim()
|
||||
const value = item?.value?.trim() ?? ''
|
||||
if (key) acc[key] = value
|
||||
return acc
|
||||
}, {})
|
||||
)
|
||||
|
||||
const saveCollector = async (collector: CollectorSettings) => {
|
||||
try {
|
||||
setSavingCollectorId(collector.id)
|
||||
@@ -332,7 +461,17 @@ function Settings() {
|
||||
const saveIntegrations = async (values: any) => {
|
||||
try {
|
||||
setSavingIntegrations(true)
|
||||
const response = await axios.put('/api/v1/settings/integrations', values)
|
||||
const payload = {
|
||||
...values,
|
||||
barentswatch: {
|
||||
...values.barentswatch,
|
||||
client_secret:
|
||||
values.barentswatch?.client_secret === integrations?.barentswatch.client_secret.preview
|
||||
? ''
|
||||
: values.barentswatch?.client_secret,
|
||||
},
|
||||
}
|
||||
const response = await axios.put('/api/v1/settings/integrations', payload)
|
||||
setIntegrations(response.data.integrations)
|
||||
message.success('外部集成配置已保存')
|
||||
await fetchSettings()
|
||||
@@ -343,6 +482,177 @@ function Settings() {
|
||||
}
|
||||
}
|
||||
|
||||
const saveSelectedCollectorSettings = async () => {
|
||||
if (!selectedCollector || !selectedCollectorConfig) return
|
||||
try {
|
||||
setSavingCollectorConfig(true)
|
||||
const baseValues = collectorConfigForm.getFieldsValue(true)
|
||||
const headers = headersListToMap(baseValues.headers)
|
||||
|
||||
if (selectedCollector.source === 'barentswatch_vessels') {
|
||||
const integrationValues = integrationForm.getFieldsValue(true)
|
||||
await saveIntegrations({
|
||||
...integrationValues,
|
||||
barentswatch: {
|
||||
...integrationValues.barentswatch,
|
||||
endpoint: baseValues.endpoint,
|
||||
},
|
||||
})
|
||||
return
|
||||
}
|
||||
|
||||
const payload = {
|
||||
name: selectedCollector.source,
|
||||
description: `内置采集器覆盖配置:${selectedCollector.name}`,
|
||||
source_type: 'http',
|
||||
endpoint: baseValues.endpoint,
|
||||
auth_type: 'none',
|
||||
auth_config: {},
|
||||
headers,
|
||||
config: baseValues.config || {},
|
||||
}
|
||||
if (selectedCollectorConfig.config_id) {
|
||||
await axios.put(`/api/v1/datasources/configs/${selectedCollectorConfig.config_id}`, payload)
|
||||
} else {
|
||||
await axios.post('/api/v1/datasources/configs', payload)
|
||||
}
|
||||
message.success('采集器设置已保存')
|
||||
await fetchSettings()
|
||||
} catch (error: unknown) {
|
||||
const err = error as { response?: { data?: { detail?: string; message?: string } } }
|
||||
message.error(err.response?.data?.message || err.response?.data?.detail || '采集器设置保存失败')
|
||||
} finally {
|
||||
setSavingCollectorConfig(false)
|
||||
}
|
||||
}
|
||||
|
||||
const loadCredentialGuide = async (provider: string, open = true) => {
|
||||
try {
|
||||
setCredentialGuideLoading(true)
|
||||
const response = await axios.get(`/api/v1/settings/credential-guides/${provider}`)
|
||||
setCredentialGuide(response.data.guide)
|
||||
if (open) setCredentialGuideOpen(true)
|
||||
} catch {
|
||||
message.error('获取凭证教程失败')
|
||||
} finally {
|
||||
setCredentialGuideLoading(false)
|
||||
}
|
||||
}
|
||||
|
||||
const testBarentsWatchCredentials = async () => {
|
||||
try {
|
||||
const values = integrationForm.getFieldsValue(true)
|
||||
setTestingCredentialProvider('barentswatch')
|
||||
const payload = {
|
||||
...values.barentswatch,
|
||||
client_secret:
|
||||
values.barentswatch?.client_secret === integrations?.barentswatch.client_secret.preview
|
||||
? ''
|
||||
: values.barentswatch?.client_secret,
|
||||
}
|
||||
const response = await axios.post('/api/v1/settings/integrations/barentswatch/connect', payload)
|
||||
if (response.data.success) {
|
||||
setCollectorHealthStatus((prev) => ({
|
||||
...prev,
|
||||
barentswatch_vessels: { ok: true, message: response.data.message || '可用' },
|
||||
}))
|
||||
message.success(response.data.message || 'BarentsWatch AIS 连接成功')
|
||||
} else {
|
||||
setCollectorHealthStatus((prev) => ({
|
||||
...prev,
|
||||
barentswatch_vessels: { ok: false, message: response.data.message || '不可用' },
|
||||
}))
|
||||
message.error(response.data.message || 'BarentsWatch AIS 连接失败')
|
||||
await loadCredentialGuide('barentswatch', true)
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
const err = error as { response?: { data?: { detail?: string } } }
|
||||
const errorMessage = err.response?.data?.detail || 'BarentsWatch AIS 连接失败'
|
||||
setCollectorHealthStatus((prev) => ({
|
||||
...prev,
|
||||
barentswatch_vessels: { ok: false, message: errorMessage },
|
||||
}))
|
||||
message.error(errorMessage)
|
||||
await loadCredentialGuide('barentswatch', true)
|
||||
} finally {
|
||||
setTestingCredentialProvider(null)
|
||||
}
|
||||
}
|
||||
|
||||
const testSelectedCollectorConnectivity = async () => {
|
||||
if (!selectedCollector) return
|
||||
if (selectedCollector.source === 'barentswatch_vessels') {
|
||||
await testBarentsWatchCredentials()
|
||||
return
|
||||
}
|
||||
|
||||
try {
|
||||
const values = collectorConfigForm.getFieldsValue(true)
|
||||
setTestingCredentialProvider(selectedCollector.source)
|
||||
const response = await axios.post('/api/v1/datasources/configs/builtin/connect', {
|
||||
name: selectedCollector.source,
|
||||
description: `内置采集器连接验证:${selectedCollector.name}`,
|
||||
source_type: 'http',
|
||||
endpoint: values.endpoint || selectedCollectorConfig?.endpoint || selectedCollectorConfig?.default_url || '',
|
||||
auth_type: 'none',
|
||||
auth_config: {},
|
||||
headers: headersListToMap(values.headers),
|
||||
config: values.config || {},
|
||||
})
|
||||
if (response.data.success || response.data.connected) {
|
||||
setCollectorHealthStatus((prev) => ({
|
||||
...prev,
|
||||
[selectedCollector.source]: { ok: true, message: response.data.message || '可用' },
|
||||
}))
|
||||
message.success(response.data.message || '采集器健康检查通过')
|
||||
} else {
|
||||
setCollectorHealthStatus((prev) => ({
|
||||
...prev,
|
||||
[selectedCollector.source]: { ok: false, message: response.data.message || '不可用' },
|
||||
}))
|
||||
message.error(response.data.message || '采集器健康检查失败')
|
||||
}
|
||||
} catch (error: unknown) {
|
||||
const err = error as { response?: { data?: { detail?: string; message?: string } } }
|
||||
const errorMessage = err.response?.data?.message || err.response?.data?.detail || '采集器健康检查失败'
|
||||
setCollectorHealthStatus((prev) => ({
|
||||
...prev,
|
||||
[selectedCollector.source]: { ok: false, message: errorMessage },
|
||||
}))
|
||||
message.error(errorMessage)
|
||||
} finally {
|
||||
setTestingCredentialProvider(null)
|
||||
}
|
||||
}
|
||||
|
||||
const generateCredentialGuide = async () => {
|
||||
if (!credentialGuide) return
|
||||
try {
|
||||
setCredentialGuideGenerating(true)
|
||||
const response = await axios.post(`/api/v1/settings/credential-guides/${credentialGuide.provider}/generate`)
|
||||
setCredentialGuide(response.data.guide)
|
||||
message.success('已生成新的凭证教程')
|
||||
} catch {
|
||||
message.error('生成教程失败,请检查 AI 配置')
|
||||
} finally {
|
||||
setCredentialGuideGenerating(false)
|
||||
}
|
||||
}
|
||||
|
||||
const resetCredentialGuide = async () => {
|
||||
if (!credentialGuide) return
|
||||
try {
|
||||
setCredentialGuideResetting(true)
|
||||
const response = await axios.post(`/api/v1/settings/credential-guides/${credentialGuide.provider}/reset`)
|
||||
setCredentialGuide(response.data.guide)
|
||||
message.success('已恢复默认教程')
|
||||
} catch {
|
||||
message.error('重置教程失败')
|
||||
} finally {
|
||||
setCredentialGuideResetting(false)
|
||||
}
|
||||
}
|
||||
|
||||
const applyAiProviderPreset = (preset: AIProviderPreset) => {
|
||||
integrationForm.setFieldsValue({
|
||||
ai_provider: {
|
||||
@@ -1025,65 +1335,153 @@ function Settings() {
|
||||
},
|
||||
{
|
||||
key: 'collector_credentials',
|
||||
label: '采集器凭证',
|
||||
label: '采集器设置',
|
||||
forceRender: true,
|
||||
children: (
|
||||
<SettingsPanel loading={loading}>
|
||||
<Form form={integrationForm} layout="vertical" onFinish={saveIntegrations}>
|
||||
<Card size="small" title="需要凭证的采集器" style={{ marginBottom: 16 }}>
|
||||
<Space size={[6, 6]} wrap>
|
||||
{credentialCollectors.length ? credentialCollectors.map((collector) => (
|
||||
<Tag
|
||||
key={collector.source}
|
||||
color={collector.credential_status === 'supported' ? 'blue' : 'orange'}
|
||||
>
|
||||
{collector.display_name || collector.name}
|
||||
{collector.credential_status === 'supported' ? ' · 已支持配置' : ' · 待接入'}
|
||||
</Tag>
|
||||
)) : (
|
||||
<Text type="secondary">当前没有需要凭证的内置采集器。</Text>
|
||||
)}
|
||||
</Space>
|
||||
</Card>
|
||||
<Space direction="vertical" size={16} style={{ width: '100%' }}>
|
||||
<Card
|
||||
size="small"
|
||||
title={<Space><ApiOutlined />BarentsWatch AIS</Space>}
|
||||
title="选择采集器"
|
||||
>
|
||||
<Form.Item name={['barentswatch', 'endpoint']} label="AIS Endpoint">
|
||||
<Input placeholder="https://live.ais.barentswatch.no/v1/latest/combined" />
|
||||
</Form.Item>
|
||||
<Form.Item name={['barentswatch', 'client_id']} label="Client ID">
|
||||
<Input autoComplete="off" />
|
||||
</Form.Item>
|
||||
<Form.Item label="Client Secret">
|
||||
<Space direction="vertical" style={{ width: '100%' }}>
|
||||
<Space>
|
||||
<Tag color={integrations?.barentswatch.client_secret.configured ? 'green' : 'default'}>
|
||||
{integrations?.barentswatch.client_secret.configured
|
||||
? `已配置 ${integrations.barentswatch.client_secret.preview}`
|
||||
: '未配置'}
|
||||
</Tag>
|
||||
<Text type="secondary">留空表示保留现有 secret。</Text>
|
||||
</Space>
|
||||
<Form.Item name={['barentswatch', 'client_secret']} noStyle>
|
||||
<Input.Password autoComplete="new-password" placeholder="输入新 client secret" />
|
||||
</Form.Item>
|
||||
<Space.Compact style={{ width: '100%' }}>
|
||||
<Select
|
||||
showSearch
|
||||
value={selectedCollectorSource}
|
||||
style={{ width: '100%' }}
|
||||
optionFilterProp="label"
|
||||
onChange={setSelectedCollectorSource}
|
||||
options={collectors.map((collector) => ({
|
||||
value: collector.source,
|
||||
label: `${collector.display_name || collector.name} · ${collector.source}`,
|
||||
}))}
|
||||
/>
|
||||
<Tooltip title="健康检查">
|
||||
<Button
|
||||
icon={<PlugConnectIcon />}
|
||||
loading={testingCredentialProvider === (selectedCollector?.credential_provider || selectedCollector?.source)}
|
||||
disabled={!selectedCollector}
|
||||
onClick={() => { void testSelectedCollectorConnectivity() }}
|
||||
/>
|
||||
</Tooltip>
|
||||
</Space.Compact>
|
||||
{selectedCollector ? (
|
||||
<Space size={[6, 6]} wrap style={{ marginTop: 12 }}>
|
||||
<Tag color={selectedCollector.requires_credentials ? 'orange' : 'green'}>
|
||||
{selectedCollector.requires_credentials ? '需要凭证' : '无需凭证'}
|
||||
</Tag>
|
||||
<Tag>{selectedCollector.module}</Tag>
|
||||
<Tag color={selectedCollector.is_active ? 'success' : 'default'}>
|
||||
{selectedCollector.is_active ? '启用' : '禁用'}
|
||||
</Tag>
|
||||
{selectedCollectorHealth ? (
|
||||
<Tooltip title={selectedCollectorHealth.message}>
|
||||
<Tag color={selectedCollectorHealth.ok ? 'success' : 'error'}>
|
||||
{selectedCollectorHealth.ok ? '可用' : '不可用'}
|
||||
</Tag>
|
||||
</Tooltip>
|
||||
) : (
|
||||
<Tag>未检查</Tag>
|
||||
)}
|
||||
{selectedCollectorConfig?.is_overridden ? <Tag color="blue">已覆盖 endpoint</Tag> : null}
|
||||
</Space>
|
||||
</Form.Item>
|
||||
<Form.Item name={['barentswatch', 'clear_client_secret']} valuePropName="checked">
|
||||
<Checkbox>清除当前 BarentsWatch client secret</Checkbox>
|
||||
</Form.Item>
|
||||
) : null}
|
||||
</Card>
|
||||
|
||||
{selectedCollector?.requires_credentials && selectedCollector.source !== 'barentswatch_vessels' ? (
|
||||
<Alert
|
||||
showIcon
|
||||
type="warning"
|
||||
message="该采集器需要凭证"
|
||||
description={selectedCollector.credential_status === 'supported'
|
||||
? '该凭证类型已支持,但当前页面还没有专用表单。'
|
||||
: '该凭证配置入口待接入。'}
|
||||
/>
|
||||
) : null}
|
||||
|
||||
{selectedCollector?.source === 'barentswatch_vessels' ? (
|
||||
<Card
|
||||
size="small"
|
||||
title={<Space><ApiOutlined />BarentsWatch AIS 凭证</Space>}
|
||||
extra={(
|
||||
<Space>
|
||||
<Tooltip title="查看凭证获取教程">
|
||||
<Button
|
||||
size="small"
|
||||
icon={<BookOutlined />}
|
||||
onClick={() => { void loadCredentialGuide('barentswatch', true) }}
|
||||
/>
|
||||
</Tooltip>
|
||||
</Space>
|
||||
)}
|
||||
>
|
||||
<Form form={integrationForm} layout="vertical">
|
||||
<Form.Item name={['barentswatch', 'client_id']} label="Client ID">
|
||||
<Input autoComplete="off" />
|
||||
</Form.Item>
|
||||
<Form.Item name={['barentswatch', 'client_secret']} label="Client Secret">
|
||||
<Input.Password
|
||||
autoComplete="new-password"
|
||||
placeholder="输入 client secret"
|
||||
onFocus={() => {
|
||||
const preview = integrations?.barentswatch.client_secret.preview
|
||||
if (preview && integrationForm.getFieldValue(['barentswatch', 'client_secret']) === preview) {
|
||||
integrationForm.setFieldValue(['barentswatch', 'client_secret'], '')
|
||||
}
|
||||
}}
|
||||
/>
|
||||
</Form.Item>
|
||||
</Form>
|
||||
</Card>
|
||||
) : null}
|
||||
|
||||
<Card size="small" title="基础配置">
|
||||
<Form form={collectorConfigForm} layout="vertical">
|
||||
<Form.Item name="endpoint" label="Endpoint" rules={[{ required: true, message: '请输入 Endpoint' }]}>
|
||||
<Input placeholder={selectedCollectorConfig?.default_url || 'https://api.example.com'} />
|
||||
</Form.Item>
|
||||
<Form.Item label="默认 Endpoint">
|
||||
<Input value={selectedCollectorConfig?.default_url || '-'} disabled />
|
||||
</Form.Item>
|
||||
<Form.List name="headers">
|
||||
{(fields, { add, remove }) => (
|
||||
<Form.Item label="请求头">
|
||||
<Space direction="vertical" style={{ width: '100%' }}>
|
||||
{fields.map(({ key, name, ...restField }) => (
|
||||
<Space key={key} style={{ display: 'flex' }} align="baseline">
|
||||
<Form.Item {...restField} name={[name, 'key']} noStyle>
|
||||
<Input placeholder="Header" />
|
||||
</Form.Item>
|
||||
<Form.Item {...restField} name={[name, 'value']} noStyle>
|
||||
<Input placeholder="Value" />
|
||||
</Form.Item>
|
||||
<Button type="link" danger onClick={() => remove(name)}>删除</Button>
|
||||
</Space>
|
||||
))}
|
||||
<Button type="dashed" onClick={() => add()} block>添加请求头</Button>
|
||||
</Space>
|
||||
</Form.Item>
|
||||
)}
|
||||
</Form.List>
|
||||
<div style={{ display: 'grid', gridTemplateColumns: '1fr 1fr', gap: '0 16px' }}>
|
||||
<Form.Item name={['config', 'timeout']} label="超时(秒)">
|
||||
<InputNumber min={1} max={300} style={{ width: '100%' }} />
|
||||
</Form.Item>
|
||||
<Form.Item name={['config', 'retry']} label="重试次数">
|
||||
<InputNumber min={0} max={10} style={{ width: '100%' }} />
|
||||
</Form.Item>
|
||||
</div>
|
||||
</Form>
|
||||
</Card>
|
||||
|
||||
<Button
|
||||
type="primary"
|
||||
htmlType="submit"
|
||||
loading={savingIntegrations}
|
||||
style={{ marginTop: 16 }}
|
||||
loading={savingCollectorConfig || savingIntegrations}
|
||||
onClick={() => { void saveSelectedCollectorSettings() }}
|
||||
>
|
||||
保存采集器凭证
|
||||
保存采集器设置
|
||||
</Button>
|
||||
</Form>
|
||||
</Space>
|
||||
</SettingsPanel>
|
||||
),
|
||||
},
|
||||
@@ -1133,6 +1531,50 @@ function Settings() {
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
<Modal
|
||||
title={credentialGuide?.title || '凭证教程'}
|
||||
open={credentialGuideOpen}
|
||||
onCancel={() => setCredentialGuideOpen(false)}
|
||||
footer={[
|
||||
<Button
|
||||
key="reset"
|
||||
icon={<ReloadOutlined />}
|
||||
loading={credentialGuideResetting}
|
||||
onClick={() => { void resetCredentialGuide() }}
|
||||
>
|
||||
重置
|
||||
</Button>,
|
||||
<Button
|
||||
key="generate"
|
||||
icon={<RobotOutlined />}
|
||||
loading={credentialGuideGenerating}
|
||||
onClick={() => { void generateCredentialGuide() }}
|
||||
>
|
||||
教程不好用
|
||||
</Button>,
|
||||
<Button key="close" type="primary" onClick={() => setCredentialGuideOpen(false)}>
|
||||
关闭
|
||||
</Button>,
|
||||
]}
|
||||
width={760}
|
||||
centered
|
||||
>
|
||||
<Space direction="vertical" size={12} style={{ width: '100%' }}>
|
||||
{credentialGuide?.source === 'ai' ? (
|
||||
<Alert showIcon type="success" message="当前显示 AI 生成教程,可随时重置为默认教程。" />
|
||||
) : (
|
||||
<Alert showIcon type="info" message="当前显示默认教程;如果不适用,可以点击右下角“教程不好用”让 AI 重新生成。" />
|
||||
)}
|
||||
<Card loading={credentialGuideLoading} size="small" style={{ maxHeight: 520, overflow: 'auto' }}>
|
||||
<MarkdownRenderer markdown={credentialGuide?.markdown || ''} />
|
||||
</Card>
|
||||
{credentialGuide?.prompt ? (
|
||||
<Text type="secondary" style={{ fontSize: 12 }}>
|
||||
生成提示词:{credentialGuide.prompt}
|
||||
</Text>
|
||||
) : null}
|
||||
</Space>
|
||||
</Modal>
|
||||
</AppLayout>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,16 +1,26 @@
|
||||
import { useEffect, useState } from 'react'
|
||||
import { Table, Tag, Card, Row, Col, Statistic, Button } from 'antd'
|
||||
import { Table, Tag, Card, Row, Col, Statistic, Button, Tooltip } from 'antd'
|
||||
import { ReloadOutlined, CheckCircleOutlined, CloseCircleOutlined, SyncOutlined } from '@ant-design/icons'
|
||||
import { useAuthStore } from '../../stores/auth'
|
||||
import AppLayout from '../../components/AppLayout/AppLayout'
|
||||
import TableScrollRegion from '../../components/Scrollbar/TableScrollRegion'
|
||||
import { formatDateTimeZhCN } from '../../utils/datetime'
|
||||
import { getPhaseDisplay, getPhaseSummary } from '../../utils/phaseProgress'
|
||||
|
||||
interface Task {
|
||||
id: number
|
||||
collector: string
|
||||
collector?: string
|
||||
datasource_name?: string
|
||||
status: 'success' | 'failed' | 'running' | 'pending'
|
||||
phase?: string | null
|
||||
phase_progress?: number | null
|
||||
phase_message?: string | null
|
||||
phase_current?: number | null
|
||||
phase_total?: number | null
|
||||
phase_unit?: string | null
|
||||
records_processed: number
|
||||
total_records?: number | null
|
||||
progress?: number | null
|
||||
started_at: string
|
||||
completed_at: string
|
||||
duration_seconds: number
|
||||
@@ -53,7 +63,20 @@ function Tasks() {
|
||||
title: '收集器',
|
||||
dataIndex: 'collector',
|
||||
key: 'collector',
|
||||
render: (c: string) => <Tag color="blue">{c}</Tag>,
|
||||
render: (_: string, task: Task) => <Tag color="blue">{task.collector || task.datasource_name || '-'}</Tag>,
|
||||
},
|
||||
{
|
||||
title: '阶段',
|
||||
dataIndex: 'phase',
|
||||
key: 'phase',
|
||||
render: (_: string, task: Task) => {
|
||||
const detail = task.phase ? getPhaseDisplay(task) : null
|
||||
return detail ? (
|
||||
<Tooltip title={detail}>
|
||||
<Tag color={task.status === 'running' ? 'processing' : 'default'}>{getPhaseSummary(task)}</Tag>
|
||||
</Tooltip>
|
||||
) : '-'
|
||||
},
|
||||
},
|
||||
{
|
||||
title: '状态',
|
||||
|
||||
61
frontend/src/utils/phaseProgress.ts
Normal file
61
frontend/src/utils/phaseProgress.ts
Normal file
@@ -0,0 +1,61 @@
|
||||
export type PhaseProgressLike = {
|
||||
phase?: string | null
|
||||
phase_progress?: number | null
|
||||
phase_message?: string | null
|
||||
phase_current?: number | null
|
||||
phase_total?: number | null
|
||||
phase_unit?: string | null
|
||||
}
|
||||
|
||||
const BYTE_UNIT_BASE = 1024
|
||||
const BYTE_UNITS = ['B', 'KB', 'MB', 'GB'] as const
|
||||
|
||||
export function formatBytes(value: number): string {
|
||||
if (!Number.isFinite(value) || value <= 0) return '0 B'
|
||||
let size = value
|
||||
let unitIndex = 0
|
||||
while (size >= BYTE_UNIT_BASE && unitIndex < BYTE_UNITS.length - 1) {
|
||||
size /= BYTE_UNIT_BASE
|
||||
unitIndex += 1
|
||||
}
|
||||
const digits = unitIndex === 0 ? 0 : size >= 10 ? 1 : 2
|
||||
return `${size.toFixed(digits)} ${BYTE_UNITS[unitIndex]}`
|
||||
}
|
||||
|
||||
export function formatPhaseMetric(source: Pick<PhaseProgressLike, 'phase_current' | 'phase_total' | 'phase_unit'>): string | null {
|
||||
const current = source.phase_current
|
||||
const total = source.phase_total
|
||||
if (typeof current !== 'number') return null
|
||||
if (source.phase_unit === 'bytes') {
|
||||
return typeof total === 'number' && total > 0
|
||||
? `${formatBytes(current)} / ${formatBytes(total)}`
|
||||
: formatBytes(current)
|
||||
}
|
||||
if (typeof total === 'number' && total > 0) {
|
||||
return `${current} / ${total}${source.phase_unit ? ` ${source.phase_unit}` : ''}`
|
||||
}
|
||||
return `${current}${source.phase_unit ? ` ${source.phase_unit}` : ''}`
|
||||
}
|
||||
|
||||
export function getPhaseDisplay(source: PhaseProgressLike, fallback = 'running'): string {
|
||||
const phase = source.phase || fallback
|
||||
const parts = [phase]
|
||||
if (typeof source.phase_progress === 'number') {
|
||||
parts.push(`${Math.round(source.phase_progress)}%`)
|
||||
}
|
||||
if (source.phase_message) {
|
||||
parts.push(source.phase_message)
|
||||
}
|
||||
const metric = formatPhaseMetric(source)
|
||||
if (metric) {
|
||||
parts.push(metric)
|
||||
}
|
||||
return parts.join(' · ')
|
||||
}
|
||||
|
||||
export function getPhaseSummary(source: PhaseProgressLike, fallback = 'running'): string {
|
||||
const phase = source.phase || fallback
|
||||
return typeof source.phase_progress === 'number'
|
||||
? `${phase} ${Math.round(source.phase_progress)}%`
|
||||
: phase
|
||||
}
|
||||
82
planet.sh
82
planet.sh
@@ -6,6 +6,48 @@ SCRIPT_PATH="${(%):-%N}"
|
||||
SCRIPT_DIR="$(cd "$(dirname "$SCRIPT_PATH")" && pwd)"
|
||||
cd "$SCRIPT_DIR"
|
||||
|
||||
load_user_zshrc_exported_env() {
|
||||
local load_mode="${PLANET_LOAD_ZSHRC_ENV:-1}"
|
||||
[ "$load_mode" != "0" ] || return 0
|
||||
[ "$load_mode" != "false" ] || return 0
|
||||
[ -f "$HOME/.zshrc" ] || return 0
|
||||
|
||||
local env_line env_name env_value
|
||||
if [ "$load_mode" = "source" ]; then
|
||||
command -v zsh >/dev/null 2>&1 || return 0
|
||||
while IFS= read -r env_line; do
|
||||
env_name="${env_line%%=*}"
|
||||
env_value="${env_line#*=}"
|
||||
case "$env_name" in
|
||||
AI_*|SERVICE_NAME|SERVICE_VERSION|PYTHON_IMAGE|UV_IMAGE|DOCKER_BUILDKIT|COMPOSE_DOCKER_CLI_BUILD|HTTP_PROXY|HTTPS_PROXY|NO_PROXY|http_proxy|https_proxy|no_proxy)
|
||||
export "${env_name}=${env_value}"
|
||||
;;
|
||||
esac
|
||||
done < <(zsh -fc 'source ~/.zshrc >/dev/null 2>&1; env' 2>/dev/null || true)
|
||||
return 0
|
||||
fi
|
||||
|
||||
while IFS= read -r env_line; do
|
||||
env_line="$(printf "%s" "$env_line" | sed -E 's/^[[:space:]]+//;s/[[:space:]]+$//')"
|
||||
case "$env_line" in
|
||||
""|\#*) continue ;;
|
||||
export\ *) env_line="${env_line#export }" ;;
|
||||
esac
|
||||
env_name="${env_line%%=*}"
|
||||
env_value="${env_line#*=}"
|
||||
[ "$env_name" != "$env_value" ] || continue
|
||||
env_name="$(printf "%s" "$env_name" | sed -E 's/^[[:space:]]+//;s/[[:space:]]+$//')"
|
||||
env_value="$(printf "%s" "$env_value" | sed -E 's/^[[:space:]]+//;s/[[:space:]]+$//;s/^"//;s/"$//;s/^'\''//;s/'\''$//')"
|
||||
case "$env_name" in
|
||||
AI_*|SERVICE_NAME|SERVICE_VERSION|PYTHON_IMAGE|UV_IMAGE|DOCKER_BUILDKIT|COMPOSE_DOCKER_CLI_BUILD|HTTP_PROXY|HTTPS_PROXY|NO_PROXY|http_proxy|https_proxy|no_proxy)
|
||||
export "${env_name}=${env_value}"
|
||||
;;
|
||||
esac
|
||||
done < "$HOME/.zshrc"
|
||||
}
|
||||
|
||||
load_user_zshrc_exported_env
|
||||
|
||||
RED='\033[38;5;203m'
|
||||
GREEN='\033[38;5;114m'
|
||||
YELLOW='\033[38;5;221m'
|
||||
@@ -68,7 +110,23 @@ AI_PROVIDER_BUILD_STAMP_FILE="$HOME/.cache/planet/aiprovider_build.sha256"
|
||||
AI_PROVIDER_BUILD_LOG_FILE="/tmp/planet_aiprovider_build.log"
|
||||
AI_PROVIDER_IMAGE_NAME="${AI_PROVIDER_IMAGE_NAME:-planet_aiprovider:latest}"
|
||||
AI_PROVIDER_CONTAINER_NAME="${AI_PROVIDER_CONTAINER_NAME:-planet_aiprovider}"
|
||||
PLANET_AI_PROVIDER_RUNTIME_ENV_FILE="${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-/tmp/planet_aiprovider_runtime.env}"
|
||||
AI_PROVIDER_RECREATE_REQUIRED=0
|
||||
AI_PROVIDER_RUNTIME_ENV_NAMES=(
|
||||
SERVICE_NAME
|
||||
SERVICE_VERSION
|
||||
AI_PROVIDER
|
||||
AI_PROVIDER_API
|
||||
AI_BASE_URL
|
||||
AI_API_KEY
|
||||
AI_MODEL
|
||||
AI_TIMEOUT_SECONDS
|
||||
AI_HTTP_RETRY_ATTEMPTS
|
||||
AI_MAX_TOKENS
|
||||
AI_ANTHROPIC_VERSION
|
||||
AI_ANALYSIS_SYSTEM_PROMPT
|
||||
AI_PROVIDER_SERVICE_TOKEN
|
||||
)
|
||||
|
||||
prepend_path_once() {
|
||||
local path_entry="$1"
|
||||
@@ -590,6 +648,8 @@ compute_ai_provider_build_fingerprint() {
|
||||
find aiprovider \
|
||||
-type f \
|
||||
! -path '*/__pycache__/*' \
|
||||
! -name '.env' \
|
||||
! -name '.env.*' \
|
||||
! -name '*.pyc' \
|
||||
! -name '*.pyo' \
|
||||
| LC_ALL=C sort \
|
||||
@@ -633,6 +693,9 @@ run_ai_provider_container_manually() {
|
||||
if [ -f "$SCRIPT_DIR/aiprovider/.env" ]; then
|
||||
env_file_args=(--env-file "$SCRIPT_DIR/aiprovider/.env")
|
||||
fi
|
||||
if [ -s "$PLANET_AI_PROVIDER_RUNTIME_ENV_FILE" ]; then
|
||||
env_file_args+=(--env-file "$PLANET_AI_PROVIDER_RUNTIME_ENV_FILE")
|
||||
fi
|
||||
|
||||
docker run -d \
|
||||
--name "$AI_PROVIDER_CONTAINER_NAME" \
|
||||
@@ -641,6 +704,22 @@ run_ai_provider_container_manually() {
|
||||
"$AI_PROVIDER_IMAGE_NAME" >/dev/null
|
||||
}
|
||||
|
||||
write_ai_provider_runtime_env_file() {
|
||||
local env_name env_value
|
||||
|
||||
: > "$PLANET_AI_PROVIDER_RUNTIME_ENV_FILE"
|
||||
chmod 600 "$PLANET_AI_PROVIDER_RUNTIME_ENV_FILE" 2>/dev/null || true
|
||||
|
||||
for env_name in "${AI_PROVIDER_RUNTIME_ENV_NAMES[@]}"; do
|
||||
if [ -n "${(P)env_name+x}" ]; then
|
||||
env_value="${(P)env_name}"
|
||||
printf "%s=%s\n" "$env_name" "$env_value" >> "$PLANET_AI_PROVIDER_RUNTIME_ENV_FILE"
|
||||
fi
|
||||
done
|
||||
|
||||
export PLANET_AI_PROVIDER_RUNTIME_ENV_FILE
|
||||
}
|
||||
|
||||
recreate_ai_provider_container() {
|
||||
local ai_provider_port="${1:-$DEFAULT_AI_PROVIDER_PORT}"
|
||||
|
||||
@@ -1156,6 +1235,7 @@ start_ai_provider_service() {
|
||||
local retry=1
|
||||
set_wait_detail "启动 AI Provider"
|
||||
|
||||
write_ai_provider_runtime_env_file
|
||||
ensure_ai_provider_image_current
|
||||
|
||||
while [ "$retry" -le "$AI_PROVIDER_START_MAX_RETRIES" ]; do
|
||||
@@ -1279,7 +1359,7 @@ start_backend_service() {
|
||||
if ai_provider_service_healthy "$ai_provider_port"; then
|
||||
log_note "AI Provider 已健康,复用现有服务,跳过启动/重建"
|
||||
else
|
||||
log_note "AI Provider 当前不健康,先执行托底启动"
|
||||
log_note "AI Provider 未就绪,正在启动本地代理服务"
|
||||
start_ai_provider_service "$ai_provider_port"
|
||||
fi
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "planet"
|
||||
version = "0.43.0"
|
||||
version = "0.45.0"
|
||||
description = "智能星球计划 - 态势感知系统"
|
||||
requires-python = ">=3.14"
|
||||
dependencies = [
|
||||
|
||||
Reference in New Issue
Block a user