Compare commits

...

4 Commits

Author SHA1 Message Date
linkong
06aca980d0 release: bump version to 0.68.1
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
release / images (push) Has been cancelled
ci / delivery (push) Has been cancelled
2026-05-28 18:26:15 +08:00
linkong
f3f1ceb833 release: bump version to 0.68.0
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / delivery (push) Has been cancelled
release / images (push) Has been cancelled
2026-05-28 17:10:05 +08:00
rayd1o
b18ffa0b0a release: bump version to 0.67.0
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / delivery (push) Has been cancelled
release / images (push) Has been cancelled
2026-05-27 13:50:16 +08:00
d15a9d488a release: bump version to 0.66.3
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / delivery (push) Has been cancelled
release / images (push) Has been cancelled
2026-05-26 17:26:19 +08:00
66 changed files with 3382 additions and 781 deletions

4
.gitignore vendored
View File

@@ -28,8 +28,8 @@ dist/
downloads/
eggs/
.eggs/
lib/
lib64/
/lib/
/lib64/
parts/
sdist/
var/

15
TODO.md
View File

@@ -11,10 +11,8 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
- [ ] Import authoritative China POV / coastline / claim-line source packages through the three standard Earth boundary source collectors, then rebuild a versioned PMTiles artifact so highest zoom `8-10` preserves trusted source geometry instead of seed data.
- [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, Kosovo, Gaza, and the official dashed maritime claim line before implementing final visual changes.
- [ ] Earth high-resolution basemap tiles: implement the viewport-loaded imagery layer described in [Earth High Resolution Basemap Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-resolution-basemap-tiles-plan.md), using high-precision coastline as the alignment reference instead of replacing the globe with one huge texture.
- [ ] Presentation controller ownership: replace the singleton card fallback in [presentation-controller.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/presentation-controller.js) with a presentation/card token check before BGP/News migrate onto the shared controller, so connectors only attach to their owning card.
- [ ] BGP frontend maintainability: split [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) by responsibility into data loading, marker rendering, overlays, and animation once the current interaction behavior is stable.
- [ ] Optional BGP marker experiment: evaluate HTML markers for BGP incident/collector points if WebGL marker density or fixed screen-size clickability becomes a real blocker.
- [ ] Earth news cruise: connect Earth news to the generic cruise queue via a news adapter rather than coupling news-specific sequencing into `main.js`.
## Compute Centers And Location
@@ -45,13 +43,6 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
- [ ] Compatibility schema: cover adapter type, base URL pattern, auth header, thinking/reasoning defaults, stream path, tool-call capability, multimodal capability, and provider-specific request patches.
- [ ] BGP geography fallback: evaluate `inetnum` / `inet6num` whois as a finer fallback layer after `prefix_geography`, `OpenGeoFeed`, and RIR delegated data.
## Platform
- [ ] Earth preferences scope: keep current device-local Earth preferences in `localStorage`; only design backend user preferences if account-level synchronization becomes a real product requirement.
- [ ] System logs: finish a usable Planet log viewing flow that covers backend, frontend, AI Provider, and collector/task logs, with filtering and tailing.
- [ ] Console UI modernization: gradually replace Ant Design with Planet-owned components and a consistent Tabler Icons based icon system.
- [ ] Earth live sync: design a unified realtime invalidation path for summary/BGP/satellite updates if polling and current WebSocket channels become insufficient.
## Archive
Archived items stay here so old context is not lost. Completed items remain checked; obsolete, invalid, or superseded items stay unchecked and include the reason.
@@ -75,6 +66,11 @@ Archived items stay here so old context is not lost. Completed items remain chec
- [x] Added OpenGeoFeed as a high-quality prefix geography override source.
- [x] Made RIR delegated data a prefix geography fallback rather than the primary source.
- [x] Added route leak and path instability / flap detectors after the activity layer work.
- [x] Console UI modernization. Admin is now the only console, legacy Ant Design / Admin Next code paths and dependencies have been removed, and current console UI uses Planet-owned components.
- [x] Earth news cruise adapter. News cruise now uses `news-cruise-adapter.js` and is wired from `main.js` instead of keeping news-specific sequencing directly in the main Earth loop.
- [x] Presentation controller ownership. `PresentationController` now guards async ownership through active request identity checks, and current callers pass per-request card targets so stale connector/card work cannot overwrite the active presentation.
- [x] Earth live sync. Database writes now flow through `earth_data_change_events`, `earth_db_change_listener`, layer adapters, cache invalidation, and the `earth_updates` WebSocket channel; the Earth frontend debounces updates and refreshes BGP, cables, compute centers, satellites, vessels, news, and interactables by layer.
- [x] System logs. Log sources now normalize into `LogEvent`, Admin supports snapshot filtering plus WebSocket tail/follow, task/detail views deep-link into prefiltered logs, and Admin runtime errors report through the `admin-client` log source.
### Obsolete Or Superseded
@@ -85,3 +81,4 @@ Archived items stay here so old context is not lost. Completed items remain chec
- [ ] Earth surface material overlay for boundary calibration. Superseded by the high-precision boundary tile plan; future work must use source-faithful boundary/coastline data rather than overlay calibration against the coarse base map.
- [ ] Hardcoded Earth news source extraction as a standalone task. Superseded by the broader Earth news source configuration and collector plans.
- [ ] Country-level compute-center fallback placement as a standalone task. Superseded by the shared location pipeline and registry/manual-review backlog.
- [ ] Earth preferences backend sync scope. Superseded by the current product decision to keep Earth preferences device-local in `localStorage` until account-level synchronization becomes a real requirement.

View File

@@ -1 +1 @@
0.66.2
0.68.1

View File

@@ -19,7 +19,6 @@ from app.models.datasource_config import DataSourceConfig
from app.models.task import CollectionTask
from app.models.user import User
from app.models.vessel import AISRawObservation
from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
from app.services.scheduler import (
sync_datasource_job,
)
@@ -165,6 +164,8 @@ async def _load_latest_tasks(
async def _load_collected_record_counts(
db: AsyncSession,
sources: list[str],
*,
exact_vessel_counts: bool = False,
) -> dict[str, int]:
if not sources:
return {}
@@ -185,14 +186,46 @@ async def _load_collected_record_counts(
or "ais" in source
]
if vessel_sources:
raw_result = await db.execute(
select(AISRawObservation.source, func.count(AISRawObservation.id))
.where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
.where(AISRawObservation.source.in_(vessel_sources))
.group_by(AISRawObservation.source)
if exact_vessel_counts:
exact_result = await db.execute(
select(AISRawObservation.source, func.count(AISRawObservation.id))
.where(AISRawObservation.source.in_(vessel_sources))
.group_by(AISRawObservation.source)
)
for source, count in exact_result.all():
counts[source] = max(counts.get(source, 0), int(count or 0))
return counts
# AIS raw observations can be tens of millions of rows. Use planner
# statistics for the datasource list instead of blocking page load on
# source-level count(*) scans.
stats_result = await db.execute(
text(
"""
SELECT
COALESCE(pg_class.reltuples, 0)::bigint AS total_rows,
pg_stats.most_common_vals::text AS source_values,
pg_stats.most_common_freqs::text AS source_freqs
FROM pg_class
LEFT JOIN pg_stats
ON pg_stats.schemaname = 'public'
AND pg_stats.tablename = 'ais_raw_observations'
AND pg_stats.attname = 'source'
WHERE pg_class.relname = 'ais_raw_observations'
LIMIT 1
"""
)
)
for source, count in raw_result.all():
counts[source] = max(counts.get(source, 0), int(count or 0))
stats = stats_result.mappings().first()
if stats:
total_rows = int(stats["total_rows"] or 0)
values = str(stats["source_values"] or "").strip("{}")
freqs = str(stats["source_freqs"] or "").strip("{}")
source_values = [value.strip('"') for value in values.split(",") if value]
source_freqs = [float(value) for value in freqs.split(",") if value]
for source, freq in zip(source_values, source_freqs):
if source in vessel_sources:
counts[source] = max(counts.get(source, 0), int(round(total_rows * freq)))
return counts
@@ -929,7 +962,7 @@ async def get_datasource_row(
[datasource],
include_endpoint=include_endpoint,
)
record_counts = await _load_collected_record_counts(db, [datasource.source])
record_counts = await _load_collected_record_counts(db, [datasource.source], exact_vessel_counts=True)
return {
"data": serialize_datasource_row(
datasource,

View File

@@ -1,7 +1,6 @@
from __future__ import annotations
import os
import json
import subprocess
import sys
@@ -9,13 +8,11 @@ from datetime import datetime
from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import ROOT_DIR
from app.core.security import get_current_user
from app.db.session import get_db
from app.models.system_log import AuditLog, SystemLog
from app.models.user import User
from app.services.persistent_logs import record_audit_log, record_system_log
from app.services.system_control import (
@@ -38,6 +35,7 @@ from app.services.system_logs import (
append_buffer_log,
list_log_sources,
normalize_log_level,
read_database_log_snapshot,
read_log_snapshot,
)
from app.services.earth_layer_cache import earth_layer_cache
@@ -45,42 +43,6 @@ from app.services.earth_layer_cache import earth_layer_cache
router = APIRouter()
def _compact_log_context(context: dict | None) -> str:
if not context:
return ""
allowed = {
key: value
for key, value in (context or {}).items()
if key
in {
"status",
"duration_ms",
"provider",
"model",
"result_provider",
"result_model",
"collector_name",
"datasource_id",
"task_id",
"snapshot_id",
"raw_count",
"transformed_count",
"saved_count",
"created",
"updated",
"unchanged",
"deleted",
"result_count",
"status_code",
"error_type",
"error",
}
}
if not allowed:
return ""
return json.dumps(allowed, ensure_ascii=False, sort_keys=True)
class RestartTaskCreate(BaseModel):
action: str
@@ -154,6 +116,46 @@ class EarthClientLogEventResponse(BaseModel):
level: str
async def ingest_client_log_event(
source_id: str,
*,
service: str,
event: str,
default_module: str,
default_category: str,
payload: EarthClientLogEventCreate,
request: Request,
) -> EarthClientLogEventResponse:
normalized_level = normalize_log_level(payload.level)
append_buffer_log(
source_id,
level=normalized_level,
message=payload.message,
context={
"category": payload.category or "",
"url": payload.url or "",
"module": payload.module or "",
"detail": payload.detail or "",
},
)
await record_system_log(
source=source_id,
service=service,
module=payload.module or default_module,
event=event,
level=normalized_level,
message=payload.message,
category=payload.category or default_category,
context={
"url": payload.url or "",
"detail": payload.detail or "",
"module": payload.module or "",
"client_ip": request.client.host if request.client else "",
},
)
return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level)
class EarthLayerCacheStatusResponse(BaseModel):
prefix: str
key_count: int
@@ -376,100 +378,6 @@ async def get_system_log_sources(
}
async def read_database_log_snapshot(
source_id: str,
*,
limit: int,
level: str,
levels: str | None,
start_date: str | None,
end_date: str | None,
search: str | None,
db: AsyncSession,
) -> dict | None:
selected_levels = set(normalize_log_level(item) for item in (levels or level).split(",") if item.strip())
selected_levels.discard("all")
search_query = (search or "").strip().lower()
lines: list[str] = []
if source_id == "system-db":
query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(limit * 5)
result = await db.execute(query)
records = result.scalars().all()
for record in records:
record_level = normalize_log_level(record.level)
if selected_levels and record_level not in selected_levels:
continue
occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
if start_date and occurred_at and occurred_at < start_date:
continue
if end_date and occurred_at and occurred_at > end_date:
continue
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
record_level.upper(),
record.source,
record.category or "",
record.event or "",
f"request_id={record.request_id}" if record.request_id else "",
record.message,
_compact_log_context(record.context),
]
if part
)
if search_query and search_query not in line.lower():
continue
lines.append(line)
elif source_id == "audit-db":
query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(limit * 5)
result = await db.execute(query)
records = result.scalars().all()
for record in records:
occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
if start_date and occurred_at and occurred_at < start_date:
continue
if end_date and occurred_at and occurred_at > end_date:
continue
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
"INFO",
record.action,
record.target_type or "",
record.target_id or "",
record.result or "",
]
if part
)
if search_query and search_query not in line.lower():
continue
lines.append(line)
else:
return None
lines = list(reversed(lines[:limit]))
return {
"source_id": source_id,
"name": "系统事件" if source_id == "system-db" else "审计事件",
"kind": "database",
"location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
"description": "数据库持久化日志",
"category": "database" if source_id == "system-db" else "audit",
"status": "ok" if lines else "empty",
"level": level,
"selected_levels": sorted(selected_levels),
"search_query": search or "",
"available_levels": ["all", "error", "warning", "info", "debug"],
"daily_markers": [],
"line_limit": limit,
"line_count": len(lines),
"lines": lines,
}
@router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
async def get_system_log_snapshot(
source_id: str,
@@ -533,31 +441,28 @@ async def ingest_earth_client_log(
payload: EarthClientLogEventCreate,
request: Request,
):
normalized_level = normalize_log_level(payload.level)
append_buffer_log(
return await ingest_client_log_event(
"earth-client",
level=normalized_level,
message=payload.message,
context={
"category": payload.category or "",
"url": payload.url or "",
"module": payload.module or "",
"detail": payload.detail or "",
},
)
await record_system_log(
source="earth-client",
service="earth",
module=payload.module or "earth-client",
event="earth.client.runtime_log",
level=normalized_level,
message=payload.message,
category=payload.category or "client-runtime",
context={
"url": payload.url or "",
"detail": payload.detail or "",
"module": payload.module or "",
"client_ip": request.client.host if request.client else "",
},
default_module="earth-client",
default_category="client-runtime",
payload=payload,
request=request,
)
@router.post("/logs/admin-client", response_model=EarthClientLogEventResponse)
async def ingest_admin_client_log(
payload: EarthClientLogEventCreate,
request: Request,
):
return await ingest_client_log_event(
"admin-client",
service="admin",
event="admin.client.runtime_log",
default_module="admin-client",
default_category="client-runtime",
payload=payload,
request=request,
)
return {"accepted": True, "source_id": "earth-client", "level": normalized_level}

View File

@@ -29,7 +29,8 @@ async def list_tasks(
SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
ct.task_type, ct.source, ds.source as datasource_source
FROM collection_tasks ct
JOIN data_sources ds ON ct.datasource_id = ds.id
WHERE 1=1
@@ -39,12 +40,19 @@ async def list_tasks(
if datasource_id:
query += " AND ct.datasource_id = :datasource_id"
count_query += " WHERE ct.datasource_id = :datasource_id"
count_query += " AND ct.datasource_id = :datasource_id"
params["datasource_id"] = datasource_id
if status:
query += " AND ct.status = :status"
count_query += " AND ct.status = :status"
params["status"] = status
statuses = [item.strip() for item in status.split(",") if item.strip()]
if len(statuses) > 1:
placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
query += f" AND ct.status IN ({placeholders})"
count_query += f" AND ct.status IN ({placeholders})"
params.update({f"status_{index}": item for index, item in enumerate(statuses)})
else:
query += " AND ct.status = :status"
count_query += " AND ct.status = :status"
params["status"] = statuses[0] if statuses else status
query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
@@ -76,6 +84,9 @@ async def list_tasks(
"phase_unit": t[13],
"total_records": t[14],
"progress": t[15],
"task_type": t[16],
"source": t[17] or t[18],
"datasource_source": t[18],
}
for t in tasks
],

View File

@@ -6,11 +6,14 @@ from typing import Optional
from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
from jose import jwt, JWTError
from sqlalchemy import text
from app.core.config import settings
from app.core.logging import get_logger
from app.core.time import to_iso8601_utc
from app.core.websocket.manager import manager
from app.db.session import async_session_factory
from app.services.log_tail import LOG_TAIL_CHANNEL, log_tail_manager
logger = get_logger(__name__, service="api")
router = APIRouter()
@@ -37,6 +40,28 @@ async def authenticate_token(token: str) -> Optional[dict]:
return None
async def load_websocket_user_role(user_id: str | None) -> str | None:
if not user_id:
return None
try:
async with async_session_factory() as db:
result = await db.execute(
text("SELECT role, is_active FROM users WHERE id = :id"),
{"id": int(user_id)},
)
row = result.fetchone()
except Exception as exc:
logger.warning_event(
"WebSocket user role lookup failed",
event="auth.websocket.role_lookup_failed",
context={"user_id": user_id, "error": str(exc)},
)
return None
if row is None or not row[1]:
return None
return str(row[0] or "")
@router.websocket("/ws")
async def websocket_endpoint(
websocket: WebSocket,
@@ -59,6 +84,7 @@ async def websocket_endpoint(
is_anonymous = payload is None
user_id = str(payload.get("sub")) if payload else f"anonymous:{id(websocket)}"
user_role = await load_websocket_user_role(user_id) if payload else None
supported_channels = ["vessels", "earth_news", EARTH_UPDATES_CHANNEL] if is_anonymous else [
"gpu_clusters",
"submarine_cables",
@@ -70,6 +96,8 @@ async def websocket_endpoint(
"earth_news",
EARTH_UPDATES_CHANNEL,
]
if user_role == "super_admin":
supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
await manager.connect(websocket, user_id)
try:
@@ -100,6 +128,7 @@ async def websocket_endpoint(
payload_data = data.get("data", {})
if not isinstance(payload_data, dict):
payload_data = {}
log_tail_config = None
channels = payload_data.get("channels", [])
if isinstance(channels, str):
channels = [channels]
@@ -108,6 +137,26 @@ async def websocket_endpoint(
channel = payload_data.get("channel")
if channel and channel not in channels:
channels = [*channels, channel]
if LOG_TAIL_CHANNEL in channels:
if user_role != "super_admin":
await websocket.send_json(
{
"type": "subscription_error",
"data": {"channel": LOG_TAIL_CHANNEL, "detail": "Only super_admin can subscribe logs"},
}
)
channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
else:
try:
log_tail_config = await log_tail_manager.subscribe(websocket, payload_data)
except ValueError as exc:
await websocket.send_json(
{
"type": "subscription_error",
"data": {"channel": LOG_TAIL_CHANNEL, "detail": str(exc)},
}
)
channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
if is_anonymous:
channels = [channel for channel in channels if channel in supported_channels]
vessel_subscription = None
@@ -131,14 +180,20 @@ async def websocket_endpoint(
"action": "subscribe",
"channels": [
*channels,
*([LOG_TAIL_CHANNEL] if log_tail_config else []),
*(["vessels"] if vessel_subscription else []),
],
"vessels": vessel_subscription,
"logs_tail": log_tail_config.__dict__ if log_tail_config else None,
},
}
)
elif data.get("type") == "unsubscribe":
channels = data.get("data", {}).get("channels", [])
if isinstance(channels, str):
channels = [channels]
if LOG_TAIL_CHANNEL in channels:
await log_tail_manager.unsubscribe(websocket)
manager.unsubscribe(websocket, channels)
await websocket.send_json(
{
@@ -159,4 +214,5 @@ async def websocket_endpoint(
except WebSocketDisconnect:
pass
finally:
await log_tail_manager.disconnect(websocket)
manager.disconnect(websocket, user_id)

View File

@@ -322,6 +322,21 @@ class AISStreamCollector(BaseCollector):
last_success_at=now if data else None,
lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
)
if snapshot_id is not None:
from app.models.data_snapshot import DataSnapshot
snapshot = await db.get(DataSnapshot, snapshot_id)
if snapshot:
snapshot.record_count = records_added
snapshot.status = "success"
snapshot.completed_at = now
snapshot.summary = {
"created": records_added,
"updated": 0,
"unchanged": 0,
"deleted": 0,
"storage": "ais_raw_observations",
}
await db.commit()
await self.update_progress(records_added, force=True)
return records_added

View File

@@ -25,11 +25,17 @@ FALLBACK_GROUPS = (
"starlink",
"gps-ops",
"galileo",
"glonass",
"glo-ops",
"beidou",
"leo",
"geo",
"iridium-next",
"stations",
"visual",
"weather",
"science",
"cubesat",
"amateur",
"last-30-days",
)
FETCH_RETRY_ATTEMPTS = 3
FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
@@ -220,27 +226,28 @@ class CelesTrakTLECollector(BaseCollector):
try:
for group in FALLBACK_GROUPS:
group_url = self._group_url(group)
try:
body_path = await self._downloader.download_file(
client,
group_url,
extension=".json",
accept="application/json",
validate_existing=self._validate_json_file,
)
except DownloadHTTPStatusError as exc:
if not self._is_not_updated_response(exc):
raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
cached_path = self._downloader.get_cached_file(
group_url,
".json",
validate_existing=self._validate_json_file,
)
if cached_path is None:
cached_path = self._downloader.get_cached_file(
group_url,
".json",
validate_existing=self._validate_json_file,
)
if cached_path is not None:
body_path = cached_path
else:
try:
body_path = await self._downloader.download_file(
client,
group_url,
extension=".json",
accept="application/json",
validate_existing=self._validate_json_file,
)
except DownloadHTTPStatusError as exc:
if not self._is_not_updated_response(exc):
raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
raise RuntimeError(
f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
) from exc
body_path = cached_path
group_records = await self._load_downloaded_payload(
body_path,

View File

@@ -119,6 +119,21 @@ class VesselAISCollector(BaseCollector):
last_success_at=now if data else None,
lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
)
if snapshot_id is not None:
from app.models.data_snapshot import DataSnapshot
snapshot = await db.get(DataSnapshot, snapshot_id)
if snapshot:
snapshot.record_count = records_added
snapshot.status = "success"
snapshot.completed_at = now
snapshot.summary = {
"created": records_added,
"updated": 0,
"unchanged": 0,
"deleted": 0,
"storage": "ais_raw_observations",
}
await db.commit()
await self._broadcast_vessel_snapshot(data)
await self.update_progress(records_added, force=True)

View File

@@ -54,6 +54,9 @@ DATA_WRITE_JOB_TYPES = (JOB_TYPE_COLLECT, JOB_TYPE_CLEAR_DATA, JOB_TYPE_CLEAR_CA
SOURCE_LOCK_JOB_STATUSES = (JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
QUEUE_POLL_SECONDS = 0.35
JOB_STALE_LOCK_MINUTES = 90
ORPHAN_CANCELLING_GRACE_SECONDS = 30
JOB_RECOVERY_SWEEP_SECONDS = 15
DATA_DELETE_BATCH_SIZE = 50_000
DEFAULT_WORKER_CONCURRENCY = 2
RUNNING_DATA_JOB_TASKS: dict[int, asyncio.Task[Any]] = {}
@@ -281,6 +284,7 @@ class DataJobWorker:
self._task: asyncio.Task[None] | None = None
self._stop_event: asyncio.Event | None = None
self._running: set[asyncio.Task[Any]] = set()
self._last_recovery_sweep_at: datetime | None = None
def start(self) -> None:
if self._task and not self._task.done():
@@ -303,6 +307,11 @@ class DataJobWorker:
await self._recover_stale_running_jobs()
while not self._stop_event.is_set():
self._running = {task for task in self._running if not task.done()}
if (
self._last_recovery_sweep_at is None
or (_utcnow() - self._last_recovery_sweep_at).total_seconds() >= JOB_RECOVERY_SWEEP_SECONDS
):
await self._recover_stale_running_jobs()
if len(self._running) >= self.concurrency:
await asyncio.sleep(QUEUE_POLL_SECONDS)
continue
@@ -316,7 +325,9 @@ class DataJobWorker:
self._running.add(runner)
async def _recover_stale_running_jobs(self) -> None:
self._last_recovery_sweep_at = _utcnow()
cutoff = _utcnow() - timedelta(minutes=JOB_STALE_LOCK_MINUTES)
orphan_cancelling_cutoff = _utcnow() - timedelta(seconds=ORPHAN_CANCELLING_GRACE_SECONDS)
async with async_session_factory() as db:
result = await db.execute(
select(CollectionTask)
@@ -332,6 +343,21 @@ class DataJobWorker:
job.error_message = "Marked failed after stale data job lock timeout"
if stale_jobs:
await db.commit()
orphan_result = await db.execute(
select(CollectionTask)
.where(CollectionTask.status == JOB_STATUS_CANCELLING)
.where(CollectionTask.locked_at.is_(None))
.where(CollectionTask.requested_cancel_at.is_not(None))
.where(CollectionTask.requested_cancel_at < orphan_cancelling_cutoff)
)
for job in orphan_result.scalars().all():
if job.id in RUNNING_DATA_JOB_TASKS:
continue
await _cancel_task_without_runner(
db,
job,
reason=job.cancel_reason or "cancelled_after_orphaned_runner",
)
async def _claim_next_job(self) -> int | None:
async with async_session_factory() as db:
@@ -477,15 +503,22 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
await db.commit()
await _broadcast_task_update(task)
count_result = await db.execute(
select(CollectedData.id).where(CollectedData.source == source)
deleted_count = await _delete_table_rows_by_source(
db,
task,
table_name="collected_data",
source_column="source",
source=source,
)
derived_deleted_counts = await _clear_derived_datasource_data_in_batches(
db,
task,
source,
progress_offset=deleted_count,
)
collected_ids = [row[0] for row in count_result.all()]
derived_deleted_counts = await clear_derived_datasource_data(db, source)
if collected_ids:
await db.execute(CollectedData.__table__.delete().where(CollectedData.id.in_(collected_ids)))
deleted_count = len(collected_ids)
derived_deleted_count = sum(derived_deleted_counts.values())
if any(key.startswith("ais_") for key in derived_deleted_counts):
await db.execute(text("ANALYZE ais_raw_observations"))
task.records_processed = deleted_count + derived_deleted_count
task.total_records = task.records_processed
@@ -504,10 +537,99 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
task.phase = "completed"
task.phase_message = "数据库数据已清理"
task.completed_at = _utcnow()
datasource = await db.get(DataSource, task.datasource_id)
if datasource is not None:
datasource.last_status = JOB_STATUS_SUCCESS
datasource.last_run_at = task.completed_at
await db.execute(
DataSnapshot.__table__.update()
.where(DataSnapshot.source == source)
.values(is_current=False)
)
await db.commit()
await _broadcast_task_update(task)
async def _delete_table_rows_by_source(
db: AsyncSession,
task: CollectionTask,
*,
table_name: str,
source_column: str,
source: str,
progress_offset: int = 0,
) -> int:
deleted = 0
while True:
result = await db.execute(
text(
f"""
WITH doomed AS (
SELECT ctid
FROM {table_name}
WHERE {source_column} = :source
LIMIT :batch_size
),
deleted_rows AS (
DELETE FROM {table_name}
USING doomed
WHERE {table_name}.ctid = doomed.ctid
RETURNING 1
)
SELECT COUNT(*) FROM deleted_rows
"""
),
{"source": source, "batch_size": DATA_DELETE_BATCH_SIZE},
)
batch_deleted = max(int(result.scalar_one() or 0), 0)
if batch_deleted <= 0:
break
deleted += batch_deleted
task.records_processed = progress_offset + deleted
task.phase_current = task.records_processed
task.phase_unit = "records"
task.phase_message = f"正在删除数据:{task.records_processed}"
await db.commit()
await _broadcast_task_update(task)
return deleted
async def _clear_derived_datasource_data_in_batches(
db: AsyncSession,
task: CollectionTask,
source: str,
progress_offset: int = 0,
) -> dict[str, int]:
deleted_counts: dict[str, int] = {}
if source in {"barentswatch_vessels", "aisstream_vessels"}:
deleted_counts["ais_conflict_records"] = await _delete_table_rows_by_source(
db,
task,
table_name="ais_conflict_records",
source_column="selected_source",
source=source,
progress_offset=progress_offset + sum(deleted_counts.values()),
)
deleted_counts["ais_source_health"] = await _delete_table_rows_by_source(
db,
task,
table_name="ais_source_health",
source_column="source",
source=source,
progress_offset=progress_offset + sum(deleted_counts.values()),
)
deleted_counts["ais_raw_observations"] = await _delete_table_rows_by_source(
db,
task,
table_name="ais_raw_observations",
source_column="source",
source=source,
progress_offset=progress_offset + sum(deleted_counts.values()),
)
return deleted_counts
return await clear_derived_datasource_data(db, source)
async def _run_clear_cache_job(db: AsyncSession, task: CollectionTask) -> None:
source = str(task.source or (task.payload or {}).get("source") or "").strip()
if not source:

View File

@@ -24,6 +24,7 @@ EARTH_LAYER_ADAPTERS: tuple[EarthLayerAdapter, ...] = (
tables=frozenset({"vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
layers=("vessels",),
cache_patterns=("vessels*", "summary*"),
derived_models=("ais_raw_observations", "ais_conflict_records", "ais_source_health"),
),
EarthLayerAdapter(
sources=frozenset(
@@ -163,17 +164,24 @@ async def clear_derived_datasource_data(db: AsyncSession, source: str) -> dict[s
from app.models.bgp_anomaly import BGPAnomaly
from app.models.bgp_incident import BGPIncident
from app.models.bgp_observation import BGPObservation
from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth
model_by_key: dict[str, Any] = {
"bgp_observations": BGPObservation,
"bgp_anomalies": BGPAnomaly,
"bgp_incidents": BGPIncident,
"ais_raw_observations": AISRawObservation,
"ais_conflict_records": AISConflictRecord,
"ais_source_health": AISSourceHealth,
}
deleted_counts: dict[str, int] = {}
for key in adapter.derived_models:
model = model_by_key.get(key)
if model is None:
continue
result = await db.execute(model.__table__.delete().where(model.source == source))
if key == "ais_conflict_records":
result = await db.execute(model.__table__.delete().where(model.selected_source == source))
else:
result = await db.execute(model.__table__.delete().where(model.source == source))
deleted_counts[key] = int(result.rowcount or 0)
return deleted_counts

View File

@@ -0,0 +1,161 @@
from __future__ import annotations
import asyncio
from dataclasses import dataclass, field
from datetime import UTC, datetime
from typing import Any
from fastapi import WebSocket
from app.db.session import async_session_factory
from app.services.system_logs import (
DEFAULT_LOG_LINE_LIMIT,
LOG_SOURCES,
MAX_LOG_LINE_LIMIT,
read_database_log_events,
read_log_events,
)
DATABASE_LOG_SOURCE_IDS = {"system-db", "audit-db"}
LOG_TAIL_CHANNEL = "logs_tail"
LOG_TAIL_INTERVAL_SECONDS = 1.5
LOG_TAIL_SCAN_MULTIPLIER = 5
@dataclass(frozen=True)
class LogTailConfig:
source_id: str
limit: int = DEFAULT_LOG_LINE_LIMIT
level: str = "all"
levels: str | None = None
start_date: str | None = None
end_date: str | None = None
search: str | None = None
@dataclass
class LogTailSubscription:
config: LogTailConfig
emitted_cursors: set[str] = field(default_factory=set)
task: asyncio.Task | None = None
class LogTailManager:
def __init__(self) -> None:
self._subscriptions: dict[WebSocket, LogTailSubscription] = {}
def normalize_config(self, payload: dict[str, Any]) -> LogTailConfig:
source_id = str(payload.get("source_id") or payload.get("source") or "").strip()
if not source_id:
raise ValueError("source_id is required")
if source_id not in LOG_SOURCES and source_id not in DATABASE_LOG_SOURCE_IDS:
raise ValueError("Log source not found")
try:
limit = int(payload.get("limit") or DEFAULT_LOG_LINE_LIMIT)
except (TypeError, ValueError) as exc:
raise ValueError("limit must be a number") from exc
if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
raise ValueError(f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
return LogTailConfig(
source_id=source_id,
limit=limit,
level=str(payload.get("level") or "all"),
levels=str(payload.get("levels")).strip() if payload.get("levels") else None,
start_date=str(payload.get("start_date")).strip() if payload.get("start_date") else None,
end_date=str(payload.get("end_date")).strip() if payload.get("end_date") else None,
search=str(payload.get("search")).strip() if payload.get("search") else None,
)
async def subscribe(self, websocket: WebSocket, payload: dict[str, Any]) -> LogTailConfig:
config = self.normalize_config(payload)
await self.unsubscribe(websocket)
subscription = LogTailSubscription(config=config)
subscription.task = asyncio.create_task(self._run_tail(websocket, subscription))
self._subscriptions[websocket] = subscription
return config
async def unsubscribe(self, websocket: WebSocket) -> None:
subscription = self._subscriptions.pop(websocket, None)
if subscription and subscription.task:
subscription.task.cancel()
try:
await subscription.task
except asyncio.CancelledError:
pass
async def disconnect(self, websocket: WebSocket) -> None:
await self.unsubscribe(websocket)
async def _run_tail(self, websocket: WebSocket, subscription: LogTailSubscription) -> None:
first_frame = True
while True:
events = await self._read_events(subscription.config)
if first_frame:
visible_events = events[-subscription.config.limit :]
subscription.emitted_cursors.update(event.cursor for event in visible_events)
await self._send_frame(websocket, subscription.config, "snapshot", visible_events)
first_frame = False
else:
new_events = [
event
for event in events
if event.cursor not in subscription.emitted_cursors
]
if new_events:
visible_events = new_events[-subscription.config.limit :]
subscription.emitted_cursors.update(event.cursor for event in visible_events)
await self._send_frame(websocket, subscription.config, "append", visible_events)
await asyncio.sleep(LOG_TAIL_INTERVAL_SECONDS)
async def _read_events(self, config: LogTailConfig):
scan_limit = max(config.limit * LOG_TAIL_SCAN_MULTIPLIER, config.limit)
if config.source_id in DATABASE_LOG_SOURCE_IDS:
async with async_session_factory() as db:
events = await read_database_log_events(
config.source_id,
scan_limit=scan_limit,
level=config.level,
levels=config.levels,
start_date=config.start_date,
end_date=config.end_date,
search=config.search,
db=db,
)
return events or []
events = read_log_events(
config.source_id,
scan_limit=scan_limit,
level=config.level,
levels=config.levels,
start_date=config.start_date,
end_date=config.end_date,
search=config.search,
)
return events or []
async def _send_frame(self, websocket: WebSocket, config: LogTailConfig, mode: str, events) -> None:
await websocket.send_json(
{
"type": "data_frame",
"channel": LOG_TAIL_CHANNEL,
"timestamp": datetime.now(UTC).isoformat(),
"payload": {
"mode": mode,
"source_id": config.source_id,
"line_count": len(events),
"lines": [event.line for event in events],
"filters": {
"limit": config.limit,
"level": config.level,
"levels": config.levels,
"start_date": config.start_date,
"end_date": config.end_date,
"search": config.search,
},
"status": "ok",
},
}
)
log_tail_manager = LogTailManager()

View File

@@ -5,6 +5,7 @@ import os
import re
import shutil
import subprocess
import hashlib
from collections import Counter, deque
from dataclasses import dataclass
@@ -13,6 +14,9 @@ from pathlib import Path
from typing import Any
from app.core.security import redis_client
from app.models.system_log import AuditLog, SystemLog
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
DEFAULT_LOG_LINE_LIMIT = 200
MAX_LOG_LINE_LIMIT = 1000
@@ -99,6 +103,16 @@ class StructuredLogEntry:
search_text: str
@dataclass(frozen=True)
class LogEvent:
source_id: str
cursor: str
timestamp: datetime | None
level: str | None
line: str
search_text: str
@dataclass
class DailyLogMarker:
date_token: str
@@ -135,7 +149,7 @@ LOG_SOURCES: dict[str, LogSource] = {
name="前端开发服务",
kind="file",
location=_state_log_path("frontend.log"),
description="控制台与 Earth 前端开发服务输出。",
description="控制台与智能星球前端开发服务输出。",
category="service",
fallback_locations=("/tmp/planet_frontend.log",),
),
@@ -150,13 +164,22 @@ LOG_SOURCES: dict[str, LogSource] = {
),
"earth-client": LogSource(
source_id="earth-client",
name="Earth 浏览器端",
name="智能星球浏览器端",
kind="buffer",
location="redis://planet:system_logs:earth-client",
description="Earth 浏览器端上报的运行时错误与关键业务日志。",
description="智能星球浏览器端上报的运行时错误与关键业务日志。",
category="client",
buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:earth-client",
),
"admin-client": LogSource(
source_id="admin-client",
name="控制台浏览器端",
kind="buffer",
location="redis://planet:system_logs:admin-client",
description="控制台浏览器端上报的运行时错误。",
category="client",
buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:admin-client",
),
}
@@ -365,6 +388,59 @@ def build_buffer_entry(payload: dict[str, Any]) -> StructuredLogEntry:
)
def compact_log_context(context: dict | None) -> str:
if not context:
return ""
allowed = {
key: value
for key, value in (context or {}).items()
if key
in {
"status",
"duration_ms",
"provider",
"model",
"result_provider",
"result_model",
"collector_name",
"datasource_id",
"task_id",
"snapshot_id",
"raw_count",
"transformed_count",
"saved_count",
"created",
"updated",
"unchanged",
"deleted",
"result_count",
"status_code",
"error_type",
"error",
"route",
"module",
}
}
if not allowed:
return ""
return json.dumps(allowed, ensure_ascii=False, sort_keys=True)
def context_search_aliases(context: dict | None) -> str:
if not context:
return ""
aliases: list[str] = []
for key, value in sorted((context or {}).items()):
if value is None or isinstance(value, (dict, list, tuple, set)):
continue
normalized_key = str(key).strip()
normalized_value = str(value).strip()
if not normalized_key or not normalized_value:
continue
aliases.append(f"{normalized_key}={normalized_value}")
return " ".join(aliases)
def read_file_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
path = resolve_file_log_path(source)
if not path.exists():
@@ -437,6 +513,178 @@ def read_source_entries(source: LogSource, scan_limit: int) -> list[StructuredLo
return []
def _database_event_from_system_record(record: SystemLog) -> LogEvent:
record_level = normalize_log_level(record.level)
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
record_level.upper(),
record.source,
record.category or "",
record.event or "",
f"request_id={record.request_id}" if record.request_id else "",
record.message,
compact_log_context(record.context),
]
if part
)
search_text = " ".join(
[
line,
f"id={record.id}",
f"user_id={record.user_id}" if record.user_id else "",
context_search_aliases(record.context),
json.dumps(record.context or {}, ensure_ascii=False, sort_keys=True),
]
).lower()
return LogEvent(
source_id="system-db",
cursor=f"system-db:{record.id}",
timestamp=record.occurred_at,
level=None if record_level == LOG_LEVEL_ALL else record_level,
line=line,
search_text=search_text,
)
def _database_event_from_audit_record(record: AuditLog) -> LogEvent:
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
"INFO",
record.action,
record.target_type or "",
record.target_id or "",
record.result or "",
f"request_id={record.request_id}" if record.request_id else "",
]
if part
)
search_text = " ".join(
[
line,
f"id={record.id}",
f"actor_id={record.actor_id}" if record.actor_id else "",
record.actor_name or "",
context_search_aliases(record.details),
json.dumps(record.details or {}, ensure_ascii=False, sort_keys=True),
]
).lower()
return LogEvent(
source_id="audit-db",
cursor=f"audit-db:{record.id}",
timestamp=record.occurred_at,
level=LOG_LEVEL_INFO,
line=line,
search_text=search_text,
)
async def read_database_log_events(
source_id: str,
*,
scan_limit: int,
level: str = LOG_LEVEL_ALL,
levels: str | None = None,
start_date: str | None = None,
end_date: str | None = None,
search: str | None = None,
db: AsyncSession,
) -> list[LogEvent] | None:
selected_levels = normalize_log_levels(level, levels)
search_query = (search or "").strip()
if source_id == "system-db":
query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(scan_limit)
result = await db.execute(query)
events = [_database_event_from_system_record(record) for record in result.scalars().all()]
elif source_id == "audit-db":
query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(scan_limit)
result = await db.execute(query)
events = [_database_event_from_audit_record(record) for record in result.scalars().all()]
else:
return None
events = list(reversed(events))
return [
event
for event in events
if event_matches_levels(event, selected_levels)
and event_matches_search(event, search_query)
and event_matches_date_range(event, start_date, end_date)
]
async def read_database_log_snapshot(
source_id: str,
*,
limit: int,
level: str,
levels: str | None,
start_date: str | None,
end_date: str | None,
search: str | None,
db: AsyncSession,
) -> dict[str, Any] | None:
events = await read_database_log_events(
source_id,
scan_limit=limit * 5,
level=level,
levels=levels,
start_date=start_date,
end_date=end_date,
search=search,
db=db,
)
if events is None:
return None
visible_events = events[-limit:]
selected_levels = normalize_log_levels(level, levels)
return {
"source_id": source_id,
"name": "系统事件" if source_id == "system-db" else "审计事件",
"kind": "database",
"location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
"description": "数据库持久化日志",
"category": "database" if source_id == "system-db" else "audit",
"status": "ok" if visible_events else "empty",
"level": level,
"selected_levels": list(selected_levels),
"search_query": search or "",
"available_levels": ["all", "error", "warning", "info", "debug"],
"daily_markers": build_daily_log_markers_from_events(events),
"line_limit": limit,
"line_count": len(visible_events),
"lines": [event.line for event in visible_events],
}
def _stable_hash(value: str) -> str:
return hashlib.sha1(value.encode("utf-8", errors="replace")).hexdigest()[:16]
def build_log_events(source_id: str, entries: list[StructuredLogEntry]) -> list[LogEvent]:
events: list[LogEvent] = []
seen: dict[str, int] = {}
for entry in entries:
stable_value = entry.raw_line or entry.display_line
digest = _stable_hash(stable_value)
occurrence = seen.get(digest, 0) + 1
seen[digest] = occurrence
events.append(
LogEvent(
source_id=source_id,
cursor=f"{source_id}:{digest}:{occurrence}",
timestamp=entry.timestamp,
level=entry.level,
line=entry.display_line,
search_text=entry.search_text,
)
)
return events
def matches_levels(entry: StructuredLogEntry, selected_levels: tuple[str, ...]) -> bool:
if not selected_levels:
return True
@@ -469,6 +717,34 @@ def matches_search(entry: StructuredLogEntry, search: str | None) -> bool:
return query in entry.search_text
def event_matches_levels(event: LogEvent, selected_levels: tuple[str, ...]) -> bool:
if not selected_levels:
return True
return event.level in selected_levels
def event_matches_date_range(event: LogEvent, start_date: str | None, end_date: str | None) -> bool:
if not start_date and not end_date:
return True
if event.timestamp is None:
return False
date_token = event.timestamp.astimezone(UTC).date().isoformat()
if start_date and date_token < start_date:
return False
if end_date and date_token > end_date:
return False
return True
def event_matches_search(event: LogEvent, search: str | None) -> bool:
if search is None:
return True
query = search.strip().lower()
if not query:
return True
return query in event.search_text
def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str, Any]]:
grouped: dict[str, list[StructuredLogEntry]] = {}
for entry in entries:
@@ -503,6 +779,69 @@ def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str,
return [marker.__dict__ for marker in markers]
def build_daily_log_markers_from_events(events: list[LogEvent]) -> list[dict[str, Any]]:
grouped: dict[str, list[LogEvent]] = {}
for event in events:
if event.timestamp is None:
continue
date_token = event.timestamp.astimezone(UTC).date().isoformat()
grouped.setdefault(date_token, []).append(event)
markers: list[DailyLogMarker] = []
for date_token, group in sorted(grouped.items()):
level_counts = Counter(
event.level
for event in group
if event.level in SUPPORTED_LOG_LEVELS and event.level != LOG_LEVEL_ALL
)
dominant_level = LOG_LEVEL_INFO
if level_counts:
dominant_level = sorted(
level_counts.items(),
key=lambda item: (
-item[1],
("error", "warning", "info", "debug").index(item[0]),
),
)[0][0]
markers.append(
DailyLogMarker(
date_token=date_token,
total=len(group),
dominant_level=dominant_level,
)
)
return [marker.__dict__ for marker in markers]
def read_log_events(
source_id: str,
scan_limit: int,
*,
level: str = LOG_LEVEL_ALL,
levels: str | None = None,
start_date: str | None = None,
end_date: str | None = None,
search: str | None = None,
) -> list[LogEvent] | None:
source = LOG_SOURCES.get(source_id)
if source is None:
return None
selected_levels = normalize_log_levels(level, levels)
search_query = (search or "").strip()
events = build_log_events(source_id, read_source_entries(source, scan_limit))
marker_events = [
event
for event in events
if event_matches_levels(event, selected_levels) and event_matches_search(event, search_query)
]
return [
event
for event in marker_events
if event_matches_date_range(event, start_date, end_date)
]
def read_log_snapshot(
source_id: str,
limit: int,
@@ -520,18 +859,18 @@ def read_log_snapshot(
selected_levels = normalize_log_levels(level, levels)
search_query = (search or "").strip()
scan_limit = max(min(MAX_LOG_LINE_LIMIT * 5, 5000), limit * 5, BUFFER_LOG_LIMIT if source.kind == "buffer" else 1000)
all_entries = read_source_entries(source, scan_limit)
marker_entries = [
entry
for entry in all_entries
if matches_levels(entry, selected_levels) and matches_search(entry, search_query)
all_events = build_log_events(source_id, read_source_entries(source, scan_limit))
marker_events = [
event
for event in all_events
if event_matches_levels(event, selected_levels) and event_matches_search(event, search_query)
]
filtered_entries = [
entry
for entry in marker_entries
if matches_date_range(entry, start_date, end_date)
filtered_events = [
event
for event in marker_events
if event_matches_date_range(event, start_date, end_date)
]
visible_entries = filtered_entries[-limit:]
visible_events = filtered_events[-limit:]
compatibility_level = selected_levels[0] if len(selected_levels) == 1 else LOG_LEVEL_ALL
return {
@@ -552,8 +891,8 @@ def read_log_snapshot(
LOG_LEVEL_INFO,
LOG_LEVEL_DEBUG,
],
"daily_markers": build_daily_log_markers(marker_entries),
"daily_markers": build_daily_log_markers_from_events(marker_events),
"line_limit": limit,
"line_count": len(visible_entries),
"lines": [entry.display_line for entry in visible_entries],
"line_count": len(visible_events),
"lines": [event.line for event in visible_events],
}

View File

@@ -672,6 +672,39 @@ async def test_ingest_earth_client_log_accepts_public_events():
app.dependency_overrides.clear()
@pytest.mark.asyncio
async def test_ingest_admin_client_log_accepts_public_events():
transport = ASGITransport(app=app)
try:
with patch("app.api.v1.system_control.append_buffer_log") as mock_append_buffer_log:
with patch("app.api.v1.system_control.record_system_log", new_callable=AsyncMock) as mock_record_system_log:
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.post(
"/api/v1/system/logs/admin-client",
json={
"level": "error",
"message": "控制台发生未处理 Promise 错误",
"category": "unhandledrejection",
"module": "admin",
"url": "http://test/logs",
"detail": "stack preview",
},
)
assert response.status_code == 200
data = response.json()
assert data["accepted"] is True
assert data["source_id"] == "admin-client"
mock_append_buffer_log.assert_called_once()
mock_record_system_log.assert_awaited_once()
persisted_kwargs = mock_record_system_log.await_args.kwargs
assert persisted_kwargs["source"] == "admin-client"
assert persisted_kwargs["event"] == "admin.client.runtime_log"
assert persisted_kwargs["category"] == "unhandledrejection"
assert persisted_kwargs["context"]["url"] == "http://test/logs"
finally:
app.dependency_overrides.clear()
@pytest.mark.asyncio
async def test_earth_layer_cache_status_requires_super_admin(auth_headers, monkeypatch):
def override_get_current_user():

View File

@@ -2,8 +2,10 @@ from __future__ import annotations
import json
from datetime import UTC, datetime
from pathlib import Path
from app.models.system_log import SystemLog
from app.services import system_logs
@@ -40,10 +42,10 @@ def test_read_log_snapshot_uses_structured_buffer_timestamp_level_and_search(mon
{
"earth-client": system_logs.LogSource(
source_id="earth-client",
name="Earth 浏览器端",
name="智能星球浏览器端",
kind="buffer",
location="redis://planet:system_logs:earth-client",
description="Earth 浏览器端上报日志",
description="智能星球浏览器端上报日志",
category="client",
buffer_key=system_logs.get_buffer_log_key("earth-client"),
)
@@ -156,6 +158,52 @@ def test_append_buffer_log_persists_normalized_level(monkeypatch):
assert payload["message"] == "feed delayed"
def test_list_log_sources_includes_admin_client(monkeypatch):
fake_redis = FakeRedis()
monkeypatch.setattr(system_logs, "redis_client", fake_redis)
sources = system_logs.list_log_sources()
admin_source = next(item for item in sources if item["source_id"] == "admin-client")
assert admin_source["kind"] == "buffer"
assert admin_source["category"] == "client"
def test_read_log_events_returns_stable_cursors(tmp_path: Path, monkeypatch):
log_path = tmp_path / "backend.log"
log_path.write_text(
"\n".join(
[
"2026-04-22 08:00:00 INFO service booted",
"2026-04-22 08:01:00 ERROR service failed",
]
),
encoding="utf-8",
)
monkeypatch.setattr(
system_logs,
"LOG_SOURCES",
{
"backend": system_logs.LogSource(
source_id="backend",
name="后端服务",
kind="file",
location=str(log_path),
description="测试文件日志",
category="service",
)
},
)
events = system_logs.read_log_events("backend", 50, level="error")
assert events is not None
assert len(events) == 1
assert events[0].source_id == "backend"
assert events[0].cursor.startswith("backend:")
assert events[0].line.endswith("ERROR service failed")
def test_infer_log_level_prefers_leading_prefix_over_query_string():
line = 'INFO: 127.0.0.1 - "GET /api/v1/system/logs/backend?limit=200&level=error&levels=error HTTP/1.1" 200 OK'
@@ -215,3 +263,22 @@ def test_read_log_snapshot_strips_nul_bytes_from_file_lines(tmp_path: Path, monk
"ERROR: bind failed",
"2026-04-23 23:41:32 INFO service=backend message=request served",
]
def test_database_system_log_search_matches_context_key_value_aliases():
record = SystemLog(
id=2218,
occurred_at=datetime(2026, 5, 28, 9, 14, 50, tzinfo=UTC),
source="backend",
service="collector",
module="app.services.collectors.base",
event="collector.run.failed",
level="error",
message="Collector run failed",
context={"collector_name": "celestrak_tle", "datasource_id": 20, "task_id": 26906},
)
event = system_logs._database_event_from_system_record(record)
assert system_logs.event_matches_search(event, "task_id=26906")
assert system_logs.event_matches_search(event, "datasource_id=20")

View File

@@ -8,6 +8,70 @@ This project follows the repository versioning rule:
- `improvement` -> `+0.0.1`bugfix + 小功能混合)
- `bugfix` -> `+0.0.1`
## [0.68.1] — 2026-05-28
Released: 2026-05-28
### Highlights
- 修复 CelesTrak active 未更新窗口下清库后无法恢复的问题fallback 会优先复用本地有效 group 缓存。
- 修复数据源任务队列“查看日志”无法按 `task_id=...` 命中数据库结构化日志的问题。
### Added / Fixed / Improved
- CelesTrak fallback group 列表改为公开可用分组,移除失效 group并在没有 active 缓存时仍可从本地 group 缓存恢复采集。
- 数据库日志搜索补充 JSON context 的 `key=value` 别名,支持 `task_id=26906``datasource_id=20` 这类控制台跳转查询。
- 补充 CelesTrak 缓存边界、数据源任务日志跳转和运维恢复说明的中英文文档。
---
## [0.68.0] — 2026-05-28
Released: 2026-05-28
### Highlights
- 新增数据源任务队列的实时指标校准和批量删除进度,让大表清理、取消和完成状态在控制台中可感知。
- 新增智能星球 interactable 可插拔聚类策略,支持稳定 3D 球面聚类、动态屏幕聚类和 250% 以上自动散开。
- 改进新设备启动流程,`planet.sh` 会在启动前同步前端依赖,避免缺失依赖导致控制台动态导入失败。
### Added / Fixed / Improved
- 数据源列表改为中文记录数指标,并对 AIS 大表使用统计估算 + 单条详情精确校准,降低首次加载成本。
- 数据删除任务改为分批删除并广播进度,清理 AIS 衍生表后自动 `ANALYZE`,同时修复取消中任务恢复和状态文案。
- Earth BGP、算力中心和 interactable 图层默认使用 `stable-spherical` 聚类,船舶实时层保留 `dynamic-screen`
- 新增中英文 Earth interactable clustering 文档,并补充采集队列、后端删除语义和前端依赖同步说明。
---
## [0.67.0] — 2026-05-27
Released: 2026-05-27
### Highlights
- 新增控制台日志实时跟随与前端运行时错误上报,帮助在控制台内直接排查 Admin / Earth 客户端异常。
- 重构智能星球 Interactable 聚合和 wheel 缩放输入,保持真实地理锚点稳定,同时让鼠标滚轮和触控板拥有各自合适的缩放手感。
### Added / Fixed / Improved
- 新增 `/ws` 日志 tail 通道、数据库/文件日志统一事件读取,以及 Admin `error` / `unhandledrejection` / React ErrorBoundary 上报链路。
- 优化控制台日志页状态颜色、跟随体验和运行时错误展示,并将 Admin 本地工具模块从 `lib` 命名迁移为局部 `utils`
- 修复智能星球国界/高清材质壳半径对齐、Interactable cluster 圆点显示、缩放目标累积和触控板连续缩放问题。
- 更新 `planet.sh``.gitignore`,避免新环境构建污染锁文件并移除前端 `lib` 目录特殊放行。
- 补充智能星球前端、渲染层级、控制台状态与运行时日志文档。
---
## [0.66.3] — 2026-05-26
Released: 2026-05-26
### Highlights
- 修复控制台 Admin 动态导入时缺失 `../lib/utils` 导致 Vite 返回 500 的问题。
- 补齐前端依赖安装状态,确保 Markdown Mermaid 渲染器可解析 `mermaid` 包。
### Added / Fixed / Improved
- 新增 Admin 本地 `cn``formatNumber` 工具模块恢复布局、UI 组件和 Dashboard 的共享工具引用。
- 放开 `.gitignore``frontend/src/admin/lib` 的源码例外,避免 utility module 再次被全局 `lib/` 规则漏提交。
- 重启前端开发服务并验证 `/admin` 无 Vite overlay 和 console error。
---
## [0.66.2] — 2026-05-26
Released: 2026-05-26

View File

@@ -2,7 +2,7 @@
## 背景
状态Phase 1 已经开始落地Phase 2 的 BGP 事件 / 观测站迁移和 Phase 3 的算力中心迁移也已完成。`frontend/public/earth/js/interactable.js` 已新增AIS 船只、BGP 事件、BGP 观测站和算力中心图层已经改为通过 `createInteractableLayer()` 使用通用批量 `Points`、hover / locked overlay、默认 glow、状态更新、asset icon 预加载、屏幕空间 picking、固定 / 距离缩放和跨 Interactable 同坐标避让。登陆点因 `THREE.Points` 边缘深度裁切和贴地层级要求,已退回专用 `THREE.Sprite` 黄色球路径,并与海缆同高度同 renderOrder。后续阶段聚焦把可复用的扩圈 / 雷达扇形动画正式沉淀成 `animations` 扩展。
状态Phase 1 已经开始落地Phase 2 的 BGP 事件 / 观测站迁移和 Phase 3 的算力中心迁移也已完成。`frontend/public/earth/js/interactable.js` 已新增AIS 船只、BGP 事件、BGP 观测站和算力中心图层已经改为通过 `createInteractableLayer()` 使用通用批量 `Points`、hover / locked overlay、默认 glow、状态更新、asset icon 预加载、屏幕空间 picking、固定 / 距离缩放和跨 Interactable 同坐标关系元数据。登陆点因 `THREE.Points` 边缘深度裁切和贴地层级要求,已退回专用 `THREE.Sprite` 黄色球路径,并与海缆同高度同 renderOrder。后续阶段聚焦把可复用的扩圈 / 雷达扇形动画正式沉淀成 `animations` 扩展。
当前实现说明和接入示例见:
@@ -100,9 +100,10 @@ createInteractableLayer({
| `picking.throttleMs` | `100` | `80` | hover picking 节流。 |
| `picking.skipWhileDragging` | `true` | `true` | 拖动和惯性期间跳过 hover picking。 |
| `zIndexPolicy` | `"surface-icon"` | `"surface-icon"` | 预设层级策略,避免每个业务图层手写高度和 renderOrder。 |
| `avoidance.enabled` | `true / false` | `true` | 是否参与跨 Interactable 的同坐标避让。默认开启,同一经纬度下的图标会沿地表切平面小幅排开,方便辨认和选择。 |
| `avoidance.radius` | `number` | `1.1` | 同坐标避让的第一圈半径,单位为地球本地坐标单位。 |
| `avoidance.enabled` | `true / false` | `true` | 是否记录跨 Interactable 的同坐标关系。该配置只生成 overlap 元数据,不允许移动真实 marker 坐标。 |
| `avoidance.precision` | `number` | `4` | 经纬度归并精度,默认约等于只处理几乎完全重叠的图标。 |
| `cluster.enabled` | `true / false` | 跟随 avoidance | 是否参与跨 Interactable 的当前帧屏幕重叠合并。只影响显示,不改变业务坐标。 |
| `cluster.maxMarkersPerDot` | `number` | `14` | 单个聚合圆点的对象上限;超过后按屏幕局部邻近关系拆成多个较小圆点,避免一个点过大或跨区域串联。 |
| `legend` | `{ label, color, shape }[]` | `[]` | 可选图例声明,业务层也可以继续自己导出。 |
| `metadata` | object | `{}` | 业务扩展数据,不参与渲染但参与 tooltip / info-card / search。 |

View File

@@ -21,6 +21,7 @@ This is the current Intelligent Planet documentation entry point. Docs are organ
- [Earth Satellite Footprint Policy](/home/ray/dev/linkong/planet/docs/technical/en/earth-satellite-footprint-policy.md): satellite footprint display boundaries and strategy
- [BGP Context](/home/ray/dev/linkong/planet/docs/technical/en/earth-bgp-context.md): BGP rendering, aggregation, and collector implementation in Earth
- [Earth Interactable Usage](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-usage.md): `Interactable` API, lifecycle, and integration examples
- [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md): pluggable cluster strategies, stable spherical clustering, and dynamic screen clustering boundaries
- [Earth Toolbar and Overlay Coordination](/home/ray/dev/linkong/planet/docs/technical/en/earth-toolbar-overlay-coordination.md): close matrix for toolbar buttons, search, settings, news, and layer overlays
## Frontend Implementation

View File

@@ -75,6 +75,10 @@ async def run(self, db):
Manual trigger, data clearing, and cache clearing now enter the PostgreSQL data job queue. `collection_tasks` remains the task ledger. Collectors only own `fetch -> transform -> save`; the `data_jobs.py` worker claims `collect` / `clear_data` / `clear_cache` / `earth_refresh` jobs and writes progress back. Earth layer refresh relationships live in `earth_layer_adapters.py`; do not hand-code cache invalidation or WebSocket broadcasts inside individual collectors or buttons.
Data deletion runs in batches so AIS-scale tables are not locked by one huge statement. A `clear_data` job clears `collected_data`, then source-specific AIS derived tables, and broadcasts `records_processed` as it goes; the console queue renders only user-facing text such as `Deleting data` and `Delete complete`, while internal table names remain in logs and raw task details. After AIS cleanup, the backend runs `ANALYZE ais_raw_observations` so datasource-list estimates converge quickly. The datasource directory uses PostgreSQL statistics for AIS record counts by default to avoid a cold-start `count(*)`; opening a single datasource detail row requests the exact count for that source.
The CelesTrak TLE collector prefers the complete `active` catalog. If CelesTrak returns the "GP data has not updated" HTTP 403, the backend first reuses the active raw download cache under `$PLANET_CACHE_DIR/downloads/celestrak`; if that cache is missing, it enters fallback group mode. Fallback group mode uses only currently valid public CelesTrak groups, including `starlink`, `gps-ops`, `galileo`, `glo-ops`, `beidou`, `geo`, `iridium-next`, `stations`, `visual`, `weather`, `science`, `cubesat`, `amateur`, and `last-30-days`. Disaster-recovery fallback prefers valid local group caches before touching the network, so small-group update windows or unreliable HEAD metadata do not incorrectly fail recovery. Console `Clear Data` and `Clear Cache` jobs only touch database rows, Earth layer cache, and dashboard cache; they do not remove this raw download cache.
## III. Collector List
| Collector | Data type | Content | Frequency |

View File

@@ -130,7 +130,7 @@ Responsibilities:
- Terrain tile fetch, decode, displacement, and shading
- Whole-globe land/ocean and border base overlays
The Earth surface is a stack of near-concentric shells, not a single mesh. The base sphere and HD texture overlay in `earth.js`, plus the land/ocean base in `country-boundaries.js`, need explicit radius separation. At far zoom, GPU depth precision drops; neighboring shells that are too close can z-fight and show black flicker blocks or snow. The current stable spacing is `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = 0.48`. When adding or adjusting whole-globe surface overlays, update [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md) and verify at 50% zoom.
The Earth surface is a stack of near-concentric shells, not a single mesh. The base sphere and HD texture overlay in `earth.js`, plus the land/ocean base in `country-boundaries.js`, need explicit radius separation. At far zoom, GPU depth precision drops; neighboring shells that are too close can z-fight and show black flicker blocks or snow. The current stable spacing is `landAltitudeOffset = 0.32` and `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` for the HD texture shell. Country borders, coastlines, claim lines, and country hover lines must use that same HD texture shell radius so they do not drift relative to the texture while the globe rotates. When adding or adjusting whole-globe surface overlays, update [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md) and verify at 50% zoom.
### 7. Layer Modules
@@ -313,6 +313,18 @@ If future cable, satellite, or news cruise is added, do not copy a new set of `m
- [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js)
- The business adapter pattern from [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
## View Control Feedback
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) owns the Earth zoom state, and every zoom entry point must ultimately call `setZoomLevel()` to write the camera distance. Do not write `camera.position.z` from other modules, or the zoom percentage, drag sensitivity, and Interactable clustering thresholds will diverge again.
Interactable clustering is selected per layer through `cluster.strategy`. `stable-spherical` uses discrete zoom bands and local 3D bucket clustering, so BGP, compute centers, and Earth interactables do not regroup while the globe rotates inside the same band. `dynamic-screen` keeps the projection-based behavior for high-frequency realtime layers such as vessels, and `none` disables clustering. Stable cluster dots stay rigidly aligned to their 3D centroid projection and do not participate in 2D avoidance. See [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md) for strategy configuration and tuning.
Wheel input has two paths. Traditional mouse wheels keep the 10% step and short animation, using `wheelZoomTarget` as the logical base for continuous wheel input. Trackpads and high-precision wheels use the pixel delta for continuous zoom and call `setZoomLevel()` directly instead of passing through the 10% stepped animation. The trackpad path also filters a short-window, old-direction residual delta after a real direction change so inertia tails do not pull a just-reversed zoom back in the previous direction.
The gesture capsule updates at most once every 90ms and fades after 760ms. It is view feedback, not data loading progress, and should not be written into layer loading state.
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) only calls `setZoomLevel()` and `showZoomStatusCapsule()` for pinch zoom and motion zoom. Mouse wheel and zoom-button capsule feedback should stay in `controls.js` so the same zoom feedback does not spread across modules.
## Recommended Change Approach
For future Earth changes:

View File

@@ -0,0 +1,84 @@
# Intelligent Planet Interactable Clustering
Earth interactable icons are managed by `createInteractableLayer`. Rendering still uses Three.js `Points`, and picking, hover, locked state, tooltips, and cruise focus still depend on marker `userData`; the clustering strategy only decides which markers are represented by a cluster dot.
## Strategies
`cluster.strategy` supports three modes:
- `stable-spherical`: stable spherical clustering. It clusters by local 3D positions on the globe and caches topology by zoom band. Rotation and small zoom changes within the same band only update projection and size. Use it for semi-static layers such as BGP, compute centers, and Earth interactables.
- `dynamic-screen`: dynamic screen-space clustering. This keeps the previous projection-based behavior and evaluates visible relationships per frame. Use it for high-frequency realtime layers such as vessels, or as a fallback.
- `none`: no clustering. Every marker is shown independently. Use it for low-count layers or precision-first views.
`cluster: false` is equivalent to `strategy: "none"`. If a layer enables clustering without declaring a strategy, it keeps the compatible `dynamic-screen` behavior.
## Stable Spherical
`stable-spherical` moves cluster identity from screen distance to globe distance:
- Each marker uses `icon_base_position` as its geographic anchor.
- By default, zoom levels above `2.5` force clustering off and show every marker as its original icon.
- The current zoom maps to a discrete band; rotation and small zoom changes inside the same band do not recompute topology.
- Clusters are recomputed only when the band, data revision, visibility, or filter state changes.
- A cluster centroid is computed from member 3D positions and normalized back to the globe shell, so the cluster dot stays rigidly aligned to its geographic center.
- Cluster dots do not participate in 2D avoidance, so screen-space repulsion cannot push them away from their real geographic projection.
- Band changes use a small hysteresis margin so zooming at a boundary does not repeatedly bounce between two bands.
- Newly created marker and cluster dots run a short scale + opacity ease. This is only a rendering transition; it does not change marker coordinates, picking objects, or locked state.
The stable strategy uses spherical bucket/hash neighbor lookup and must not use an all-pairs loop. Most frames only pay projection and material-size cost; topology cost is paid only on band or data changes.
## Configuration
```js
const computeCenterIconLayer = createInteractableLayer({
id: "computeCenters",
// ...
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
minCount: 2,
maxMarkersPerDot: 14,
transitionMs: 220,
bandHysteresis: 0.08,
disableAboveZoom: 2.5,
bands: [
{ key: "far", maxZoom: 1.7, distance: 15 },
{ key: "mid", maxZoom: 2.6, distance: 8 },
{ key: "near", maxZoom: 3.5, distance: 4 },
{ key: "detail", maxZoom: Infinity, distance: 0 },
],
},
});
```
Realtime layers can stay dynamic:
```js
const vesselIconLayer = createInteractableLayer({
id: "vessels",
// ...
cluster: {
strategy: "dynamic-screen",
enabled: true,
maxMarkersPerDot: 10,
},
});
```
## Tuning
- `distance` is the 3D globe-distance threshold and uses the same unit as `CONFIG.earthRadius`. Larger values cluster more aggressively.
- The farthest band usually uses a larger `distance`; the nearest detail band usually uses `0` to split clusters into original icons.
- `bandHysteresis` controls band-boundary stickiness. Too little can flicker near thresholds; too much can make band changes feel late.
- `transitionMs` controls cluster split/merge easing. Keep it around 160-260ms; longer durations can make realtime layers feel sluggish.
- `disableAboveZoom` controls the precision-view threshold. It defaults to `2.5`; above that zoom, no cluster dots are generated. Set it to `false` to disable the hard threshold.
- High-frequency realtime layers should prefer `dynamic-screen` so frequent data changes do not trigger stable topology recomputation.
- If a layer behaves poorly, switch it back to `dynamic-screen` or use `cluster: false`.
## Acceptance Checks
- Rotating the globe inside one zoom band should not cause clusters to flicker or regroup.
- Cluster dots should stay aligned with the globe-surface centroid and should not be pushed by avoidance.
- Zooming into the detail band should restore the layer's original icon textures and click behavior.
- Zoom levels above 250% should not show cluster dots.
- Realtime layers such as vessels should reflect incoming updates immediately.

View File

@@ -18,14 +18,14 @@ Note: the layer control panel order and the registration / startup load order ar
| 0 | Earth base sphere | `earth.js` | `CONFIG.earthRadius` | Surface picking fallback target | Dark base; still visible when all optional map layers are off. |
| 0.2 | Country dark tint | `country-boundaries.js` | `tintAltitudeOffset` | Raycast disabled | Used when HD texture is off. |
| 0.86 | Land/ocean base fill | `country-boundaries.js` | `landAltitudeOffset = 0.32`; ocean `#010609`, land `#080f1b` | Raycast disabled | Base map remains usable even when country borders are off; radius is separated from the base sphere to avoid far-zoom z-fighting. |
| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset = 0.48` | Surface picking target when visible | HD texture always overlays the land/ocean base fill; radius must stay above the land/ocean base and far enough from the base sphere. |
| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | Surface picking target when visible | HD texture always overlays the land/ocean base fill; radius must stay above the land/ocean base and far enough from the base sphere. |
| 1 | Atmospheric glow and clouds | `earth.js` | Atmosphere / cloud spheres | Not in normal object selection path | Cloud layer controlled by the "Cloud Layer" toggle. |
| 1 | Submarine cables | `cables.js` | `CABLE_CONFIG.line.renderOrder` | Cable picking path | Preserves existing cable layer level. |
| 1.2 | Real terrain | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` plus terrain displacement | Raycast disabled | Terrain overlays HD texture; temporarily hidden when HD texture is off, restores to prior state when re-enabled. |
| 2.05 | Grid lines | `earth.js` | `CONFIG.earthRadius + 0.14` | Raycast disabled | Low-opacity lines over HD texture. |
| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset` | Raycast disabled | Only needs to stay above HD texture. |
| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`; claim lines have no extra lift | Raycast disabled | Line geometry still has its own `renderOrder`, but it shares the exact same radius as the HD texture shell to avoid parallax while the globe rotates. |
| 2.29 | Country border hover glow | `country-boundaries.js` | Hover radius + glow offset | `depthTest: false`, raycast disabled | Additive glow to reinforce border edge and terrain hover visibility. |
| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset` | `depthTest: false`, raycast disabled | Neon red-orange hover line; China and Taiwan share the same highlight group. |
| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: false`, raycast disabled | Neon red-orange hover line; aligned with the normal borders and HD texture shell to avoid ghosting or floating; China and Taiwan share the same highlight group. |
| 3 | Satellite footprint fill / Iridium coverage ring | `satellites.js`, `iridium-footprint-adapter.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-tested; Iridium adapter fill / ring use the same renderOrder | Footprint above land / texture / terrain and country borders, below compute centers and satellites. |
| 3-5 | BGP markers and overlays | `bgp.js` | Each marker's own renderOrder | BGP picking path | Preserves existing BGP visual level. |
| 4.5 | Compute centers | `compute-centers.js` | `COMPUTE_CENTER_RENDER_ORDER` | Compute center picking path | Surface facilities, below satellites. |
@@ -52,7 +52,8 @@ The Earth surface is not a single mesh. It is a stack of near-concentric shells:
Maintenance rules:
- Do not reach first for hiding layers at far zoom. Check neighboring shell `altitudeOffset`, `renderOrder`, `depthTest`, and `depthWrite` first.
- Whole-globe overlays such as the land/ocean base and HD texture must keep explicit separation from `CONFIG.earthRadius`; the current stable values are `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = 0.48`.
- Whole-globe overlays such as the land/ocean base and HD texture must keep explicit separation from `CONFIG.earthRadius`; the current stable values are `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`.
- Country borders, coastlines, claim lines, and country hover lines must use `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET`, exactly matching the HD texture shell. Do not give border lines an independent lower or higher offset, because globe rotation will make the lines appear to drift relative to the surface texture.
- Any new whole-globe or near-whole-globe surface overlay must be screenshot-verified at 50% zoom and at common close zooms, with no black blocks, snow, flicker, or obvious floating.
- If these radii change, update this document and the intent around the constants in `frontend/public/earth/js/constants.js`.

View File

@@ -92,6 +92,33 @@ The Admin sidebar theme switcher still reuses shared [SegmentedControl.tsx](/hom
- Dark mode uses the same dark base, `#202938` slider, and dark external shadow semantics as Docs.
- Product code only hides the text label and keeps icon + tooltip behavior; it should not recreate the private slider DOM.
## Admin Status Colors
Admin status labels should use [StatusText](/home/ray/dev/linkong/planet/frontend/src/admin/patterns/patterns.tsx) or [Badge](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/badge.tsx). Color variables come from [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin/styles.css). New states should map to an existing tone instead of adding page-local hex colors.
`StatusText` is an indicator-light pill: the pill background and border stay on the component base color, while only the dot and text use the status color. `Badge` does not carry the indicator-light meaning, so it may use a light same-tone background and border for stronger hierarchy.
| Tone | Color variable | Meaning | Examples |
| --- | --- | --- | --- |
| `success` | `--an-success` | available, successful, connected, enabled | log source `Available`, collection `Success` |
| `warning` | `--an-warning` | needs attention but is not necessarily failed | missing log file, degraded or skipped state |
| `danger` | `--an-danger` | failed, unavailable, permission/connection error | `Docker unavailable`, endpoint failure |
| `info` / `running` | `--an-info` | in progress, syncing, informational | `Following`, `Syncing` |
| `neutral` | `--an-muted` | empty, disabled, not reported yet, unknown | `No reports yet`, disabled |
| `ai` | fixed purple | AI-specific emphasis | AI generation or model action |
The log source list follows the same rule: `ok` is success; `empty` means the source exists but has not reported yet and is neutral; `missing` means an expected log file is not present and is warning; `docker_unavailable` / `source_unavailable` are danger.
## Admin Runtime Logs
Admin runtime errors are reported through [runtimeLogs.ts](/home/ray/dev/linkong/planet/frontend/src/admin/runtimeLogs.ts) to `/api/v1/system/logs/admin-client`. The reporter only runs on Admin routes and skips `/earth`, `/docs`, login, and registration pages so public-page noise does not enter the Admin client log source.
[AdminErrorBoundary.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/AdminErrorBoundary.tsx) catches React render failures and reuses the same reporter; global `error` and `unhandledrejection` events use that channel as well. Reporting failures must stay silent, because the logging path must not create another frontend error.
The Logs page follows log increments through the `/ws` `logs_tail` channel. File logs and database logs are both normalized into line events by the backend. When adding a new log source, wire it through the backend source registry and tail manager instead of adding a page-local poller.
The datasource task queue `View Logs` action opens `/logs?source=system-db&search=task_id=<id>`. Backend database-log search indexes must expand simple JSON context fields into `key=value` aliases such as `task_id=26906` and `datasource_id=20`, so historical task logs remain discoverable without rerunning the task.
## Current Shared Components
### 1. `Scrollbar`

View File

@@ -172,6 +172,10 @@ After selecting a task, the page shows the effective prompt, whether it is custo
The legacy link `/settings?tab=ai` redirects to `/ai?tab=providers`.
## Datasources and Task Logs
`/datasources` is the datasource directory. Built-in sources can be filtered by product domain, level, enabled state, latest run state, collected-data state, and keyword. With no rows selected the main action triggers all matching sources; selecting rows changes it to `Trigger Selected N`. The queue button opens a grouped task panel for running, completed, failed, and skipped work. Failed rows can be retried, completed rows can jump back to their datasource detail, and each task can open `/logs` filtered by its task id.
## System Settings
`/settings` manages system-level configuration. Sub-tabs:

View File

@@ -75,7 +75,7 @@ Cleanup order and boundaries:
- Docker cleanup targets resources whose Compose project is `planet`, plus the explicit volumes `planet_postgres_data`, `planet_redis_data`, `postgres_data`, and `redis_data`; do not delete unlabeled volumes by a broad `planet_*` pattern, because another local project could own them.
- Local build state removes `.venv`, frontend `node_modules` / `dist`, Planet state, and scattered Python / Vite cache directories. `$PLANET_CACHE_DIR/downloads` is preserved so upstream raw downloads such as CelesTrak can survive database resets and local rebuild cleanup.
After the reset, run `./planet.sh init` again to recreate tables and default seed data. Old collected records are not restored, and Earth OOBE is evaluated from the backend's real collection state on the next visit. When CelesTrak later returns its "GP data has not updated" HTTP 403, the backend first reuses the preserved download cache to repopulate the database; if no cache exists, wait for the next CelesTrak update window or use Space-Track as a fallback.
After the reset, run `./planet.sh init` again to recreate tables and default seed data. Old collected records are not restored, and Earth OOBE is evaluated from the backend's real collection state on the next visit. When CelesTrak later returns its "GP data has not updated" HTTP 403, the backend first reuses the preserved download cache to repopulate the database; if no active cache exists, it tries valid CelesTrak fallback group caches; if no download cache exists at all, wait for the next CelesTrak update window or use Space-Track. Datasource `Clear Data` and `Clear Cache` actions in the console do not delete `$PLANET_CACHE_DIR/downloads/celestrak`.
## Health Check
@@ -286,6 +286,8 @@ bun run build
Do not use `npm run ...`. In the WSL / Windows mixed environment Bun avoids Node/npm path inconsistencies.
`./planet.sh start` / `init` now runs `bun install` before startup instead of only checking whether the Vite entry file exists. This keeps new devices, cleaned `node_modules`, and lockfile changes synchronized before the console loads, avoiding dynamic-import 500s caused by missing frontend dependencies.
Validate the frontend build:
```bash

View File

@@ -21,6 +21,7 @@
- [智能星球卫星覆盖策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-satellite-footprint-policy.md):卫星 footprint 的显示边界和策略
- [BGP 态势上下文](/home/ray/dev/linkong/planet/docs/technical/zh/earth-bgp-context.md)BGP 在智能星球中的渲染、聚合和观测站实现
- [智能星球可交互图标接入](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)`Interactable` 的接口、生命周期和接入示例
- [智能星球可交互图标聚类策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-clustering.md):可插拔 cluster strategy、稳定球面聚类和动态屏幕聚类的适用边界
- [智能星球工具栏与浮层协同](/home/ray/dev/linkong/planet/docs/technical/zh/earth-toolbar-overlay-coordination.md):工具栏按钮与搜索、设置、新闻、图层浮层的关闭矩阵
## 前端技术实现

View File

@@ -75,6 +75,10 @@ async def run(self, db):
手动触发、删除数据、清理缓存现在统一进入 PostgreSQL 数据作业队列,任务账本仍是 `collection_tasks`。采集器只负责 `fetch -> transform -> save`,由 `data_jobs.py` worker 领取 `collect` / `clear_data` / `clear_cache` / `earth_refresh` 任务并回写进度。Earth 图层刷新关系集中在 `earth_layer_adapters.py`,不要再在单个采集器或按钮里手写缓存失效和 WebSocket 广播。
删除数据任务按批次执行,避免 AIS 这类千万级表一次性锁表。`clear_data` 会先清 `collected_data`,再按来源清理 AIS 衍生表,并持续广播 `records_processed`;前端任务队列只展示“正在删除数据 / 删除完成”,内部表名只保留在日志和原始任务详情。删除结束后后端会 `ANALYZE ais_raw_observations`,让数据源列表的估算指标尽快收敛。数据源目录页默认使用 PostgreSQL 统计信息估算 AIS 大表记录数,避免冷启动做 `count(*)`;打开单条详情时再用精确计数校准当前数据源。
CelesTrak TLE 采集优先拉取完整 `active` 目录。如果 CelesTrak 返回“本轮 GP 数据未更新”的 403后端先复用 `$PLANET_CACHE_DIR/downloads/celestrak` 下的 active 原始下载缓存;没有 active 缓存时进入 fallback group 模式。fallback group 只使用 CelesTrak 当前公开有效的分组,例如 `starlink``gps-ops``galileo``glo-ops``beidou``geo``iridium-next``stations``visual``weather``science``cubesat``amateur``last-30-days`。救灾 fallback 会优先使用本地有效 group 缓存,避免 CelesTrak 小分组在未更新窗口或 HEAD 元数据异常时被误判失败。控制台的“删除数据库”和“清理缓存”只处理数据库记录、Earth layer cache 和 dashboard cache不删除该原始下载缓存。
## 三、采集器列表
| 采集器 | 数据类型 | 数据内容 | 采集频率 |

View File

@@ -138,7 +138,7 @@ Browser Camera provider 的手势识别管线在 [motion-browser-provider.js](/h
- terrain tile 拉取、解码、位移、着色
- 海陆基座与国界底图的整球 overlay
智能星球地表是多层近似同心球,不是单一 mesh。`earth.js` 的基座球、高清材质 overlay、云层/大气,以及 `country-boundaries.js` 的海陆基座都需要明确半径间距。远距视图下 GPU 深度精度会下降,相邻 shell 过近会 z-fighting表现为黑色闪烁块或雪花。当前稳定策略是让海陆基座使用 `landAltitudeOffset = 0.32`,高清材质使用 `textureOverlayAltitudeOffset = 0.48`后续新增或调整整球地表 overlay 时,必须同步检查 [智能星球渲染图层顺序](/home/ray/dev/linkong/planet/docs/technical/zh/earth-render-layer-order.md),并在 50% 缩放视图验证。
智能星球地表是多层近似同心球,不是单一 mesh。`earth.js` 的基座球、高清材质 overlay、云层/大气,以及 `country-boundaries.js` 的海陆基座都需要明确半径间距。远距视图下 GPU 深度精度会下降,相邻 shell 过近会 z-fighting表现为黑色闪烁块或雪花。当前稳定策略是让海陆基座使用 `landAltitudeOffset = 0.32`,高清材质使用 `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`国界线、coastline、claim 线和国界 hover 线也必须使用同一个高清材质壳半径,避免转动地球时相对高清贴图产生视差漂浮感。后续新增或调整整球地表 overlay 时,必须同步检查 [智能星球渲染图层顺序](/home/ray/dev/linkong/planet/docs/technical/zh/earth-render-layer-order.md),并在 50% 缩放视图验证。
### 7. 图层模块
@@ -381,23 +381,24 @@ asset 图标大小由 `Interactable` 的 `icon.fitSize` 控制。SVG / 图片文
`Interactable` 不再把图标本体额外抬离业务高度。`altitudeOffset` 就是 marker、hover glow、locked glow 和 picking 共同使用的地表高度;这样船只图标会继续贴着船只轨迹线,不会因为单独抬高显示位置而显得漂浮。后续如果要解决边缘 glow 裁切,应优先考虑 glow 纹理、overlay 尺寸或图层专属特效,而不是把通用图标层整体抬高。
跨 Interactable 的同坐标避让也在公共层处理。每个 marker 会保留 `icon_base_position` 作为业务原始位置;当多个 Interactable marker 归入同一个经纬度 key 时,公共层会把它们沿地表切平面排成小圈,并刷新已创建的 `THREE.Points` geometry。这样视觉位置和屏幕空间 picking 位置一致,不需要业务层再单独判断“算力中心和 BGP 事件重叠”这类场景
跨 Interactable 的同坐标关系也在公共层记录,但真实位置必须始终以 `icon_base_position` 为准。缩放、避让、聚合和后续 spiderfy 展开都只能改变屏幕表现,不能写回 `marker.position``THREE.Points` 里的业务锚点;巡航定位、详情卡、搜索定位和 picking 返回对象都必须落回真实经纬度。多个图标归入同一个经纬度 key 时,公共层只写 `icon_avoidance_*` 元数据,供业务层弱化 halo 或显示聚合提示;真正的低缩放聚合应通过独立 cluster glyph / screen layout 层实现,而不是把对象沿地表切平面挪开
`Interactable` 的单点显示只由全局地图缩放决定170% 及以下强制显示小圆点,超过 170% 显示原图标。cluster 现在由 `cluster.strategy` 决定:`stable-spherical` 使用离散 zoom band 和 3D 球面分桶BGP、算力中心和 Earth interactable 在同一 band 内旋转或细微缩放时不会重新计算聚合拓扑;`dynamic-screen` 保留屏幕空间聚类,适合船只这类实时高频图层;`none` 关闭聚类。稳定球面聚类的 cluster 圆点刚性落在成员 3D 质心投影上,不参与 2D 避让避免缩放时被推离真实地理位置。cluster 圆点大小随包含对象数量增长,数量过多时按稳定地理顺序拆成多个较小圆点;数量默认只在 hover tooltip 中显示。这个过程只设置 `icon_cluster_*` 展示元数据和重建渲染 Points不改变每个 marker 的真实经纬度。
接口细节、生命周期和接入示例见:
- [智能星球可交互图标接入](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)
- [智能星球可交互图标聚类策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-clustering.md)
### 视角控制反馈
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) 统一维护 Earth 缩放状态。滚轮缩放、缩放按钮和触屏双指捏合最终都会更新 `zoomLevel`,并通过 `showZoomStatusCapsule()` 显示当前缩放比例:
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) 统一维护智能星球缩放状态,所有入口最终都必须调用 `setZoomLevel()` 写相机距离。不要在其他模块直接写 `camera.position.z`,否则缩放百分比、拖拽灵敏度和 Interactable 聚合阈值会再次分叉。
```javascript
showGestureStatusMessage(`缩放 ${Math.round(zoomLevel * 100)}%`, "info");
```
滚轮输入分两类处理:传统鼠标滚轮保留 10% 档位和短动画,并用 `wheelZoomTarget` 作为连续滚动的逻辑基准;触控板 / 高精度滚轮按 pixel delta 连续缩放,直接调用 `setZoomLevel()`,不走 10% 档位动画。触控板路径还会过滤反向后的短窗口旧方向小 residual delta避免惯性尾巴把用户刚反向的缩放又拉回旧方向。
该提示每 90ms 最多更新一次,显示 760ms 后淡出。它是视角反馈,不是数据加载进度,也不应该写进图层 loading 状态。
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 只负责在双指捏合缩放时调用 `setZoomLevel()``showZoomStatusCapsule()`。鼠标滚轮与缩放按钮的胶囊提示应继续放在 `controls.js`,避免同一种缩放反馈散落在多个模块。
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 只负责在双指捏合缩放和动捕缩放时调用 `setZoomLevel()``showZoomStatusCapsule()`。鼠标滚轮与缩放按钮的胶囊提示应继续放在 `controls.js`,避免同一种缩放反馈散落在多个模块。
拖拽地球的旋转灵敏度会根据当前缩放连续衰减,而不是按某个缩放阈值分段:

View File

@@ -0,0 +1,84 @@
# 智能星球可交互图标聚类策略
智能星球的可交互图标由 `createInteractableLayer` 统一管理。图标仍然使用 Three.js `Points` 渲染拾取、hover、locked、tooltip 和巡航焦点继续依赖 marker 的 `userData`;聚类策略只决定“哪些 marker 被合成一个 cluster dot”。
## 策略
`cluster.strategy` 支持三种模式:
- `stable-spherical`:稳定球面聚类。按地球局部 3D 坐标聚合,并用 zoom band 缓存拓扑;旋转和同档缩放只更新投影和尺寸,不重算谁和谁聚在一起。适合 BGP、超算/GPU 中心和 Earth interactable 这类半静态图层。
- `dynamic-screen`:动态屏幕聚类。保留原来的屏幕空间聚类逻辑,每帧按可见投影关系判断。适合船舶等高频实时图层,也可作为稳定策略的回退。
- `none`:不聚类。所有 marker 独立显示,适合低数量或需要精确展示的图层。
`cluster: false` 等价于 `strategy: "none"`。未显式声明 `strategy` 时,保持兼容行为:开启聚类的旧图层继续走 `dynamic-screen`
## Stable Spherical
`stable-spherical` 的核心是把聚类身份从屏幕距离迁到球面距离:
- 每个 marker 使用 `icon_base_position` 作为真实地理锚点。
- 默认 zoom 超过 `2.5` 时强制关闭聚类,所有 marker 展开为原图标。
- 当前 zoom 只映射到离散 band同一 band 内旋转地球或细微缩放不会重算拓扑。
- 跨 band、数据变更、图层显隐变化时才重新计算 cluster。
- cluster 质心由成员 3D 坐标平均后 normalize 回球壳半径,因此 cluster dot 刚性贴在地理质心投影上。
- cluster dot 不参与 2D 避让,避免被屏幕排斥推离真实地理位置。
- band 切换带有少量 hysteresis避免缩放停在临界点时在两个 band 之间来回跳。
- 新生成的 marker / cluster dot 会执行短 scale + opacity ease这只是渲染过渡不改变 marker 的真实经纬度、拾取对象或 locked 状态。
稳定策略使用球面 bucket/hash 邻域查询,禁止用全量双循环。这样大多数帧只承担投影与材质尺寸更新,聚类成本只在 band 或数据版本变化时支付。
## 配置示例
```js
const computeCenterIconLayer = createInteractableLayer({
id: "computeCenters",
// ...
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
minCount: 2,
maxMarkersPerDot: 14,
transitionMs: 220,
bandHysteresis: 0.08,
disableAboveZoom: 2.5,
bands: [
{ key: "far", maxZoom: 1.7, distance: 15 },
{ key: "mid", maxZoom: 2.6, distance: 8 },
{ key: "near", maxZoom: 3.5, distance: 4 },
{ key: "detail", maxZoom: Infinity, distance: 0 },
],
},
});
```
实时层可以保留动态策略:
```js
const vesselIconLayer = createInteractableLayer({
id: "vessels",
// ...
cluster: {
strategy: "dynamic-screen",
enabled: true,
maxMarkersPerDot: 10,
},
});
```
## 调参原则
- `distance` 是球面 3D 距离阈值,单位与 `CONFIG.earthRadius` 一致。值越大,越容易聚合。
- 最远 band 用较大的 `distance` 降低视觉密度;最近 band 通常设为 `0`,让图标完全解散。
- `bandHysteresis` 控制 band 边界滞回。值太小容易临界闪烁,值太大会让切换略显迟钝。
- `transitionMs` 控制聚散过渡时间。建议保持在 160-260ms过长会让实时层显得拖泥带水。
- `disableAboveZoom` 控制精细查看阈值。默认 `2.5`,超过后不再生成 cluster设为 `false` 可关闭这个硬阈值。
- 高实时性图层优先用 `dynamic-screen`,避免数据频繁变更时触发稳定策略的拓扑重算。
- 若某图层出现异常,可临时切回 `dynamic-screen``cluster: false`
## 验收重点
- 同一 zoom band 内旋转地球cluster 不应闪烁或重新聚散。
- cluster dot 应跟随地球表面质心,不被避让逻辑推开。
- 放大到 detail band 后,应恢复该图层原本的图标纹理和点击行为。
- zoom 超过 250% 后不应再显示 cluster dot。
- vessel 等实时层更新后,图标和 cluster 应立即反映最新数据。

View File

@@ -19,14 +19,14 @@
| 0 | 智能星球基座球 | `earth.js` | `CONFIG.earthRadius` | 地表拾取兜底目标 | 深色基座,所有可选地图层关闭时仍可见。 |
| 0.2 | 国界暗色 tint | `country-boundaries.js` | `tintAltitudeOffset` | 禁用 raycast | 高清材质关闭时使用。 |
| 0.86 | 海陆基座填充 | `country-boundaries.js` | `landAltitudeOffset = 0.32`; 海洋 `#010609`,陆地 `#080f1b` | 禁用 raycast | 即使国界线关闭,基座地图仍保持可用;半径与基座球拉开以避免远距 z-fighting。 |
| 0.96 | 高清 Earth 材质 | `earth.js` | `textureOverlayAltitudeOffset = 0.48` | 可见时作为地表拾取目标 | 高清材质始终压过海陆基座填充;半径必须高于海陆基座并与基座球保持足够间距。 |
| 0.96 | 高清 Earth 材质 | `earth.js` | `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | 可见时作为地表拾取目标 | 高清材质始终压过海陆基座填充;半径必须高于海陆基座并与基座球保持足够间距。 |
| 1 | 大气辉光和云图 | `earth.js` | 大气 / 云层球 | 不走普通对象选择路径 | 云图由“大气云图”图层开关控制。 |
| 1 | 海缆 / 登陆点 | `cables.js` | 海缆线和登陆点都使用 `renderOrder = 1`;半径偏移都为 `0.2`;登陆点是专用 `THREE.Sprite` 黄色扁平球 | 海缆走海缆拾取路径;登陆点 `depthTest: false` 保持球体完整,并用相机到球心的球体遮挡判断避免背面穿透 | 登陆点和海缆同层贴地,避免地表设施层的凌空感。 |
| 1.2 | 真实地形 | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` 加地形位移 | 禁用 raycast | 地形压过高清材质;高清材质关闭时临时隐藏,重新开启后恢复原状态。 |
| 2.05 | 经纬线 | `earth.js` | `CONFIG.earthRadius + 0.14` | 禁用 raycast | 低透明度显示在高清材质上。 |
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset = 0.115` | `depthTest: true`,禁用 raycast | 线层使用独立 line geometry 与 `renderOrder` 控制;地形 `depthWrite: false`,所以地形开启时仍可见。 |
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`claim 线不再额外抬高 | `depthTest: true`,禁用 raycast | 线层使用独立 line geometry 与 `renderOrder` 控制,但半径与高清材质壳完全一致,避免转动地球时与高清贴图出现视差。 |
| 2.29 | 国界 hover 光晕 | `country-boundaries.js` | hover 半径加 glow 偏移 | `depthTest: false`,禁用 raycast | 用 additive 光晕增强交界边和地形开启时的 hover 可见性。 |
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset = 0.115` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;与普通国界线同源半径对齐,避免重影;中国和中国(台湾)共享高亮组。 |
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;与普通国界线和高清材质同源半径对齐,避免重影和漂浮感;中国和中国(台湾)共享高亮组。 |
| 3 | 卫星 footprint 填充 / Iridium coverage ring | `satellites.js`, `iridium-footprint-adapter.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-testedIridium adapter 的 fill / ring 也使用同一 renderOrder | Footprint 在 land / texture / terrain 和国界线之上,但在算力中心和卫星之下。 |
| 3-4.5 | BGP 观测站、事件扩散圈和事件 marker | `bgp.js`, `interactable.js` | BGP 观测站和事件 marker 均使用 `Interactable` 批量 `THREE.Points`;事件 marker 使用 `BGP_EVENT_RENDER_ORDER = 4.5`;观测站主图标使用 `BGP_COLLECTOR_RENDER_ORDER = 4.4``BGP_CONFIG.collectorAltitudeOffset = 0.2`;事件 overlay 进入 `bgp-event-overlay-layer`;观测站 halo 和覆盖扇形进入 `bgp-collector-radar-layer` | BGP 事件和观测站都通过 `Interactable` 屏幕空间 picking并参与同坐标避让 | BGP 观测站主图标与船只同层BGP 事件与算力中心同层;向外扩散圈、观测站雷达/覆盖动画继续由 BGP 业务逻辑驱动。 |
| 4.3 | AIS 船只轨迹线 | `vessels.js` | `VESSEL_RENDER_ORDER - 0.1``CONFIG.earthRadius + VESSEL_CONFIG.track.altitudeOffset` | 跟随船只显隐,不单独参与拾取 | 选中船只后显示最近轨迹,低于船只 marker。 |
@@ -55,7 +55,8 @@ Earth 的地表不是单一 mesh而是多层近似同心球基座球、海
维护规则:
- 不要用“远距隐藏图层”作为第一反应;先检查相邻 shell 的 `altitudeOffset``renderOrder``depthTest``depthWrite`
- 海陆基座和高清材质这类整球 overlay 必须与 `CONFIG.earthRadius` 保持明确间距;当前稳定值为 `landAltitudeOffset = 0.32``textureOverlayAltitudeOffset = 0.48`
- 海陆基座和高清材质这类整球 overlay 必须与 `CONFIG.earthRadius` 保持明确间距;当前稳定值为 `landAltitudeOffset = 0.32``textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`
- 国界线、coastline、claim 线和国界 hover 线必须使用 `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET`,与高清材质壳完全同半径;不要再用低于或高于高清材质的独立线层 offset否则转动地球时会产生相对地表的视差漂浮感。
- 新增整球或近整球地表 overlay 时,必须在 50% 缩放和常用近距视图各截一次图,确认没有黑块、雪花、闪烁,也没有明显漂浮感。
- 如果必须调整这些半径,需同步更新本文和 `frontend/public/earth/js/constants.js` 的注释/常量意图。

View File

@@ -114,7 +114,7 @@ curl http://localhost:8010/health
如果 `ss -ltnp` 显示前端已经监听 `0.0.0.0:3000`,但 Windows PowerShell 中 `Test-NetConnection <Windows局域网IP> -Port 3000` 仍失败,问题通常不在 Vite 或 `.zshrc`,而是在 Windows 侧端口占用、旧 `portproxy` 或防火墙。
`./planet.sh start --allow-lan` 会直接开放 `3000` / `8000` / `8010`,并在启动前检测端口、旧 `portproxy` 和 Windows 防火墙规则。端口被 Windows 侧 listener 占用时,脚本会请求管理员 PowerShell 清理;缺少入站放行时,也会触发一次 UAC 管理员 PowerShell 请求来自动创建。若自动请求被取消,可以手动清理:
`./planet.sh start --allow-lan` 会直接开放 `3000` / `8000` / `8010`,并在启动前检测端口、旧 `portproxy` 和 Windows 防火墙规则。端口被 Windows 侧 listener 占用时,脚本会请求管理员 PowerShell 清理;缺少入站允许规则时,也会触发一次 UAC 管理员 PowerShell 请求来自动创建。若自动请求被取消,可以手动清理:
```powershell
netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
@@ -128,7 +128,7 @@ New-NetFirewallRule -DisplayName "WSL Planet 8010" -Direction Inbound -Action Al
局域网设备访问 Windows 对外端口,例如 `http://<Windows局域网IP>:3000/earth`
如果 `wslinfo --networking-mode` 输出 `mirrored`,还需要检查 Hyper-V firewall。普通 Windows 防火墙规则存在时Hyper-V firewall 仍可能拦截外部设备进入 WSL。管理员 PowerShell 中按端口放行
如果 `wslinfo --networking-mode` 输出 `mirrored`,还需要检查 Hyper-V firewall。普通 Windows 防火墙规则存在时Hyper-V firewall 仍可能拦截外部设备进入 WSL。管理员 PowerShell 中按端口开放访问
```powershell
New-NetFirewallHyperVRule -Name "Planet-Frontend-3000" -DisplayName "Planet Frontend 3000" -Direction Inbound -VMCreatorId "{40E0AC32-46A5-438A-A0B2-2B479E8F2E90}" -Protocol TCP -LocalPorts 3000 -Action Allow

View File

@@ -92,6 +92,33 @@ Admin 侧栏底部主题切换继续复用共享 [SegmentedControl.tsx](/home/ra
- dark 下使用与 Docs 一致的深色底座、`#202938` slider 和深色外投影。
- 业务侧只隐藏文字 label 并保留 icon + tooltip不重写 slider DOM。
## 控制台状态颜色
Admin 的状态标签统一走 [StatusText](/home/ray/dev/linkong/planet/frontend/src/admin/patterns/patterns.tsx) 或 [Badge](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/badge.tsx),颜色变量来自 [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin/styles.css)。新增状态时不要在页面里临时写 hex先判断语义再映射到现有 tone。
`StatusText` 是带圆点的指示灯:胶囊背景和边框保持组件原色,只让圆点和文字变成状态色。`Badge` 不带指示灯语义,可以使用同 tone 的浅色背景和边框强化信息层级。
| Tone | 颜色变量 | 语义 | 示例 |
| --- | --- | --- | --- |
| `success` | `--an-success` | 可用、成功、已连接、已启用 | 日志源 `可用`、采集 `成功` |
| `warning` | `--an-warning` | 需要关注但不一定失败 | 日志文件 `暂无日志`、降级或跳过 |
| `danger` | `--an-danger` | 失败、不可用、权限/连接错误 | `Docker 不可用`、接口失败 |
| `info` / `running` | `--an-info` | 进行中、同步中、普通信息态 | `跟随中``同步中` |
| `neutral` | `--an-muted` | 空态、停用、未上报、未知 | `暂无上报``停用` |
| `ai` | 固定紫色 | AI 相关突出态 | AI 生成、模型动作 |
日志源列表遵循同一规则:`ok` 显示 success`empty` 表示来源存在但还没有上报,显示 neutral`missing` 表示期望的日志文件暂时不存在,显示 warning`docker_unavailable` / `source_unavailable` 显示 danger。
## 控制台运行时日志
控制台运行时错误由 [runtimeLogs.ts](/home/ray/dev/linkong/planet/frontend/src/admin/runtimeLogs.ts) 统一上报到 `/api/v1/system/logs/admin-client`。它只在控制台路由内启用,跳过 `/earth``/docs`、登录和注册页面,避免公开页面噪声进入控制台日志源。
[AdminErrorBoundary.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/AdminErrorBoundary.tsx) 捕获 React 渲染错误并复用同一上报函数;全局 `error``unhandledrejection` 也进入该通道。上报失败必须静默处理,不能因为日志系统异常再制造新的前端错误。
日志页通过 `/ws``logs_tail` channel 跟随日志增量;文件日志和数据库日志都由后端统一转换成行事件。新增日志源时优先接入后端 source registry 和 tail manager不要在日志页写独立轮询器。
数据源任务队列的“查看日志”入口跳转到 `/logs?source=system-db&search=task_id=<id>`。后端数据库日志搜索索引必须把 JSON context 中的简单字段同时展开为 `key=value` 别名,例如 `task_id=26906``datasource_id=20`,这样历史任务日志不依赖重新执行任务也能被精确查到。
## 当前共享组件
### 1. `Scrollbar`

View File

@@ -236,7 +236,7 @@ Base URL 输入框尾端的插头图标会触发连接测试。测试通过会
## 数据探索
- `/datasources`:数据源目录。`内置源` 支持按产品域、层级、启用状态、最近执行状态、是否已有采集数据和关键词筛选;未勾选时主按钮显示“触发全部”,勾选多行后会变成“触发已选 N”并只提交所选数据源。右上角队列按钮空态显示队列图标有任务时显示纯圆环总进度点击后打开队列浮层按运行中、完成、失败和跳过分组失败项可重试完成项可跳到详情。`实时源` 面向 AISStream / WebSocket 长连接,展示连接健康、累计入库、时间窗统计和启动 / 停止 / 重连操作。接口、凭证、请求头的编辑统一在 `/collection-management` 的"采集器"。
- `/datasources`:数据源目录。`内置源` 支持按产品域、层级、启用状态、最近执行状态、是否已有采集数据和关键词筛选;未勾选时主按钮显示“触发全部”,勾选多行后会变成“触发已选 N”并只提交所选数据源。右上角队列按钮空态显示队列图标有任务时显示纯圆环总进度点击后打开队列浮层按运行中、完成、失败和跳过分组失败项可重试完成项可跳到详情或按任务编号打开系统日志`实时源` 面向 AISStream / WebSocket 长连接,展示连接健康、累计入库、时间窗统计和启动 / 停止 / 重连操作。接口、凭证、请求头的编辑统一在 `/collection-management` 的"采集器"。
- `/data`:采集后数据表,适合排查"数据是否已经进入系统"、"更新时间是否符合预期"、"某个数据源是否产出有效记录"
- `/bgp`BGP 专题页面,列表 + 详情 + 研判,与智能星球的 BGP 图层互补
- `/alerts/system``/alerts/bgp``/alerts/situational`系统、BGP、态势告警

View File

@@ -75,7 +75,7 @@
- Docker 清理只针对 Compose project 为 `planet` 的资源,以及显式列出的 `planet_postgres_data``planet_redis_data``postgres_data``redis_data`;不要按 `planet_*` 模式删除没有 label 的 volume避免误删同机其他项目。
- 本地编译状态会删除 `.venv`、前端 `node_modules` / `dist`、Planet state以及散落的 Python / Vite 缓存目录;`$PLANET_CACHE_DIR/downloads` 会保留,用于保存 CelesTrak 这类受上游下载窗口限制的原始文件缓存。
重置后重新执行 `./planet.sh init` 会重建表和默认数据,但不会恢复旧采集结果;首次进入 Earth 时 OOBE 会重新按后端真实采集状态判断。触发 CelesTrak 采集时,如果上游返回“本轮 GP 数据未更新”的 403后端会优先用保留的下载缓存重新写入数据库如果下载缓存也不存在,只能等待 CelesTrak 下一次更新窗口或使用 Space-Track 作为 fallback
重置后重新执行 `./planet.sh init` 会重建表和默认数据,但不会恢复旧采集结果;首次进入 Earth 时 OOBE 会重新按后端真实采集状态判断。触发 CelesTrak 采集时,如果上游返回“本轮 GP 数据未更新”的 403后端会优先用保留的下载缓存重新写入数据库如果 active 缓存不存在,会尝试有效 CelesTrak 分组缓存作为 fallback如果下载缓存也不存在只能等待 CelesTrak 下一次更新窗口或使用 Space-Track。控制台里的数据源“删除数据库”和“清理缓存”不会删除 `$PLANET_CACHE_DIR/downloads/celestrak`
## 健康检查
@@ -150,7 +150,7 @@ Planet 开发环境建议使用 WSL2。WSL1 下网络、文件系统和进程模
wsl --set-version Ubuntu 2
```
`--allow-lan` 会让前端、后端和 AI Provider 直接对开发机开放:前端 `3000`、后端 `8000`、AI Provider `8010`。脚本启动前会检查这三个端口;如果 WSL/Linux 侧无法释放端口,并检测到 Windows 侧 listener 或旧 `portproxy`,会请求管理员 PowerShell 清理。WSL 中运行时Windows 本机一般可以通过 `localhost` 访问,局域网其他机器访问 Windows 局域网 IP 时还需要 Windows 防火墙放行
`--allow-lan` 会让前端、后端和 AI Provider 直接对开发机开放:前端 `3000`、后端 `8000`、AI Provider `8010`。脚本启动前会检查这三个端口;如果 WSL/Linux 侧无法释放端口,并检测到 Windows 侧 listener 或旧 `portproxy`,会请求管理员 PowerShell 清理。WSL 中运行时Windows 本机一般可以通过 `localhost` 访问,局域网其他机器访问 Windows 局域网 IP 时还需要 Windows 防火墙允许访问
建议按顺序排查:
@@ -162,7 +162,7 @@ curl http://localhost:8010/health
ss -ltnp | grep -E ':3000|:8000|:8010'
```
如果服务已经启动但局域网 IP 仍访问失败,优先清理旧 `portproxy` 并确认 Windows 防火墙放行。脚本会自动检测并请求管理员 PowerShell 处理;自动请求被取消时,手动兜底命令如下:
如果服务已经启动但局域网 IP 仍访问失败,优先清理旧 `portproxy` 并确认 Windows 防火墙允许访问。脚本会自动检测并请求管理员 PowerShell 处理;自动请求被取消时,手动兜底命令如下:
```powershell
netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
@@ -286,6 +286,8 @@ bun run build
不要使用 `npm run ...`。项目在 WSL / Windows 混合环境优先依赖 Bun避免 Node/npm 路径差异。
`./planet.sh start` / `init` 会在启动前执行一次 `bun install`,而不是只检查 Vite 入口文件是否存在。这样新设备、清过 `node_modules` 的环境或 lockfile 已变更的环境,都能在进入控制台前同步前端依赖,避免动态 import 因缺失依赖返回 500。
验证前端构建:
```bash

View File

@@ -16,12 +16,16 @@
## Current Version
- `main` 当前主线历史推导到:`0.16.5`
- `dev` 当前开发分支历史推导到:`0.66.2`
- `dev` 当前开发分支历史推导到:`0.68.1`
## Timeline
| Version | Type | Branch | Commit | Summary |
| --- | --- | --- | --- | --- |
| `0.68.1` | bugfix | `dev` | `pending` | 修复 CelesTrak fallback group/cache 恢复链路,并让数据源任务日志可按 task_id / datasource_id 搜索 |
| `0.68.0` | feature | `dev` | `pending` | 新增数据源任务队列实时指标、AIS 大表分批删除和智能星球可插拔聚类策略,并让新设备启动前同步前端依赖 |
| `0.67.0` | feature | `dev` | `pending` | 新增控制台日志实时跟随和运行时错误上报,重构智能星球 Interactable 聚合、wheel 缩放输入、国界壳半径和开发脚本锁文件保护 |
| `0.66.3` | bugfix | `dev` | `pending` | 补上 Admin utility module 并放开前端源码 lib 例外,修复控制台动态导入 500 与 Mermaid 包解析失败 |
| `0.66.2` | bugfix | `dev` | `pending` | 统一智能星球、控制台和文档的中文显示命名,清理 Docs catalog、使用手册、控制台入口和智能星球设置中的中英混排文案 |
| `0.66.1` | bugfix | `dev` | `pending` | CelesTrak active 限频时复用持久下载缓存并完整 fallback groupdestroy 保留原始下载缓存,同时修复采集失败 toast 重复弹出 |
| `0.66.0` | feature | `dev` | `pending` | Admin 正式化为唯一控制台,新增数据作业/outbox 与 Earth interactables 管线,补齐 AI/采集日志,修复 CelesTrak 完整 active 目录采集和内置源启停判断 |

View File

@@ -1,6 +1,6 @@
{
"name": "planet-frontend",
"version": "0.66.2",
"version": "0.68.1",
"private": true,
"packageManager": "bun@1",
"dependencies": {

View File

@@ -2822,6 +2822,7 @@ label.is-disabled.earth-mobile-settings-card {
display: flex;
align-items: center;
gap: 12px;
min-width: 0;
}
.earth-settings-segmented {
@@ -2908,10 +2909,17 @@ label.is-disabled.earth-mobile-settings-card {
display: inline-flex;
align-items: center;
justify-content: center;
flex: 0 0 calc(30px * var(--hud-scale));
width: calc(30px * var(--hud-scale));
height: calc(30px * var(--hud-scale));
min-width: calc(30px * var(--hud-scale));
min-height: calc(30px * var(--hud-scale));
max-width: calc(30px * var(--hud-scale));
max-height: calc(30px * var(--hud-scale));
aspect-ratio: 1 / 1;
padding: 0;
border: 1px solid rgba(122, 180, 255, 0.24);
border-radius: 999px;
border-radius: 50%;
background:
radial-gradient(circle at 35% 22%, rgba(255, 255, 255, 0.2), transparent 58%),
linear-gradient(180deg, rgba(121, 159, 207, 0.18), rgba(72, 101, 139, 0.24));
@@ -2926,12 +2934,16 @@ label.is-disabled.earth-mobile-settings-card {
}
.earth-settings-reload-action--text {
flex-basis: auto;
width: auto;
max-width: none;
min-width: calc(46px * var(--hud-scale));
padding: 0 calc(12px * var(--hud-scale));
aspect-ratio: auto;
font: inherit;
font-size: calc(0.7rem * var(--hud-scale));
font-weight: 600;
border-radius: 999px;
}
.earth-settings-reload-action[hidden] {
@@ -3076,6 +3088,7 @@ label.is-disabled.earth-mobile-settings-card {
.earth-settings-slider {
flex: 1 1 auto;
min-width: calc(120px * var(--hud-scale));
width: 100%;
height: calc(4px * var(--hud-scale));
appearance: none;

View File

@@ -333,6 +333,9 @@ const bgpEventIconLayer = createInteractableLayer({
pulseOffset: Math.random() * Math.PI * 2,
}),
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
},
});
const bgpCollectorIconLayer = createInteractableLayer({
@@ -402,6 +405,9 @@ const bgpCollectorIconLayer = createInteractableLayer({
};
},
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
},
});
function clamp(value, min, max) {

View File

@@ -214,7 +214,6 @@ const computeCenterIconLayer = createInteractableLayer({
},
icon: {
coordinates: "canvas",
colorable: false,
fitSize: COMPUTE_CENTER_ICON_FIT_SIZE,
glowBlur: 16,
getSource({ marker, item }) {
@@ -248,6 +247,9 @@ const computeCenterIconLayer = createInteractableLayer({
pulseOffset: Math.random() * Math.PI * 2,
}),
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
},
});
export function formatComputeCenterTypeLabel(siteType) {

View File

@@ -64,6 +64,8 @@ export const SURFACE_HOVER_INFO_MODES = {
export const DEFAULT_SURFACE_HOVER_INFO_MODE =
SURFACE_HOVER_INFO_MODES.FULL;
export const EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48;
export const CRUISE_CONFIG = {
dwellMs: 7_000,
focusDurationMs: 1_400,
@@ -233,8 +235,12 @@ export const COUNTRY_BOUNDARY_CONFIG = {
tilePrefetchRing: 1,
tileDebounceMs: 180,
tileCacheLimit: 150,
lineAltitudeOffset: 0.115,
hoverAltitudeOffset: 0.115,
// Keep boundary/coastline strokes on the same shell as the high-res earth
// texture overlay. A lower or higher radius creates visible parallax while
// the globe rotates.
lineAltitudeOffset: EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET,
hoverAltitudeOffset: EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET,
claimLineAltitudeOffset: 0,
hoverMissStickyMs: 160,
lineColor: 0x7fc7ff,
lineOpacity: 0.58,
@@ -540,7 +546,7 @@ export const EARTH_MATERIAL_CONFIG = {
shininess: 12,
emissive: 0x010609,
opacity: 1,
textureOverlayAltitudeOffset: 0.48,
textureOverlayAltitudeOffset: EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET,
textureOverlayOpacity: 0.88,
textureOverlayRenderOrder: 0.96,
textureOverlaySpecular: 0x05080d,

View File

@@ -185,6 +185,11 @@ const KEYBOARD_ROTATION_STOP_SPEED = 0.012;
const KEYBOARD_ZOOM_STEP = 0.1;
const WHEEL_ZOOM_STEP = 0.1;
const WHEEL_ZOOM_DURATION_MS = 180;
const WHEEL_TRACKPAD_PIXEL_THRESHOLD = 48;
const WHEEL_TRACKPAD_DEADZONE = 0.35;
const WHEEL_TRACKPAD_RESIDUAL_WINDOW_MS = 140;
const WHEEL_TRACKPAD_RESIDUAL_RATIO = 0.65;
const WHEEL_TRACKPAD_SENSITIVITY = 0.0024;
const TARGET_SWITCH_ZOOM_IN_PHASE = 0.28;
const TARGET_SWITCH_ROTATE_PHASE = 0.5;
let settingsModalTimer = null;
@@ -655,7 +660,7 @@ function stopKeyboardRotationControl({ actionId = null, restoreAutoRotate = fals
}
function applyKeyboardZoom(direction) {
setZoomLevel(zoomLevel + direction * KEYBOARD_ZOOM_STEP, activeCamera);
setZoomLevel(getZoomLevelFromCamera(activeCamera) + direction * KEYBOARD_ZOOM_STEP, activeCamera);
showZoomStatusCapsule({ force: true });
}
@@ -1318,6 +1323,19 @@ function clampEarthZoomLevel(nextZoom) {
return Math.min(CONFIG.maxZoom, Math.max(CONFIG.minZoom, parsedZoom));
}
function getZoomLevelFromCamera(camera = activeCamera) {
const cameraZ = Number(camera?.position?.z);
if (!Number.isFinite(cameraZ) || cameraZ <= 0) {
return clampEarthZoomLevel(zoomLevel);
}
return clampEarthZoomLevel(CONFIG.defaultCameraZ / cameraZ);
}
function syncZoomLevelFromCamera(camera = activeCamera) {
zoomLevel = getZoomLevelFromCamera(camera);
return zoomLevel;
}
function formatZoomPercent(zoom) {
return `${Math.round(zoom * 100)}%`;
}
@@ -2188,8 +2206,7 @@ function setDefaultEarthZoom(nextZoom, { persist = true, applyToCurrentView = tr
syncDefaultEarthZoomUi(defaultEarthZoom);
if (applyToCurrentView && activeCamera) {
zoomLevel = defaultEarthZoom;
applyZoom(activeCamera);
setZoomLevel(defaultEarthZoom, activeCamera);
}
if (persist) {
@@ -3110,12 +3127,7 @@ export function applyImmediateView(targetEarthObj, camera, options = {}) {
targetEarthObj.rotation.x = nextRotation.x;
targetEarthObj.rotation.y = nextRotation.y;
targetEarthObj.rotation.z = nextRotation.z;
zoomLevel = zoom;
if (camera) {
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
updateZoomDisplay(zoomLevel, camera.position.z.toFixed(0));
}
setZoomLevel(zoom, camera);
}
export function setZoomLevel(nextZoom, camera = activeCamera) {
@@ -3127,13 +3139,16 @@ export function setZoomLevel(nextZoom, camera = activeCamera) {
return zoomLevel;
}
export function showZoomStatusCapsule({ force = false } = {}) {
export function showZoomStatusCapsule({ force = false, zoom = null } = {}) {
const now = Date.now();
if (!force && now - lastZoomStatusUpdateTime < ZOOM_STATUS_UPDATE_INTERVAL_MS) {
return;
}
lastZoomStatusUpdateTime = now;
showGestureStatusMessage(`缩放 ${Math.round(zoomLevel * 100)}%`, "info");
const currentZoom = Number.isFinite(Number(zoom))
? clampEarthZoomLevel(zoom)
: syncZoomLevelFromCamera(activeCamera);
showGestureStatusMessage(`缩放 ${Math.round(currentZoom * 100)}%`, "info");
}
function cancelSettingsSheetAnimation() {
@@ -4692,27 +4707,25 @@ function setupZoomControls(camera) {
const MAX_PERCENT = CONFIG.maxZoom * 100;
function doZoomStep(direction) {
let currentPercent = Math.round(zoomLevel * 100);
let currentPercent = Math.round(getZoomLevelFromCamera(camera) * 100);
let newPercent =
direction > 0 ? currentPercent + CLICK_STEP : currentPercent - CLICK_STEP;
if (newPercent > MAX_PERCENT) newPercent = MAX_PERCENT;
if (newPercent < MIN_PERCENT) newPercent = MIN_PERCENT;
zoomLevel = newPercent / 100;
applyZoom(camera);
setZoomLevel(newPercent / 100, camera);
showZoomStatusCapsule({ force: true });
}
function doContinuousZoom(direction) {
let currentPercent = Math.round(zoomLevel * 100);
let currentPercent = Math.round(getZoomLevelFromCamera(camera) * 100);
let newPercent = direction > 0 ? currentPercent + 1 : currentPercent - 1;
if (newPercent > MAX_PERCENT) newPercent = MAX_PERCENT;
if (newPercent < MIN_PERCENT) newPercent = MIN_PERCENT;
zoomLevel = newPercent / 100;
applyZoom(camera);
setZoomLevel(newPercent / 100, camera);
showZoomStatusCapsule();
}
@@ -4775,10 +4788,8 @@ function setupZoomControls(camera) {
bindListener(zoomOut, "touchend", () => handleMouseUp(-1));
bindListener(zoomValue, "click", () => {
const startZoomVal = zoomLevel;
const startZoomVal = getZoomLevelFromCamera(camera);
const targetZoom = getDefaultEarthZoomLevel();
const startDistance = CONFIG.defaultCameraZ / startZoomVal;
const targetDistance = CONFIG.defaultCameraZ / targetZoom;
animateValue(
0,
@@ -4786,14 +4797,11 @@ function setupZoomControls(camera) {
600,
(progress) => {
const ease = 1 - Math.pow(1 - progress, 3);
zoomLevel = startZoomVal + (targetZoom - startZoomVal) * ease;
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
const distance =
startDistance + (targetDistance - startDistance) * ease;
updateZoomDisplay(zoomLevel, distance.toFixed(0));
const nextZoom = startZoomVal + (targetZoom - startZoomVal) * ease;
setZoomLevel(nextZoom, camera);
},
() => {
zoomLevel = targetZoom;
setZoomLevel(targetZoom, camera);
showStatusMessage(getZoomResetStatusMessage(targetZoom), "info");
},
);
@@ -4802,9 +4810,51 @@ function setupZoomControls(camera) {
function setupWheelZoom(camera, renderer) {
let wheelZoomFrameId = null;
let wheelZoomTarget = zoomLevel;
let wheelZoomStart = zoomLevel;
let wheelZoomTarget = getZoomLevelFromCamera(camera);
let wheelZoomStart = wheelZoomTarget;
let wheelZoomStartAt = 0;
let lastTrackpadDirection = 0;
let suppressedTrackpadDirection = 0;
let suppressedTrackpadUntil = 0;
let suppressedTrackpadMagnitude = 0;
function getWheelPixelDelta(event) {
if (event.deltaMode === WheelEvent.DOM_DELTA_LINE) {
return event.deltaY * 16;
}
if (event.deltaMode === WheelEvent.DOM_DELTA_PAGE) {
return event.deltaY * window.innerHeight;
}
return event.deltaY;
}
function isTrackpadWheel(event, pixelDelta) {
return event.deltaMode === WheelEvent.DOM_DELTA_PIXEL &&
Math.abs(pixelDelta) < WHEEL_TRACKPAD_PIXEL_THRESHOLD;
}
function shouldSuppressTrackpadResidual(pixelDelta) {
const direction = Math.sign(pixelDelta);
const magnitude = Math.abs(pixelDelta);
const now = performance.now();
return direction !== 0 &&
direction === suppressedTrackpadDirection &&
now < suppressedTrackpadUntil &&
magnitude < suppressedTrackpadMagnitude * WHEEL_TRACKPAD_RESIDUAL_RATIO;
}
function recordTrackpadWheel(pixelDelta) {
const direction = Math.sign(pixelDelta);
const magnitude = Math.abs(pixelDelta);
if (direction === 0) return;
const now = performance.now();
if (lastTrackpadDirection !== 0 && direction !== lastTrackpadDirection) {
suppressedTrackpadDirection = lastTrackpadDirection;
suppressedTrackpadUntil = now + WHEEL_TRACKPAD_RESIDUAL_WINDOW_MS;
suppressedTrackpadMagnitude = magnitude;
}
lastTrackpadDirection = direction;
}
function stopWheelZoomAnimation() {
if (wheelZoomFrameId !== null) {
@@ -4824,28 +4874,55 @@ function setupWheelZoom(camera, renderer) {
1,
);
const ease = 1 - Math.pow(1 - progress, 3);
zoomLevel = wheelZoomStart + (wheelZoomTarget - wheelZoomStart) * ease;
applyZoom(camera);
setZoomLevel(
wheelZoomStart + (wheelZoomTarget - wheelZoomStart) * ease,
camera,
);
if (progress < 1) {
wheelZoomFrameId = window.requestAnimationFrame(animateWheelZoom);
return;
}
zoomLevel = wheelZoomTarget;
applyZoom(camera);
setZoomLevel(wheelZoomTarget, camera);
wheelZoomFrameId = null;
wheelZoomStartAt = 0;
}
function startWheelZoomAnimation() {
wheelZoomStart = zoomLevel;
wheelZoomStart = getZoomLevelFromCamera(camera);
wheelZoomStartAt = 0;
if (wheelZoomFrameId === null) {
wheelZoomFrameId = window.requestAnimationFrame(animateWheelZoom);
}
}
function applyMouseWheelZoom(direction) {
suppressedTrackpadDirection = 0;
suppressedTrackpadUntil = 0;
const baseZoom = wheelZoomFrameId === null
? getZoomLevelFromCamera(camera)
: wheelZoomTarget;
wheelZoomTarget = clampEarthZoomLevel(
baseZoom + direction * WHEEL_ZOOM_STEP,
);
stopWheelZoomAnimation();
startWheelZoomAnimation();
showZoomStatusCapsule({ force: true, zoom: wheelZoomTarget });
}
function applyTrackpadWheelZoom(pixelDelta) {
if (Math.abs(pixelDelta) < WHEEL_TRACKPAD_DEADZONE) return;
if (shouldSuppressTrackpadResidual(pixelDelta)) return;
stopWheelZoomAnimation();
const currentZoom = getZoomLevelFromCamera(camera);
const nextZoom = currentZoom * Math.exp(-pixelDelta * WHEEL_TRACKPAD_SENSITIVITY);
wheelZoomTarget = setZoomLevel(nextZoom, camera);
wheelZoomStart = wheelZoomTarget;
recordTrackpadWheel(pixelDelta);
showZoomStatusCapsule({ force: true, zoom: wheelZoomTarget });
}
cleanupFns.push(stopWheelZoomAnimation);
bindListener(
@@ -4853,25 +4930,17 @@ function setupWheelZoom(camera, renderer) {
"wheel",
(e) => {
e.preventDefault();
const direction = e.deltaY < 0 ? 1 : -1;
const baseZoom =
wheelZoomFrameId === null ? zoomLevel : wheelZoomTarget;
wheelZoomTarget = clampEarthZoomLevel(
baseZoom + direction * WHEEL_ZOOM_STEP,
);
startWheelZoomAnimation();
showZoomStatusCapsule({ force: true });
const pixelDelta = getWheelPixelDelta(e);
if (isTrackpadWheel(e, pixelDelta)) {
applyTrackpadWheelZoom(pixelDelta);
return;
}
applyMouseWheelZoom(pixelDelta < 0 ? 1 : -1);
},
{ passive: false },
);
}
function applyZoom(camera) {
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
const distance = camera.position.z.toFixed(0);
updateZoomDisplay(zoomLevel, distance);
}
function animateValue(start, end, duration, onUpdate, onComplete) {
const animationToken = ++focusViewAnimationToken;
const startTime = performance.now();
@@ -5800,7 +5869,7 @@ export function focusEarthView(camera, options = {}) {
const startRotX = earthObj.rotation.x;
const startRotY = earthObj.rotation.y;
const startRotZ = earthObj.rotation.z;
const startZoom = zoomLevel;
const startZoom = getZoomLevelFromCamera(camera);
const defaultZoom = getDefaultEarthZoomLevel();
const shouldRestoreZoomViaDefault =
zoomTransitionMode === "restore-current-via-default" &&
@@ -5829,30 +5898,26 @@ export function focusEarthView(camera, options = {}) {
if (progress < rotateStartProgress) {
const zoomProgress = progress / rotateStartProgress;
const zoomEase = 1 - Math.pow(1 - zoomProgress, 3);
zoomLevel = startZoom + (defaultZoom - startZoom) * zoomEase;
setZoomLevel(startZoom + (defaultZoom - startZoom) * zoomEase, camera);
} else if (progress <= rotateEndProgress) {
zoomLevel = defaultZoom;
setZoomLevel(defaultZoom, camera);
} else {
const zoomProgress = (progress - rotateEndProgress) / (1 - rotateEndProgress);
const zoomEase = 1 - Math.pow(1 - zoomProgress, 3);
zoomLevel = defaultZoom + (startZoom - defaultZoom) * zoomEase;
setZoomLevel(defaultZoom + (startZoom - defaultZoom) * zoomEase, camera);
}
} else {
earthObj.rotation.x = startRotX + (nextRotation.x - startRotX) * ease;
earthObj.rotation.y = startRotY + (nextRotation.y - startRotY) * ease;
earthObj.rotation.z = startRotZ + (nextRotation.z - startRotZ) * ease;
zoomLevel = startZoom + (zoom - startZoom) * ease;
setZoomLevel(startZoom + (zoom - startZoom) * ease, camera);
}
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
updateZoomDisplay(zoomLevel, camera.position.z.toFixed(0));
},
() => {
zoomLevel = shouldRestoreZoomViaDefault ? startZoom : zoom;
setZoomLevel(shouldRestoreZoomViaDefault ? startZoom : zoom, camera);
earthObj.rotation.x = nextRotation.x;
earthObj.rotation.y = nextRotation.y;
earthObj.rotation.z = nextRotation.z;
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
updateZoomDisplay(zoomLevel, camera.position.z.toFixed(0));
if (!suppressStatus) {
showStatusMessage("视角已重置", "info");
}
@@ -5863,7 +5928,7 @@ export function focusEarthView(camera, options = {}) {
}
export function getZoomLevel() {
return zoomLevel;
return syncZoomLevelFromCamera(activeCamera);
}
export function getDefaultEarthZoomLevel() {

View File

@@ -210,7 +210,7 @@ function boundaryLineRadius({ claim = false } = {}) {
return (
CONFIG.earthRadius +
COUNTRY_BOUNDARY_CONFIG.lineAltitudeOffset +
(claim ? 0.018 : 0)
(claim ? COUNTRY_BOUNDARY_CONFIG.claimLineAltitudeOffset : 0)
);
}

View File

@@ -93,6 +93,9 @@ const earthInteractableLayer = createInteractableLayer({
...item,
type: "earth_interactable",
}),
cluster: {
strategy: "stable-spherical",
},
});
export async function loadEarthInteractables(earth, { silent = false } = {}) {

File diff suppressed because it is too large Load Diff

View File

@@ -20,7 +20,6 @@ import {
showStatusMessage,
queueStatusMessage,
updateCoordinatesDisplay,
updateZoomDisplay,
updateEarthStats,
setEarthStatValue,
setLoading,
@@ -301,6 +300,8 @@ export let scene;
export let camera;
export let renderer;
const WEBGL_GPU_DIAGNOSTICS_EVENT = "earth:gpu-diagnostics";
let isDragging = false;
let previousMousePosition = { x: 0, y: 0 };
let targetRotation = { x: 0, y: 0 };
@@ -451,6 +452,29 @@ function getViewportAspect() {
return window.innerWidth / window.innerHeight;
}
function getWebGLGpuDiagnostics(activeRenderer) {
const gl = activeRenderer?.getContext?.();
if (!gl) return null;
const debugInfo = gl.getExtension?.("WEBGL_debug_renderer_info");
const attrs = gl.getContextAttributes?.() || {};
return {
requestedPowerPreference: "high-performance",
contextPowerPreference: attrs.powerPreference || null,
vendor: debugInfo ? gl.getParameter(debugInfo.UNMASKED_VENDOR_WEBGL) : gl.getParameter(gl.VENDOR),
renderer: debugInfo ? gl.getParameter(debugInfo.UNMASKED_RENDERER_WEBGL) : gl.getParameter(gl.RENDERER),
antialias: Boolean(attrs.antialias),
alpha: Boolean(attrs.alpha),
};
}
function publishWebGLGpuDiagnostics(activeRenderer) {
const diagnostics = getWebGLGpuDiagnostics(activeRenderer);
if (!diagnostics) return;
window.__planetEarthGpu = diagnostics;
console.info("[智能星球] WebGL GPU diagnostics", diagnostics);
window.dispatchEvent(new CustomEvent(WEBGL_GPU_DIAGNOSTICS_EVENT, { detail: diagnostics }));
}
function syncRendererViewport() {
if (!camera || !renderer) return;
camera.aspect = getViewportAspect();
@@ -793,12 +817,30 @@ function isSameVessel(marker1, marker2) {
return Boolean(marker1 && marker2 && marker1.userData?.mmsi === marker2.userData?.mmsi);
}
function getFirstObjectIntersection(intersections) {
return intersections.find((hit) => !hit?.cluster && hit?.object)?.object || null;
}
function getPrimaryClusterHit(...intersectionGroups) {
return intersectionGroups
.flat()
.filter((hit) => hit?.cluster && Number(hit.clusterCount) > 1)
.sort((a, b) => a.distancePxSq - b.distancePxSq)[0] || null;
}
function getClusterBriefHtml(hit) {
const count = Number(hit?.clusterCount || hit?.clusterMarkers?.length || 0);
return `<strong>共 ${count} 个对象</strong><br><span>放大后可查看单个图标</span>`;
}
function getPrimaryBGPHoverTarget(bgpAnomalyIntersects, bgpCollectorIntersects) {
if (bgpAnomalyIntersects.length > 0) {
return bgpAnomalyIntersects[0].object;
const anomalyMarker = getFirstObjectIntersection(bgpAnomalyIntersects);
if (anomalyMarker) {
return anomalyMarker;
}
if (bgpCollectorIntersects.length > 0) {
return bgpCollectorIntersects[0].object;
const collectorMarker = getFirstObjectIntersection(bgpCollectorIntersects);
if (collectorMarker) {
return collectorMarker;
}
return null;
}
@@ -808,8 +850,8 @@ function getPrimaryBGPClickTarget(
bgpAnomalyIntersects,
bgpCollectorIntersects,
) {
const anomalyMarker = bgpAnomalyIntersects[0]?.object || null;
const collectorMarker = bgpCollectorIntersects[0]?.object || null;
const anomalyMarker = getFirstObjectIntersection(bgpAnomalyIntersects);
const collectorMarker = getFirstObjectIntersection(bgpCollectorIntersects);
if (!anomalyMarker && !collectorMarker) return null;
if (!anomalyMarker) return collectorMarker;
if (!collectorMarker) return anomalyMarker;
@@ -952,19 +994,23 @@ function getMotionIconCandidates() {
const candidates = [];
if (getShowBGP()) {
getBGPEventIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "bgp", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
getBGPCollectorIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "bgp_collector", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
}
if (getShowComputeCenters()) {
getComputeCenterIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "compute_center", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
}
if (getShowVessels()) {
getVesselIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "vessel", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
}
@@ -3961,7 +4007,7 @@ export function init() {
0.1,
5000,
);
camera.position.z = CONFIG.defaultCameraZ;
setZoomLevel(getDefaultEarthZoomLevel(), camera);
setSatelliteCamera(camera);
renderer = new THREE.WebGLRenderer({
@@ -3972,6 +4018,7 @@ export function init() {
syncRendererViewport();
renderer.setClearColor(0x02040a, 1);
renderer.setPixelRatio(window.devicePixelRatio);
publishWebGLGpuDiagnostics(renderer);
const container = document.getElementById("container");
if (container) {
@@ -5287,13 +5334,17 @@ function onMouseMove(event) {
bgpAnomalyIntersects,
bgpCollectorIntersects,
);
const hoveredClusterHit = getPrimaryClusterHit(
bgpAnomalyIntersects,
bgpCollectorIntersects,
computeCenterIntersects,
);
if (hoveredBGP && !isSameBGPMarker(hoveredBGP, hoveredBGPMarker)) {
clearTransientHoverState();
}
const hoveredComputeCenterMarker =
computeCenterIntersects.length > 0 ? computeCenterIntersects[0].object : null;
const hoveredComputeCenterMarker = getFirstObjectIntersection(computeCenterIntersects);
const hoveredVesselMarker =
vesselPick.checked && vesselIntersects.length > 0 ? vesselIntersects[0].object : null;
const earthPoint = screenToEarthCoords(
@@ -5357,6 +5408,20 @@ function onMouseMove(event) {
let objectTooltipShown = false;
if (
hoveredClusterHit &&
!hoveredBGPMarker &&
!hoveredComputeCenterMarker &&
lockedObjectType !== "bgp" &&
lockedObjectType !== "bgp_collector" &&
lockedObjectType !== "compute_center"
) {
showTooltip(
event.clientX + TOOLTIP_CURSOR_OFFSET,
event.clientY + TOOLTIP_CURSOR_OFFSET,
getClusterBriefHtml(hoveredClusterHit),
);
objectTooltipShown = true;
} else if (
hoveredBGPMarker &&
getShowBGP() &&
lockedObjectType !== "bgp" &&
@@ -5663,9 +5728,7 @@ function onClick(event) {
const clickedBGPMarker = getShowBGP()
? getPrimaryBGPClickTarget(event, bgpAnomalyIntersects, bgpCollectorIntersects)
: null;
const clickedComputeCenterMarker = computeCenterIntersects.length > 0
? computeCenterIntersects[0].object
: null;
const clickedComputeCenterMarker = getFirstObjectIntersection(computeCenterIntersects);
const clickedVesselMarker = vesselIntersects.length > 0
? vesselIntersects[0].object
: null;

View File

@@ -257,6 +257,11 @@ const vesselIconLayer = createInteractableLayer({
vessel_kind: item.type,
baseScale: VESSEL_CONFIG.marker.baseScale,
}),
cluster: {
strategy: "dynamic-screen",
enabled: true,
maxMarkersPerDot: 10,
},
});
const DEFAULT_VESSEL_VIEWPORT = {

View File

@@ -4,6 +4,7 @@ import { Routes, Route, Navigate, useLocation } from 'react-router-dom'
import { useAuthStore } from './stores/auth'
import Login from './pages/Login/Login'
import { AdminErrorBoundary } from './admin/components/AdminErrorBoundary'
const Register = lazy(() => import('./pages/Register/Register'))
const VerifyEmail = lazy(() => import('./pages/VerifyEmail/VerifyEmail'))
@@ -51,7 +52,7 @@ function App() {
<Route path={DOCS_ROUTE} element={<Docs />} />
<Route path={DOCS_ROUTE_PATTERN} element={<Docs />} />
<Route path="/playground" element={<Navigate to="/ai?section=playground" replace />} />
<Route path="/*" element={<AdminRoutes />} />
<Route path="/*" element={<AdminErrorBoundary><AdminRoutes /></AdminErrorBoundary>} />
</Routes>
</Suspense>
)

View File

@@ -0,0 +1,39 @@
import { Component, type ErrorInfo, type ReactNode } from 'react'
import { reportAdminRuntimeLog } from '../runtimeLogs'
type AdminErrorBoundaryProps = {
children: ReactNode
}
type AdminErrorBoundaryState = {
hasError: boolean
}
export class AdminErrorBoundary extends Component<AdminErrorBoundaryProps, AdminErrorBoundaryState> {
state: AdminErrorBoundaryState = { hasError: false }
static getDerivedStateFromError() {
return { hasError: true }
}
componentDidCatch(error: Error, errorInfo: ErrorInfo) {
void reportAdminRuntimeLog({
level: 'error',
category: 'react-error-boundary',
module: 'admin',
message: error.message || '控制台渲染错误',
detail: `${error.stack || error.message}\n${errorInfo.componentStack || ''}`,
})
}
render() {
if (this.state.hasError) {
return (
<div className="app-route-loading">
<div className="app-route-loading__message"></div>
</div>
)
}
return this.props.children
}
}

View File

@@ -15,7 +15,7 @@ import Scrollbar from '../../../components/Scrollbar/Scrollbar'
import SegmentedControl from '../../../components/SegmentedControl/SegmentedControl'
import { useAuthStore } from '../../../stores/auth'
import { useAdminTheme, type AdminThemeMode } from '../../design/theme'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
import { adminRouteGroups, getVisibleAdminRoutes } from '../../routes/manifest'
import { useAdminSearch } from '../../search/AdminSearchContext'
import { Button } from '../ui/button'

View File

@@ -1,5 +1,5 @@
import { type HTMLAttributes } from 'react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
type BadgeTone = 'default' | 'blue' | 'green' | 'amber' | 'red' | 'purple' | 'cyan' | 'slate'

View File

@@ -1,5 +1,5 @@
import { type HTMLAttributes } from 'react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
export function Card({ className, ...props }: HTMLAttributes<HTMLDivElement>) {
return <div className={cn('an-card', className)} {...props} />

View File

@@ -1,5 +1,5 @@
import { forwardRef, type InputHTMLAttributes, type TextareaHTMLAttributes } from 'react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
export const Input = forwardRef<HTMLInputElement, InputHTMLAttributes<HTMLInputElement>>(
({ className, ...props }, ref) => <input ref={ref} className={cn('an-input', className)} {...props} />,

View File

@@ -1,6 +1,6 @@
import * as SelectPrimitive from '@radix-ui/react-select'
import { Check, ChevronDown } from 'lucide-react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
export interface SelectOption {
value: string

View File

@@ -23,7 +23,7 @@ import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '../co
import { Dialog } from '../components/ui/dialog'
import { Select } from '../components/ui/select'
import { useToast } from '../components/ui/toast'
import { formatNumber } from '../lib/utils'
import { formatNumber } from '../utils'
interface Stats {
total_datasources: number

View File

@@ -1,7 +1,9 @@
import axios from 'axios'
import { Copy, RefreshCw, Search, X } from 'lucide-react'
import { useEffect, useMemo, useState } from 'react'
import { Copy, Pause, Play, RefreshCw, Search, X } from 'lucide-react'
import { useEffect, useMemo, useRef, useState } from 'react'
import { useLocation, useNavigate } from 'react-router-dom'
import Scrollbar from '../../components/Scrollbar/Scrollbar'
import { useWebSocket } from '../../hooks/useWebSocket'
import { useAuthStore } from '../../stores/auth'
import { AdminLayout } from '../components/layout/AdminLayout'
import { Badge } from '../components/ui/badge'
@@ -20,6 +22,7 @@ const LOG_LEVEL_OPTIONS = [
{ value: 'info', label: '信息' },
{ value: 'debug', label: '调试' },
]
const LOG_TAIL_CHANNEL = 'logs_tail'
interface LogSourceSummary {
source_id: string
@@ -48,26 +51,44 @@ interface LogSnapshot {
lines: string[]
}
type StoredLogFilters = {
selectedSource?: string
lineLimit?: number
level?: string
startDate?: string
endDate?: string
searchQuery?: string
follow?: boolean
}
function readStoredFilters() {
if (typeof window === 'undefined') return null
try {
const rawValue = window.localStorage.getItem(LOG_FILTER_STORAGE_KEY)
return rawValue ? JSON.parse(rawValue) as {
selectedSource?: string
lineLimit?: number
level?: string
startDate?: string
endDate?: string
searchQuery?: string
} : null
return rawValue ? JSON.parse(rawValue) as StoredLogFilters : null
} catch {
return null
}
}
function readUrlFilters(search: string): StoredLogFilters {
const params = new URLSearchParams(search)
const lineLimit = Number(params.get('limit') || '')
return {
selectedSource: params.get('source') || undefined,
lineLimit: Number.isFinite(lineLimit) && lineLimit > 0 ? lineLimit : undefined,
level: params.get('level') || undefined,
startDate: params.get('start_date') || undefined,
endDate: params.get('end_date') || undefined,
searchQuery: params.get('search') || undefined,
follow: params.get('follow') === '1',
}
}
function statusTone(status: string) {
if (status === 'ok') return 'success'
if (status === 'missing' || status === 'empty') return 'warning'
if (status === 'missing') return 'warning'
if (status === 'empty') return 'neutral'
if (status.includes('unavailable')) return 'danger'
return 'neutral'
}
@@ -89,27 +110,75 @@ function getErrorMessage(error: unknown, fallback: string) {
export default function Logs() {
const storedFilters = readStoredFilters()
const location = useLocation()
const navigate = useNavigate()
const urlFilters = readUrlFilters(location.search)
const { user } = useAuthStore()
const { toast } = useToast()
const isSuperAdmin = user?.role === 'super_admin'
const [sources, setSources] = useState<LogSourceSummary[]>([])
const [selectedSource, setSelectedSource] = useState(storedFilters?.selectedSource || 'backend')
const [lineLimit, setLineLimit] = useState(storedFilters?.lineLimit || 200)
const [level, setLevel] = useState(storedFilters?.level || 'all')
const [startDate, setStartDate] = useState(storedFilters?.startDate || '')
const [endDate, setEndDate] = useState(storedFilters?.endDate || '')
const [searchQuery, setSearchQuery] = useState(storedFilters?.searchQuery || '')
const [submittedSearch, setSubmittedSearch] = useState(storedFilters?.searchQuery || '')
const [selectedSource, setSelectedSource] = useState(urlFilters.selectedSource || storedFilters?.selectedSource || 'backend')
const [lineLimit, setLineLimit] = useState(urlFilters.lineLimit || storedFilters?.lineLimit || 200)
const [level, setLevel] = useState(urlFilters.level || storedFilters?.level || 'all')
const [startDate, setStartDate] = useState(urlFilters.startDate || storedFilters?.startDate || '')
const [endDate, setEndDate] = useState(urlFilters.endDate || storedFilters?.endDate || '')
const [searchQuery, setSearchQuery] = useState(urlFilters.searchQuery ?? storedFilters?.searchQuery ?? '')
const [submittedSearch, setSubmittedSearch] = useState(urlFilters.searchQuery ?? storedFilters?.searchQuery ?? '')
const [followEnabled, setFollowEnabled] = useState(Boolean(urlFilters.follow || storedFilters?.follow))
const [snapshot, setSnapshot] = useState<LogSnapshot | null>(null)
const [sourcesLoading, setSourcesLoading] = useState(false)
const [logLoading, setLogLoading] = useState(false)
const [errorMessage, setErrorMessage] = useState<string | null>(null)
const scrollContainerRef = useRef<HTMLDivElement | null>(null)
const shouldStickToBottomRef = useRef(true)
const selectedSourceInfo = useMemo(
() => sources.find((source) => source.source_id === selectedSource) || null,
[selectedSource, sources],
)
const hasActiveFilters = lineLimit !== 200 || level !== 'all' || Boolean(startDate || endDate || submittedSearch.trim() || searchQuery.trim())
const hasActiveFilters = lineLimit !== 200 || level !== 'all' || Boolean(startDate || endDate || submittedSearch.trim() || searchQuery.trim() || followEnabled)
const { connected: followConnected, sendMessage } = useWebSocket({
autoConnect: followEnabled && isSuperAdmin,
onMessage: (message) => {
if (message.channel !== LOG_TAIL_CHANNEL || !message.payload) return
const payload = message.payload as { mode?: string; source_id?: string; lines?: unknown; line_count?: number; status?: string }
if (payload.source_id !== selectedSource) return
const incomingLines = Array.isArray(payload.lines) ? payload.lines.filter((line): line is string => typeof line === 'string') : []
setSnapshot((current) => {
const base = current || {
source_id: selectedSource,
name: selectedSourceInfo?.name || selectedSource,
kind: selectedSourceInfo?.kind || 'unknown',
location: selectedSourceInfo?.location || '',
description: selectedSourceInfo?.description || '',
category: selectedSourceInfo?.category || '',
status: payload.status || 'ok',
level,
selected_levels: level === 'all' ? [] : [level],
search_query: submittedSearch,
available_levels: ['all', 'error', 'warning', 'info', 'debug'],
line_limit: lineLimit,
line_count: 0,
lines: [],
}
const nextLines = payload.mode === 'snapshot'
? incomingLines
: [...(base.lines || []), ...incomingLines].slice(-lineLimit)
return {
...base,
status: payload.status || base.status,
line_limit: lineLimit,
line_count: nextLines.length,
lines: nextLines,
}
})
setErrorMessage(null)
},
onError: () => {
setErrorMessage('日志跟随连接失败,可暂停后使用手动刷新。')
},
})
const fetchSources = async () => {
if (!isSuperAdmin) return
@@ -159,8 +228,8 @@ export default function Logs() {
}, [isSuperAdmin])
useEffect(() => {
void fetchSnapshot()
}, [isSuperAdmin, selectedSource, lineLimit, level, startDate, endDate, submittedSearch])
if (!followEnabled) void fetchSnapshot()
}, [isSuperAdmin, selectedSource, lineLimit, level, startDate, endDate, submittedSearch, followEnabled])
useEffect(() => {
const timer = window.setTimeout(() => {
@@ -178,8 +247,72 @@ export default function Logs() {
startDate,
endDate,
searchQuery: submittedSearch,
follow: followEnabled,
}))
}, [endDate, level, lineLimit, selectedSource, startDate, submittedSearch])
}, [endDate, followEnabled, level, lineLimit, selectedSource, startDate, submittedSearch])
useEffect(() => {
const filters = readUrlFilters(location.search)
if (filters.selectedSource && filters.selectedSource !== selectedSource) setSelectedSource(filters.selectedSource)
if (filters.lineLimit && filters.lineLimit !== lineLimit) setLineLimit(filters.lineLimit)
if (filters.level && filters.level !== level) setLevel(filters.level)
if ((filters.startDate || '') !== startDate) setStartDate(filters.startDate || '')
if ((filters.endDate || '') !== endDate) setEndDate(filters.endDate || '')
if (filters.searchQuery !== undefined && filters.searchQuery !== searchQuery) {
setSearchQuery(filters.searchQuery)
setSubmittedSearch(filters.searchQuery)
}
if (filters.follow !== followEnabled && new URLSearchParams(location.search).has('follow')) {
setFollowEnabled(Boolean(filters.follow))
}
}, [location.search])
useEffect(() => {
if (typeof window === 'undefined') return
const params = new URLSearchParams()
if (selectedSource && selectedSource !== 'backend') params.set('source', selectedSource)
if (lineLimit !== 200) params.set('limit', String(lineLimit))
if (level !== 'all') params.set('level', level)
if (startDate) params.set('start_date', startDate)
if (endDate) params.set('end_date', endDate)
if (submittedSearch.trim()) params.set('search', submittedSearch.trim())
if (followEnabled) params.set('follow', '1')
const nextSearch = params.toString()
const currentSearch = location.search.replace(/^\?/, '')
if (nextSearch !== currentSearch) {
navigate({ pathname: location.pathname, search: nextSearch ? `?${nextSearch}` : '' }, { replace: true })
}
}, [endDate, followEnabled, level, lineLimit, location.pathname, location.search, navigate, selectedSource, startDate, submittedSearch])
useEffect(() => {
if (!followEnabled || !followConnected || !isSuperAdmin || !selectedSource) return
sendMessage({
type: 'subscribe',
data: {
channel: LOG_TAIL_CHANNEL,
source_id: selectedSource,
limit: lineLimit,
level,
levels: level === 'all' ? undefined : level,
start_date: startDate || undefined,
end_date: endDate || undefined,
search: submittedSearch.trim() || undefined,
},
})
}, [endDate, followConnected, followEnabled, isSuperAdmin, level, lineLimit, selectedSource, sendMessage, startDate, submittedSearch])
useEffect(() => {
const viewport = scrollContainerRef.current
if (!viewport || !shouldStickToBottomRef.current) return
viewport.scrollTop = viewport.scrollHeight
}, [snapshot?.lines])
useEffect(() => {
const viewport = scrollContainerRef.current
if (!viewport) return
viewport.addEventListener('scroll', handleLogScroll, { passive: true })
return () => viewport.removeEventListener('scroll', handleLogScroll)
}, [snapshot?.source_id])
const resetFilters = () => {
setLevel('all')
@@ -188,6 +321,14 @@ export default function Logs() {
setSearchQuery('')
setSubmittedSearch('')
setLineLimit(200)
setFollowEnabled(false)
}
const handleLogScroll = () => {
const viewport = scrollContainerRef.current
if (!viewport) return
const distanceToBottom = viewport.scrollHeight - viewport.scrollTop - viewport.clientHeight
shouldStickToBottomRef.current = distanceToBottom < 32
}
const copyLogs = async () => {
@@ -217,6 +358,15 @@ export default function Logs() {
<Button size="icon" variant="subtle" onClick={() => void fetchSources()} loading={sourcesLoading} aria-label="刷新日志源" title="刷新日志源">
<RefreshCw size={15} />
</Button>
<Button
size="icon"
variant={followEnabled ? 'primary' : 'subtle'}
onClick={() => setFollowEnabled((enabled) => !enabled)}
aria-label={followEnabled ? '暂停日志跟随' : '跟随日志'}
title={followEnabled ? '暂停日志跟随' : '跟随日志'}
>
{followEnabled ? <Pause size={15} /> : <Play size={15} />}
</Button>
<Button size="icon" variant="primary" onClick={() => void fetchSnapshot()} loading={logLoading} aria-label="刷新日志" title="刷新日志">
<RefreshCw size={15} />
</Button>
@@ -256,6 +406,7 @@ export default function Logs() {
</div>
<div className="an-toolbar">
{snapshot ? <Badge tone="blue">{snapshot.line_count} </Badge> : null}
{followEnabled ? <Badge tone={followConnected ? 'green' : 'amber'}>{followConnected ? '跟随中' : '连接中'}</Badge> : null}
<Button size="icon" variant="subtle" onClick={copyLogs} disabled={!snapshot?.lines?.length} aria-label="复制日志" title="复制日志">
<Copy size={15} />
</Button>
@@ -289,7 +440,7 @@ export default function Logs() {
{logLoading ? (
<div className="an-loading"><span className="an-spinner" /></div>
) : snapshot?.lines?.length ? (
<Scrollbar className="an-log-reader__scroll">
<Scrollbar className="an-log-reader__scroll" viewportRef={scrollContainerRef}>
<pre>{snapshot.lines.join('\n')}</pre>
</Scrollbar>
) : (

View File

@@ -122,6 +122,14 @@ type CollectionQueueItem = {
completedAt?: number
}
type DatasourceMetricBaseline = {
taskId: string
sourceId: string
source: string
taskType: string
count: number
}
const DEFAULT_DATASOURCE_FILTERS: DatasourceFilters = {
product: '',
module: '',
@@ -130,9 +138,12 @@ const DEFAULT_DATASOURCE_FILTERS: DatasourceFilters = {
dataStatus: '',
}
const DATASOURCE_TERMINAL_STATUSES = new Set(['success', 'completed', 'failed', 'cancelled'])
const DATASOURCE_FILTER_STORAGE_KEY = 'planet.admin.datasource.filters'
const DATASOURCE_FILTER_QUERY_KEYS = ['product', 'module', 'is_active', 'run_status', 'data_status']
const DATASOURCE_TERMINAL_STATUSES = new Set(['success', 'completed', 'failed', 'cancelled', 'canceled', 'stopped'])
const COLLECTION_QUEUE_ACTIVE_STATUSES = new Set<CollectionQueueStatus>(['queued', 'running', 'cancelling'])
const TASK_ACTIVE_STATUSES = new Set(['queued', 'pending', 'running', 'cancelling'])
const TASK_INACTIVE_STATUSES = new Set(['success', 'completed', 'failed', 'error', 'cancelled', 'canceled', 'stopped', 'idle'])
interface PlaygroundApiMessage {
id: string
@@ -213,15 +224,33 @@ function pick(record: AnyRecord, keys: string[], fallback = '-') {
return fallback
}
function formatCountZh(value: number) {
if (!Number.isFinite(value)) return '-'
const count = Math.max(0, Math.round(value))
if (count >= 100000000) return `${(count / 100000000).toFixed(count >= 1000000000 ? 1 : 2).replace(/\.0+$/, '')} 亿条`
if (count >= 10000) return `${(count / 10000).toFixed(count >= 100000 ? 1 : 2).replace(/\.0+$/, '')} 万条`
return `${count.toLocaleString('zh-CN')}`
}
function datasourceRecordCount(record: AnyRecord) {
const candidates = [record.__metric_count, record.collected_records, record.record_count, record.records, record.count, record.total]
for (const value of candidates) {
if (typeof value === 'number' && Number.isFinite(value)) return value
if (typeof value === 'string' && value.trim() && Number.isFinite(Number(value))) return Number(value)
}
return null
}
function statusTone(value: string): Tone {
const lower = value.toLowerCase()
if (/(默认|default)/.test(lower)) return 'info'
if (/(配置错误|校验失败|连接失败|failed|error|critical|down|danger|unresolved)/.test(lower)) return 'danger'
if (/(失败|配置错误|校验失败|连接失败|failed|error|critical|down|danger|unresolved)/.test(lower)) return 'danger'
if (/(取消|停止|已停止|cancelled|canceled|stopped)/.test(lower)) return 'neutral'
if (/(未配置|未启用|停用|禁用|disabled|false|missing|empty|none|可选|optional|-)/.test(lower)) return 'neutral'
if (/(已配置|configured|running|active|enabled|success|ok|healthy|connected|resolved|ack|true|valid|已读取|已上传|已提交|可用|启用)/.test(lower)) return 'success'
if (/(运行中|采集中|同步中|加载中|排队中|pending|queued|loading|sync|collect|live|stream|删除中|清缓存中|刷新中|任务中|cancelling)/.test(lower)) return 'running'
if (/(成功|完成|已完成|已配置|configured|running|active|enabled|success|completed|done|ok|healthy|connected|resolved|ack|true|valid|已读取|已上传|已提交|可用|启用)/.test(lower)) return 'success'
if (/(pending|queued|warning|degraded|partial|waiting|unknown)/.test(lower)) return 'warning'
if (/(ai|brief|model|provider|prompt)/.test(lower)) return 'ai'
if (/(loading|sync|collect|live|stream|删除中|清缓存中|刷新中|任务中|cancelling)/.test(lower)) return 'running'
return 'neutral'
}
@@ -250,11 +279,15 @@ function recordMetric(record: AnyRecord) {
}
function datasourceMetric(record: AnyRecord) {
if (typeof record.collected_records === 'number') return `${record.collected_records} records`
if (typeof record.__metric === 'string' && record.__metric) return record.__metric
if (typeof record.__metric_count === 'number') return formatCountZh(record.__metric_count)
if (typeof record.collected_records === 'number') return formatCountZh(record.collected_records)
if (typeof record.records_processed === 'number' && typeof record.total_records === 'number') {
return `${record.records_processed}/${record.total_records} records`
return `${formatCountZh(record.records_processed)} / ${formatCountZh(record.total_records)}`
}
if (typeof record.records_processed === 'number') return `${record.records_processed} records`
if (typeof record.records_processed === 'number') return formatCountZh(record.records_processed)
const count = datasourceRecordCount(record)
if (count !== null) return formatCountZh(count)
return pick(record, ['record_count', 'count', 'total', 'value', 'records'], '-')
}
@@ -263,11 +296,20 @@ function activeDatasourceTaskType(record: AnyRecord) {
}
function activeDatasourceTaskStatus(record: AnyRecord) {
return text(record.task_status || record.status || record.phase, '').toLowerCase()
const candidates = [record.task_status, record.phase, record.status]
.map((value) => text(value, '').toLowerCase())
.filter(Boolean)
return candidates.find((status) => TASK_INACTIVE_STATUSES.has(status))
|| candidates.find((status) => TASK_ACTIVE_STATUSES.has(status))
|| text(record.last_status, '').toLowerCase()
|| candidates[0]
|| ''
}
function hasActiveDatasourceTask(record: AnyRecord) {
return record.is_task_active === true || TASK_ACTIVE_STATUSES.has(activeDatasourceTaskStatus(record))
const status = activeDatasourceTaskStatus(record)
if (TASK_INACTIVE_STATUSES.has(status)) return false
return record.is_task_active === true || TASK_ACTIVE_STATUSES.has(status)
}
function isCollectTaskActive(record: AnyRecord) {
@@ -276,24 +318,43 @@ function isCollectTaskActive(record: AnyRecord) {
function datasourceStatus(record: AnyRecord) {
if (isCollectTaskActive(record)) return 'running'
const status = [record.task_status, record.phase, record.status, record.last_status]
.map((value) => text(value, '').toLowerCase())
.find((value) => value && TASK_INACTIVE_STATUSES.has(value))
if (status) return status
return text(record.last_status || record.status, record.is_active === false ? 'disabled' : 'idle')
}
function taskTerminalDisplayStatus(taskType: string, status: string) {
const type = text(taskType, 'collect')
const lower = text(status, '').toLowerCase()
const noun = taskTypeLabel(type)
if (lower === 'success' || lower === 'completed') return `${noun}成功`
if (lower === 'failed' || lower === 'error') return `${noun}失败`
if (lower === 'cancelled' || lower === 'canceled') return `${noun}已取消`
if (lower === 'stopped') return `${noun}已停止`
return ''
}
function datasourceDisplayStatus(record: AnyRecord) {
if (!hasActiveDatasourceTask(record)) return datasourceStatus(record)
const taskType = activeDatasourceTaskType(record)
const status = activeDatasourceTaskStatus(record) || text(datasourceStatus(record), '').toLowerCase()
if (status === 'queued' || status === 'pending') return `${taskTypeLabel(taskType)}排队中`
if (status === 'running' || status === 'collecting') return `${taskTypeLabel(taskType)}`
if (status === 'cancelling') return `停止${taskTypeLabel(taskType)}`
if (!hasActiveDatasourceTask(record)) return datasourceStatus(record)
if (taskType === 'clear_data') return '删除中'
if (taskType === 'clear_cache') return '清缓存中'
if (taskType === 'earth_refresh') return '刷新中'
if (taskType === 'collect') return activeDatasourceTaskStatus(record) === 'queued' ? '排队中' : '运行中'
return '任务中'
if (taskType === 'collect') return '采集中'
return `${taskTypeLabel(taskType)}`
}
function queueStatusFromTask(statusValue: unknown, isRunning?: unknown): CollectionQueueStatus {
const status = text(statusValue, '').toLowerCase()
if (status === 'success' || status === 'completed') return 'success'
if (status === 'failed' || status === 'error') return 'failed'
if (status === 'cancelled' || status === 'canceled') return 'cancelled'
if (status === 'cancelled' || status === 'canceled' || status === 'stopped') return 'cancelled'
if (status === 'skipped') return 'skipped'
if (status === 'queued' || status === 'pending') return 'queued'
if (status === 'cancelling') return 'cancelling'
@@ -302,12 +363,17 @@ function queueStatusFromTask(statusValue: unknown, isRunning?: unknown): Collect
}
function queueItemKey(item: AnyRecord) {
const taskType = text(item.task_type || item.taskType, 'collect')
const taskId = text(item.task_id || item.taskId, '')
if (taskId) return `task:${taskId}`
if (taskId) return `task:${taskType}:${taskId}`
const sourceId = text(item.datasource_id || item.source_id || item.id || item.sourceId, '')
if (sourceId) return `source:${sourceId}`
if (sourceId) return `source:${taskType}:${sourceId}`
const source = text(item.collector_name || item.source, '')
return source ? `source-name:${source}` : `queue:${Date.now()}`
return source ? `source-name:${taskType}:${source}` : `queue:${taskType}:${Date.now()}`
}
function isSameQueueTaskType(left?: string, right?: string) {
return text(left, 'collect') === text(right, 'collect')
}
function isActiveQueueStatus(status: CollectionQueueStatus) {
@@ -330,21 +396,69 @@ function taskTypeLabel(taskType?: string) {
return labels[text(taskType, 'collect')] || '任务'
}
function queueStatusLabel(status: CollectionQueueStatus, taskType?: string) {
function queueStatusLabel(status: CollectionQueueStatus | string, taskType?: string) {
const noun = taskTypeLabel(taskType)
if (status === 'queued') return `${noun}排队中`
if (status === 'running') return `${noun}`
if (status === 'cancelling') return `停止${noun}`
const labels: Record<CollectionQueueStatus, string> = {
const labels: Record<string, string> = {
queued: `${noun}排队中`,
running: `${noun}`,
cancelling: `停止${noun}`,
success: '已完成',
failed: '失败',
success: `${noun}成功`,
completed: `${noun}成功`,
failed: `${noun}失败`,
error: `${noun}失败`,
skipped: '跳过',
cancelled: '已取消',
cancelled: `${noun}已取消`,
canceled: `${noun}已取消`,
stopped: `${noun}已停止`,
idle: '空闲',
}
return labels[status]
return labels[status] || semanticLabel(status)
}
function queuePrimaryMessage(item: CollectionQueueItem) {
const type = text(item.taskType, 'collect')
if (item.status === 'queued') return queueStatusLabel('queued', type)
if (item.status === 'running') {
if (type === 'clear_data') return '正在删除数据'
if (type === 'clear_cache') return '正在清理缓存'
if (type === 'earth_refresh') return '正在刷新图层'
return '正在采集'
}
if (item.status === 'cancelling') {
if (type === 'clear_data') return '正在取消删除'
if (type === 'collect') return '正在停止采集'
return '正在取消任务'
}
if (item.status === 'success') {
if (type === 'clear_data') return '删除完成'
if (type === 'clear_cache') return '清缓存完成'
if (type === 'earth_refresh') return '刷新完成'
return '采集完成'
}
if (item.status === 'failed') {
if (type === 'clear_data') return '删除失败'
if (type === 'clear_cache') return '清缓存失败'
if (type === 'earth_refresh') return '刷新失败'
return '采集失败'
}
if (item.status === 'cancelled') {
if (type === 'clear_data') return '删除已取消'
if (type === 'collect') return '采集已取消'
return '任务已取消'
}
if (item.status === 'skipped') return item.reason ? queueReasonLabel(item.reason) : '已跳过'
return queueStatusLabel(item.status, type)
}
function snapshotStatus(record: AnyRecord) {
const status = text(record.status, '').toLowerCase()
if (status === 'running' && text(record.completed_at || record.completedAt, '')) return 'success'
if (status) return status
if (record.is_current === true) return '当前'
return '-'
}
function queueReasonLabel(reason = '') {
@@ -365,15 +479,27 @@ function formatDuration(startedAt: number, endedAt = Date.now()) {
}
function datasourceTableRow(row: AnyRecord) {
const displayStatus = datasourceDisplayStatus(row)
const taskStatus = text(row.task_status || row.phase || row.status || row.last_status, '')
const terminalStatus = taskTerminalDisplayStatus(activeDatasourceTaskType(row), taskStatus)
return {
...row,
__module: pick(row, ['module', 'source'], '数据源'),
__status: datasourceDisplayStatus(row),
__status: terminalStatus || displayStatus,
__metric: datasourceMetric(row),
__time: pick(row, ['last_run_at', 'last_run'], '-'),
}
}
function recordDisplayStatus(record: AnyRecord) {
const endpointKey = text(record.__endpointKey, '')
if (endpointKey === 'builtin' || record.is_task_active !== undefined || record.task_type || record.task_status) {
const taskStatus = text(record.task_status || record.phase || record.status || record.last_status, '')
return taskTerminalDisplayStatus(activeDatasourceTaskType(record), taskStatus) || datasourceDisplayStatus(record)
}
return semanticLabel(recordStatus(record))
}
function makeAction(label: string, icon: ReactNode, to: string) {
return { label, icon, to }
}
@@ -566,7 +692,10 @@ function defaultColumns(onSelect: (record: TableRecord) => void): Array<ColumnDe
id: 'status',
header: '状态',
size: 130,
cell: ({ row }) => <StatusText tone={statusTone(recordStatus(row.original))}>{recordStatus(row.original)}</StatusText>,
cell: ({ row }) => {
const status = recordDisplayStatus(row.original)
return <StatusText tone={statusTone(status)}>{status}</StatusText>
},
},
{ id: 'metric', header: '指标', size: 220, cell: ({ row }) => <span className="an-muted-text">{semanticLabel(recordMetric(row.original))}</span> },
{ id: 'updated', header: '更新时间', size: 180, cell: ({ row }) => row.original.__time },
@@ -824,7 +953,9 @@ function semanticLabel(value: unknown) {
const lower = raw.toLowerCase()
const labels: Record<string, string> = {
success: '成功',
completed: '完成',
failed: '失败',
error: '失败',
running: '运行中',
pending: '等待中',
queued: '排队中',
@@ -997,6 +1128,59 @@ function datasourceFiltersFromSearch(search: string): DatasourceFilters {
}
}
function hasDatasourceFilterSearch(search: string) {
const params = new URLSearchParams(search)
return DATASOURCE_FILTER_QUERY_KEYS.some((key) => params.has(key))
}
function normalizeDatasourceFilters(value: Partial<DatasourceFilters> | null | undefined): DatasourceFilters {
return {
product: text(value?.product, DEFAULT_DATASOURCE_FILTERS.product),
module: text(value?.module, DEFAULT_DATASOURCE_FILTERS.module),
isActive: ['true', 'false', ''].includes(text(value?.isActive, '')) ? text(value?.isActive, DEFAULT_DATASOURCE_FILTERS.isActive) : DEFAULT_DATASOURCE_FILTERS.isActive,
runStatus: text(value?.runStatus, DEFAULT_DATASOURCE_FILTERS.runStatus),
dataStatus: text(value?.dataStatus, DEFAULT_DATASOURCE_FILTERS.dataStatus),
}
}
function loadStoredDatasourceFilters(): DatasourceFilters {
if (typeof window === 'undefined') return DEFAULT_DATASOURCE_FILTERS
try {
const raw = window.localStorage.getItem(DATASOURCE_FILTER_STORAGE_KEY)
if (!raw) return DEFAULT_DATASOURCE_FILTERS
return normalizeDatasourceFilters(JSON.parse(raw) as Partial<DatasourceFilters>)
} catch {
return DEFAULT_DATASOURCE_FILTERS
}
}
function storeDatasourceFilters(filters: DatasourceFilters) {
if (typeof window === 'undefined') return
window.localStorage.setItem(DATASOURCE_FILTER_STORAGE_KEY, JSON.stringify(filters))
}
function initialDatasourceFilters(search: string): DatasourceFilters {
return hasDatasourceFilterSearch(search) ? datasourceFiltersFromSearch(search) : loadStoredDatasourceFilters()
}
function datasourceFiltersEqual(left: DatasourceFilters, right: DatasourceFilters) {
return left.product === right.product &&
left.module === right.module &&
left.isActive === right.isActive &&
left.runStatus === right.runStatus &&
left.dataStatus === right.dataStatus
}
function datasourceFiltersSearch(filters: DatasourceFilters) {
const params = new URLSearchParams()
params.set('product', filters.product)
params.set('module', filters.module)
params.set('is_active', filters.isActive)
params.set('run_status', filters.runStatus)
params.set('data_status', filters.dataStatus)
return `?${params.toString()}`
}
function datasourceFiltersToParams(filters: DatasourceFilters) {
const params: AnyRecord = { include_endpoint: false }
if (filters.product) params.product = filters.product
@@ -1029,8 +1213,8 @@ function snapshotRows(payload: unknown) {
...row,
__title: pick(row, ['source', 'datasource_name', 'id'], '采集快照'),
__module: '采集快照',
__status: pick(row, ['status', 'is_current'], '-'),
__metric: typeof row.record_count === 'number' ? `${row.record_count} records` : pick(row, ['record_count'], '-'),
__status: snapshotStatus(row),
__metric: typeof row.record_count === 'number' ? formatCountZh(row.record_count) : pick(row, ['record_count'], '-'),
__time: pick(row, ['completed_at', 'started_at', 'created_at'], '-'),
}))
const grouped = new Map<string, AnyRecord[]>()
@@ -1049,7 +1233,7 @@ function snapshotRows(payload: unknown) {
__rowId: `snapshot-source-${source}`,
__title: title,
__module: '采集快照',
__status: pick(current, ['status', 'is_current'], '-'),
__status: snapshotStatus(current),
__metric: `${ordered.length} 个快照`,
__time: pick(current, ['completed_at', 'started_at', 'created_at'], '-'),
__snapshots: ordered,
@@ -1096,8 +1280,8 @@ function formatSnapshotTime(record: AnyRecord) {
function snapshotOptionLabel(record: AnyRecord) {
const current = record.is_current === true ? '当前 · ' : ''
const status = semanticLabel(recordStatus(record))
const count = typeof record.record_count === 'number' ? `${record.record_count} records` : pick(record, ['record_count'], '0 records')
const status = semanticLabel(snapshotStatus(record))
const count = typeof record.record_count === 'number' ? formatCountZh(record.record_count) : pick(record, ['record_count'], '0 ')
return `${current}${formatSnapshotTime(record)} · ${status} · ${count}`
}
@@ -2098,7 +2282,7 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
const navigate = useNavigate()
const [states, setStates] = useState<SectionState[]>([])
const [activeSectionKey, setActiveSectionKey] = useState(config.sections[0]?.key || '')
const [datasourceFilters, setDatasourceFilters] = useState<DatasourceFilters>(() => datasourceFiltersFromSearch(location.search))
const [datasourceFilters, setDatasourceFilters] = useState<DatasourceFilters>(() => initialDatasourceFilters(location.search))
const datasourceFiltersRef = useRef(datasourceFilters)
const [activeGroupKey, setActiveGroupKey] = useState('')
const [hierarchyDraft, setHierarchyDraft] = useState('')
@@ -2114,6 +2298,7 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
const [datasourceSelectedRowIds, setDatasourceSelectedRowIds] = useState<Set<string>>(() => new Set())
const pendingDatasourceTasksRef = useRef<Record<string, { sourceId: string; source?: string; name: string; taskId?: number | string | null; completed?: boolean }>>({})
const completedDatasourceTasksRef = useRef<Set<string>>(new Set())
const datasourceMetricBaselinesRef = useRef<Record<string, DatasourceMetricBaseline>>({})
const datasourcePollTimersRef = useRef<Record<string, number>>({})
const [selected, setSelected] = useState<TableRecord | null>(null)
const [selectedHistory, setSelectedHistory] = useState<TableRecord[]>([])
@@ -2205,14 +2390,11 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
useEffect(() => {
if (config !== configs.datasources) return
const next = datasourceFiltersFromSearch(location.search)
const next = initialDatasourceFilters(location.search)
const current = datasourceFiltersRef.current
const unchanged = current.product === next.product &&
current.module === next.module &&
current.isActive === next.isActive &&
current.runStatus === next.runStatus &&
current.dataStatus === next.dataStatus
const unchanged = datasourceFiltersEqual(current, next)
if (!unchanged) setDatasourceSelectedRowIds(new Set())
storeDatasourceFilters(next)
setDatasourceFilters((filters) => unchanged ? filters : next)
}, [config, location.search])
@@ -2220,25 +2402,8 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
const next = { ...datasourceFiltersRef.current, [key]: value }
setDatasourceSelectedRowIds(new Set())
setDatasourceFilters(next)
const params = new URLSearchParams(location.search)
const queryKeyByFilter: Record<keyof DatasourceFilters, string> = {
product: 'product',
module: 'module',
isActive: 'is_active',
runStatus: 'run_status',
dataStatus: 'data_status',
}
;(Object.keys(queryKeyByFilter) as Array<keyof DatasourceFilters>).forEach((filterKey) => {
const queryKey = queryKeyByFilter[filterKey]
const defaultValue = DEFAULT_DATASOURCE_FILTERS[filterKey]
const nextValue = next[filterKey]
if (!nextValue || nextValue === defaultValue) {
params.delete(queryKey)
} else {
params.set(queryKey, nextValue)
}
})
navigate({ pathname: location.pathname, search: params.toString() ? `?${params.toString()}` : '' }, { replace: true })
storeDatasourceFilters(next)
navigate({ pathname: location.pathname, search: datasourceFiltersSearch(next) }, { replace: true })
}
const sectionRequestParams = useCallback((section: SectionConfig, baseParams?: AnyRecord) => {
@@ -2350,8 +2515,8 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
const index = current.findIndex((existing) => (
existing.key === item.key
|| (item.taskId && existing.taskId === item.taskId)
|| (isActiveQueueStatus(existing.status) && item.sourceId && existing.sourceId === item.sourceId)
|| (isActiveQueueStatus(existing.status) && item.source && existing.source === item.source)
|| (isActiveQueueStatus(existing.status) && isSameQueueTaskType(existing.taskType, item.taskType) && item.sourceId && existing.sourceId === item.sourceId)
|| (isActiveQueueStatus(existing.status) && isSameQueueTaskType(existing.taskType, item.taskType) && item.source && existing.source === item.source)
))
if (index < 0) return [item, ...current]
const next = [...current]
@@ -2365,8 +2530,12 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
const source = text(payload.collector_name || payload.source, '')
const taskId = payload.task_id as number | string | null | undefined
const status = queueStatusFromTask(payload.status || payload.phase, payload.is_running)
const payloadTaskType = text(payload.task_type, '')
setCollectionQueue((current) => current.map((item) => {
const matched = (taskId && item.taskId === taskId) || (sourceId && item.sourceId === sourceId) || (source && item.source === source)
const taskTypeMatched = !payloadTaskType || isSameQueueTaskType(item.taskType, payloadTaskType)
const matched = Boolean(taskId && item.taskId === taskId)
|| (taskTypeMatched && Boolean(sourceId && item.sourceId === sourceId))
|| (taskTypeMatched && Boolean(source && item.source === source))
if (!matched) return item
const terminal = ['success', 'failed', 'cancelled', 'skipped'].includes(status)
return {
@@ -2472,19 +2641,58 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
error_message: payload.error_message,
last_run_at: payload.completed_at || payload.started_at,
}
const metricKey = text(payload.task_id, '') || sourceId || source
const recordsProcessed = typeof payload.records_processed === 'number'
? payload.records_processed
: Number.isFinite(Number(payload.records_processed))
? Number(payload.records_processed)
: null
const applyLiveMetric = (row: AnyRecord, next: AnyRecord) => {
if (!metricKey || recordsProcessed === null || recordsProcessed < 0) return next
if (taskType === 'clear_data') {
let baseline = datasourceMetricBaselinesRef.current[metricKey]
if (!baseline) {
baseline = {
taskId: metricKey,
sourceId,
source,
taskType,
count: datasourceRecordCount(row) ?? 0,
}
datasourceMetricBaselinesRef.current[metricKey] = baseline
}
const nextCount = Math.max(0, baseline.count - recordsProcessed)
return {
...next,
__metric_count: nextCount,
__metric: formatCountZh(nextCount),
collected_records: nextCount,
has_collected_data: nextCount > 0,
}
}
if (taskType === 'collect' && taskActive) {
return {
...next,
__metric: `已处理 ${formatCountZh(recordsProcessed)}`,
}
}
return next
}
setStates((currentStates) => currentStates.map((state) => {
if (state.section.key !== 'builtin') return state
return {
...state,
rows: state.rows.map((row) => {
if (!isSameDatasourceRow(row, sourceId, source)) return row
return normalizeDatasourceTableRecord({ ...row, ...rowPatch, id: row.id, source: row.source })
const merged = applyLiveMetric(row, { ...row, ...rowPatch, id: row.id, source: row.source })
return normalizeDatasourceTableRecord(merged)
}),
}
}))
setSelected((current) => {
if (!current || !isSameDatasourceRow(current, sourceId, source)) return current
return normalizeDatasourceTableRecord({ ...current, ...rowPatch, id: current.id, source: current.source })
const merged = applyLiveMetric(current, { ...current, ...rowPatch, id: current.id, source: current.source })
return normalizeDatasourceTableRecord(merged)
})
patchCollectionQueueFromTask(payload)
}, [config, isSameDatasourceRow, normalizeDatasourceTableRecord, patchCollectionQueueFromTask])
@@ -2525,6 +2733,7 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
toast({ title: `${titleName} 采集已取消` })
}
}
delete datasourceMetricBaselinesRef.current[text(payload.task_id, '') || sourceId || source]
delete pendingDatasourceTasksRef.current[pendingEntry?.[0] || pendingKey]
}, [isSameDatasourceRow, toast, updateDatasourceRow])
@@ -2847,7 +3056,7 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
...item,
status: 'cancelling',
phase: 'cancelling',
phaseMessage: '正在停止任务',
phaseMessage: `正在停止${taskTypeLabel(item.taskType)}`,
updatedAt: now,
})
})
@@ -2931,6 +3140,61 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
datasourcePollTimersRef.current[pollKey] = window.setTimeout(poll, datasourceSocket.connected ? 3000 : 900)
}
const restoreActiveCollectionQueueTasks = useCallback(async () => {
if (config !== configs.datasources) return
try {
const response = await axios.get(apiPath('/tasks'), {
params: {
status: 'queued,running,cancelling',
page_size: 200,
},
})
dataArray(response.data).forEach((task) => {
const sourceId = text(task.datasource_id || task.source_id, '')
if (!sourceId) return
const taskId = task.id as number | string | null | undefined
const source = text(task.source || task.datasource_source, '')
const taskType = text(task.task_type, 'collect')
const record = {
id: sourceId,
source,
collector_name: source,
name: text(task.datasource_name || task.name || source, '数据源'),
task_id: taskId,
task_type: taskType,
task_status: task.status,
status: task.status,
phase: task.phase,
phase_message: task.phase_message,
progress: task.progress,
records_processed: task.records_processed,
total_records: task.total_records,
error_message: task.error_message,
__endpointKey: 'builtin',
__endpointLabel: '内置源',
__rowId: `active-task-${sourceId}-${taskId || taskType}`,
__title: text(task.datasource_name || task.name || source, '数据源'),
__module: '数据源任务',
__status: queueStatusLabel(queueStatusFromTask(task.status || task.phase, true), taskType),
__metric: taskId ? `task ${taskId}` : '-',
__time: text(task.started_at || task.completed_at, '-'),
}
upsertCollectionQueueItem(queueItemFromDatasourceRow(record, taskId, {
taskType,
status: queueStatusFromTask(task.status || task.phase, true),
phase: text(task.phase, text(task.status, '')),
phaseMessage: text(task.phase_message, ''),
progress: typeof task.progress === 'number' ? task.progress : 0,
recordsProcessed: typeof task.records_processed === 'number' ? task.records_processed : undefined,
totalRecords: typeof task.total_records === 'number' ? task.total_records : undefined,
}))
scheduleDatasourceTaskPoll(record, taskId)
})
} catch {
// Queue restore is best-effort; the table and explicit refresh still load normally.
}
}, [config, upsertCollectionQueueItem])
useEffect(() => {
if (config !== configs.datasources) return
const builtinState = states.find((state) => state.section.key === 'builtin')
@@ -2943,7 +3207,7 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
const taskId = row.task_id as number | string | null | undefined
const taskType = text(row.task_type, 'collect')
upsertCollectionQueueItem({
key: queueItemKey({ id: sourceId, source, task_id: taskId }),
key: queueItemKey({ id: sourceId, source, task_id: taskId, task_type: taskType }),
sourceId,
source,
name: recordTitle(row),
@@ -2960,6 +3224,10 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
})
}, [config, states, upsertCollectionQueueItem])
useEffect(() => {
void restoreActiveCollectionQueueTasks()
}, [restoreActiveCollectionQueueTasks])
const clearDatasourceData = async (record: TableRecord) => {
const id = pick(record, ['id', 'source_id', 'key', 'name'], '')
if (!id) return
@@ -4263,7 +4531,7 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
key: row.__rowId,
label: recordTitle(row),
description: '采集快照',
status: normalizeStatusLabel(row.is_current === true ? '当前' : recordStatus(row)),
status: normalizeStatusLabel(snapshotStatus(row)),
count: Array.isArray(row.__snapshots) ? row.__snapshots.length : 1,
record: { ...cleanRecord(row), __sourceEndpoint: row.__endpointKey, __sourceLabel: row.__endpointLabel, __snapshots: row.__snapshots, __snapshotSourceKey: row.__snapshotSourceKey },
}))
@@ -4796,8 +5064,8 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
</select>
</label>
{activeSnapshot ? (
<StatusText tone={statusTone(recordStatus(activeSnapshot))}>
{semanticLabel(recordStatus(activeSnapshot))}
<StatusText tone={statusTone(snapshotStatus(activeSnapshot))}>
{semanticLabel(snapshotStatus(activeSnapshot))}
</StatusText>
) : null}
</div>
@@ -5119,6 +5387,24 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
)
}
const openLogsForTask = (item: Pick<CollectionQueueItem, 'taskId' | 'sourceId' | 'source'> & { requestId?: string | number | null }) => {
const params = new URLSearchParams({ source: 'system-db' })
const requestId = text(item.requestId, '')
const taskId = text(item.taskId, '')
const sourceId = text(item.sourceId, '')
const source = text(item.source, '')
if (requestId) {
params.set('search', `request_id=${requestId}`)
} else if (taskId) {
params.set('search', `task_id=${taskId}`)
} else if (sourceId) {
params.set('search', `datasource_id=${sourceId}`)
} else if (source) {
params.set('search', source)
}
navigate(`/logs?${params.toString()}`)
}
const renderModuleActions = () => {
const actions: ReactNode[] = []
if (config === configs.datasources) {
@@ -5255,20 +5541,26 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
const source = text(selected.source || selected.collector_name, '')
const queueItem = collectionQueue.find((item) => isSameDatasourceRow(selected, item.sourceId || '', item.source || '') && isActiveQueueStatus(item.status))
const status = queueItem?.status || datasourceStatus(selected)
const taskId = queueItem?.taskId || selected.task_id
const taskId = text(queueItem?.taskId || selected.task_id, '')
return (
<section className="an-task-summary">
<div>
<strong></strong>
<p>{queueItem?.phaseMessage || text(selected.last_status || selected.phase_message, '当前没有运行中的任务。')}</p>
</div>
<StatusText tone={statusTone(status)}>{queueStatusLabel(status as CollectionQueueStatus, queueItem?.taskType || text(selected.task_type, 'collect'))}</StatusText>
<StatusText tone={statusTone(status)}>{queueStatusLabel(status, queueItem?.taskType || text(selected.task_type, 'collect'))}</StatusText>
<dl>
<dt></dt><dd>{source || sourceId || '-'}</dd>
<dt></dt><dd>{text(taskId, '-')}</dd>
<dt></dt><dd>{queueItem ? `${queueProgress(queueItem)}%` : '-'}</dd>
<dt></dt><dd>{queueItem?.updatedAt ? new Date(queueItem.updatedAt).toLocaleTimeString() : text(selected.last_run_at, '-')}</dd>
</dl>
<div className="an-task-summary__actions">
<Button size="sm" variant="subtle" onClick={() => openLogsForTask({ taskId, sourceId, source })}>
<FileText size={14} />
</Button>
</div>
</section>
)
}
@@ -5327,13 +5619,14 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
overrides: Partial<CollectionQueueItem> = {},
): CollectionQueueItem => {
const sourceId = pick(record, ['id', 'source_id', 'key', 'name'], '')
const taskType = text(overrides.taskType || record.task_type, 'collect')
return {
key: queueItemKey({ id: sourceId, source: record.source, task_id: taskId }),
key: queueItemKey({ id: sourceId, source: record.source, task_id: taskId, task_type: taskType }),
sourceId,
source: text(record.source || record.collector_name, ''),
name: recordTitle(record),
taskId,
taskType: text(record.task_type, 'collect'),
taskType,
status: 'queued',
phase: 'queued',
phaseMessage: '任务已提交',
@@ -5388,14 +5681,19 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
{group.items.map((item) => (
<article key={item.key} className={`an-collection-queue__item is-${item.status}`}>
<div>
<strong>{item.name}</strong>
<p>{item.phaseMessage || item.error || queueStatusLabel(item.status, item.taskType)}{item.taskId ? ` · task ${item.taskId}` : ''}{item.completedAt ? ` · ${formatDuration(item.createdAt, item.completedAt)}` : ''}</p>
<strong title={item.name}>{item.name}</strong>
<p title={`${item.error || queuePrimaryMessage(item)}${item.taskId ? ` · task ${item.taskId}` : ''}${item.completedAt ? ` · ${formatDuration(item.createdAt, item.completedAt)}` : ''}`}>
{item.error || queuePrimaryMessage(item)}{item.taskId ? ` · task ${item.taskId}` : ''}{item.completedAt ? ` · ${formatDuration(item.createdAt, item.completedAt)}` : ''}
</p>
</div>
<span>{queueProgress(item)}%</span>
<div className="an-collection-queue__item-actions">
<Button size="icon" variant="subtle" title="查看数据源" aria-label="查看数据源" onClick={() => void jumpToQueueRecord(item)}>
<Eye size={14} />
</Button>
<Button size="icon" variant="subtle" title="查看日志" aria-label="查看日志" onClick={() => openLogsForTask(item)}>
<FileText size={14} />
</Button>
{item.status === 'failed' ? (
<Button size="icon" variant="subtle" icon="trigger" title="重试" aria-label="重试" onClick={() => retryQueueItem(item)} />
) : null}
@@ -5557,7 +5855,7 @@ function ModuleConsole({ config }: { config: ModuleConfig }) {
) : null}
<h3>{recordTitle(selected)}</h3>
</div>
<StatusText tone={statusTone(recordStatus(selected))}>{recordStatus(selected)}</StatusText>
<StatusText tone={statusTone(recordDisplayStatus(selected))}>{recordDisplayStatus(selected)}</StatusText>
</header>
{renderRecordActions()}
{renderDatasourceTaskSummary()}

View File

@@ -1,6 +1,6 @@
import { type HTMLAttributes, type ReactNode } from 'react'
import Scrollbar from '../../components/Scrollbar/Scrollbar'
import { cn } from '../lib/utils'
import { cn } from '../utils'
export function PageFrame({
title,

View File

@@ -0,0 +1,98 @@
const RECENT_EVENT_TTL_MS = 15_000
const MAX_DETAIL_LENGTH = 4000
const recentEventMap = new Map<string, number>()
const NON_ADMIN_PATH_PREFIXES = ['/earth', '/docs', '/login', '/register', '/verify-email', '/forgot-password']
type RuntimeLogLevel = 'error' | 'warning' | 'info' | 'debug'
type AdminRuntimeLogInput = {
level?: RuntimeLogLevel
message: string
category?: string
module?: string
detail?: unknown
}
function normalizeErrorDetail(detail: unknown) {
if (!detail) return ''
if (detail instanceof Error) {
return detail.stack || detail.message || String(detail)
}
if (typeof detail === 'string') return detail
try {
return JSON.stringify(detail)
} catch {
return String(detail)
}
}
function shouldSkip(level: string, message: string, detail: string, category: string) {
const key = `${level}::${category}::${message}::${detail}`
const now = Date.now()
const lastSeenAt = recentEventMap.get(key)
recentEventMap.set(key, now)
for (const [entryKey, entryTime] of recentEventMap.entries()) {
if (now - entryTime > RECENT_EVENT_TTL_MS) {
recentEventMap.delete(entryKey)
}
}
return Boolean(lastSeenAt && now - lastSeenAt < RECENT_EVENT_TTL_MS)
}
function isAdminRoute() {
if (typeof window === 'undefined') return false
const pathname = window.location.pathname
return pathname !== '/' && !NON_ADMIN_PATH_PREFIXES.some((prefix) => pathname === prefix || pathname.startsWith(`${prefix}/`))
}
export async function reportAdminRuntimeLog({
level = 'error',
message,
category = 'runtime',
module = 'admin',
detail = '',
}: AdminRuntimeLogInput) {
if (!message || typeof window === 'undefined' || !isAdminRoute()) return
const normalizedDetail = normalizeErrorDetail(detail)
if (shouldSkip(level, message, normalizedDetail, category)) return
try {
await fetch('/api/v1/system/logs/admin-client', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
level,
message,
category,
module,
url: window.location.href,
detail: normalizedDetail.slice(0, MAX_DETAIL_LENGTH),
}),
keepalive: true,
})
} catch {
// Runtime log reporting must never create more runtime noise.
}
}
export function registerAdminRuntimeErrorHandlers() {
if (typeof window === 'undefined') return
window.addEventListener('error', (event) => {
void reportAdminRuntimeLog({
level: 'error',
category: 'window-error',
module: 'admin',
message: event.message || '控制台发生未捕获错误',
detail: event.error || `${event.filename || ''}:${event.lineno || 0}:${event.colno || 0}`,
})
})
window.addEventListener('unhandledrejection', (event) => {
void reportAdminRuntimeLog({
level: 'error',
category: 'unhandledrejection',
module: 'admin',
message: '控制台发生未处理 Promise 错误',
detail: event.reason,
})
})
}

View File

@@ -444,8 +444,9 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
}
.an-collection-queue__item {
position: relative;
display: grid;
grid-template-columns: minmax(0, 1fr) auto auto;
grid-template-columns: minmax(0, 1fr) 44px;
align-items: center;
gap: 8px;
padding: 8px;
@@ -455,6 +456,12 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
}
.an-collection-queue__item-actions {
position: absolute;
top: 6px;
right: 6px;
z-index: 2;
display: flex;
gap: 6px;
opacity: 0;
transform: translateX(4px);
pointer-events: none;
@@ -470,6 +477,7 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
.an-collection-queue__item strong,
.an-collection-queue__item p {
max-width: 100%;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
@@ -522,6 +530,10 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
transform: none;
pointer-events: auto;
}
.an-collection-queue__item {
padding-right: 104px;
}
}
.an-task-summary {
@@ -577,6 +589,12 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
white-space: nowrap;
}
.an-task-summary__actions {
grid-column: 1 / -1;
display: flex;
justify-content: flex-end;
}
.admin__content {
min-width: 0;
height: 100vh;
@@ -2455,34 +2473,64 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
background: currentColor;
}
.an-badge--green,
.an-status-pill--success {
.an-badge--green {
border-color: color-mix(in srgb, var(--an-success) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-success) 12%, var(--an-surface-alt));
color: var(--an-success);
}
.an-badge--amber,
.an-status-pill--warning {
.an-badge--amber {
border-color: color-mix(in srgb, var(--an-warning) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-warning) 12%, var(--an-surface-alt));
color: var(--an-warning);
}
.an-badge--red,
.an-status-pill--danger {
.an-badge--red {
border-color: color-mix(in srgb, var(--an-danger) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-danger) 12%, var(--an-surface-alt));
color: var(--an-danger);
}
.an-badge--blue,
.an-badge--cyan,
.an-badge--cyan {
border-color: color-mix(in srgb, var(--an-info) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-info) 12%, var(--an-surface-alt));
color: var(--an-info);
}
.an-badge--purple {
border-color: color-mix(in srgb, #7c3aed 34%, var(--an-border));
background: color-mix(in srgb, #7c3aed 12%, var(--an-surface-alt));
color: #7c3aed;
}
.an-badge--slate {
border-color: color-mix(in srgb, var(--an-muted) 28%, var(--an-border));
background: color-mix(in srgb, var(--an-muted) 10%, var(--an-surface-alt));
color: var(--an-muted);
}
.an-status-pill--success {
color: var(--an-success);
}
.an-status-pill--warning {
color: var(--an-warning);
}
.an-status-pill--danger {
color: var(--an-danger);
}
.an-status-pill--info,
.an-status-pill--running {
color: var(--an-info);
}
.an-badge--purple,
.an-status-pill--ai {
color: #7c3aed;
}
.an-badge--slate,
.an-status-pill--neutral {
color: var(--an-muted);
}
@@ -3323,7 +3371,7 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
background: var(--an-surface-alt);
}
.an-logs-source span {
.an-logs-source > span:not(.an-status-pill) {
color: var(--an-muted);
font-size: 12px;
}

View File

@@ -0,0 +1,10 @@
import { clsx, type ClassValue } from 'clsx'
import { twMerge } from 'tailwind-merge'
export function cn(...inputs: ClassValue[]) {
return twMerge(clsx(inputs))
}
export function formatNumber(value: number) {
return new Intl.NumberFormat('zh-CN').format(value)
}

View File

@@ -2,8 +2,11 @@ import React from 'react'
import ReactDOM from 'react-dom/client'
import { BrowserRouter } from 'react-router-dom'
import App from './App'
import { registerAdminRuntimeErrorHandlers } from './admin/runtimeLogs'
import './index.css'
registerAdminRuntimeErrorHandlers()
ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<BrowserRouter future={{ v7_startTransition: true, v7_relativeSplatPath: true }}>

View File

@@ -111,9 +111,13 @@ export const DOCS_METADATA: Record<string, DocsMetadataEntry> = {
zh: { title: '智能星球可交互图标接入', group: 'Earth', order: 16 },
en: { title: 'Intelligent Planet Interactable Usage', group: 'Earth', order: 16 },
},
'earth-interactable-clustering.md': {
zh: { title: '智能星球可交互图标聚类策略', group: 'Earth', order: 17 },
en: { title: 'Intelligent Planet Interactable Clustering', group: 'Earth', order: 17 },
},
'earth-toolbar-overlay-coordination.md': {
zh: { title: '智能星球工具栏与浮层协同', group: 'Earth', order: 17 },
en: { title: 'Intelligent Planet Toolbar and Overlay Coordination', group: 'Earth', order: 17 },
zh: { title: '智能星球工具栏与浮层协同', group: 'Earth', order: 18 },
en: { title: 'Intelligent Planet Toolbar and Overlay Coordination', group: 'Earth', order: 18 },
},
'frontend-admin-frontend-context.md': {
zh: { title: '控制台前端结构', group: 'Frontend', order: 20 },

View File

@@ -138,6 +138,7 @@ AI_PROVIDER_IMAGE_NAME="${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}"
AI_PROVIDER_CONTAINER_NAME="${AI_PROVIDER_CONTAINER_NAME:-planet_aiprovider}"
PLANET_AI_PROVIDER_RUNTIME_ENV_FILE="${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-$PLANET_STATE_DIR/aiprovider_runtime.env}"
PLANET_EMPTY_UV_CONFIG_FILE="$PLANET_STATE_DIR/uv.empty.toml"
PLANET_TUNA_UV_CONFIG_FILE="$PLANET_STATE_DIR/uv.tuna.toml"
PLANET_UV_CONFIG_FILE="${PLANET_UV_CONFIG_FILE:-}"
AI_PROVIDER_RECREATE_REQUIRED=0
START_RUN_ACTIVE=0
@@ -1111,55 +1112,94 @@ run_with_retry() {
run_uv_sync_with_mirror_fallback() {
local log_file="$1"
local lock_digest_before
local sync_status
lock_digest_before="$(uv_lock_digest)"
if run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"uv sync 默认源失败,已重试 ${DEPENDENCY_INSTALL_MAX_RETRIES} 次,准备切换清华源" \
"uv sync --frozen" \
run_command_quiet_unless_verbose "$log_file" uv sync --frozen --group dev; then
assert_uv_lock_unchanged "$lock_digest_before" "uv sync --frozen"
return 0
fi
assert_uv_lock_unchanged "$lock_digest_before" "uv sync --frozen"
configure_uv_tuna_index
set_wait_detail "已写入 uv.toml 清华源,重新执行 uv sync"
set_wait_detail "已准备临时清华源配置,重新执行 uv sync"
lock_digest_before="$(uv_lock_digest)"
run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"uv 环境初始化失败,清华源重试 ${DEPENDENCY_INSTALL_MAX_RETRIES} 次后仍失败" \
"uv sync --frozen (清华源)" \
run_uv_sync_with_tuna_config "$log_file"
sync_status=$?
assert_uv_lock_unchanged "$lock_digest_before" "uv sync --frozen (清华源)"
return "$sync_status"
}
run_uv_sync_with_tuna_config() {
local log_file="$1"
UV_CONFIG_FILE="$PLANET_TUNA_UV_CONFIG_FILE" \
run_command_quiet_unless_verbose "$log_file" uv sync --frozen --group dev
}
configure_uv_tuna_index() {
local uv_config="$SCRIPT_DIR/uv.toml"
local uv_config="$PLANET_TUNA_UV_CONFIG_FILE"
if [ -f "$uv_config" ] &&
grep -Fq 'name = "tsinghua"' "$uv_config" 2>/dev/null &&
grep -Fq "$PLANET_UV_TUNA_INDEX_URL" "$uv_config" 2>/dev/null &&
grep -Eq '^[[:space:]]*default[[:space:]]*=[[:space:]]*true' "$uv_config" 2>/dev/null; then
log_note "uv.toml 已配置清华源,直接重试 uv sync"
return 0
fi
if [ -f "$uv_config" ] && grep -Eq '^[[:space:]]*default[[:space:]]*=[[:space:]]*true' "$uv_config" 2>/dev/null; then
log_warn "uv.toml 已存在默认 index未覆盖用户配置"
log_note "如需手动切换清华源,可添加 [[index]] name=\"tsinghua\" 并设置 default=true"
log_note "临时 uv 清华源配置已存在,直接重试 uv sync"
return 0
fi
{
if [ -s "$uv_config" ]; then
printf "\n"
fi
printf '[[index]]\n'
printf 'name = "tsinghua"\n'
printf 'url = "%s"\n' "$PLANET_UV_TUNA_INDEX_URL"
printf 'default = true\n'
} >> "$uv_config"
} > "$uv_config"
log_note "已向 uv.toml 添加清华 PyPI 源: ${PLANET_UV_TUNA_INDEX_URL}"
chmod 600 "$uv_config" 2>/dev/null || true
log_note "已写入临时 uv 清华源配置: ${uv_config}"
}
uv_lock_digest() {
local lock_file="$SCRIPT_DIR/uv.lock"
if [ ! -f "$lock_file" ]; then
printf "missing\n"
return 0
fi
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$lock_file" | awk '{print $1}'
return 0
fi
cksum "$lock_file" | awk '{print $1 ":" $2}'
}
assert_uv_lock_unchanged() {
local expected_digest="$1"
local action_label="$2"
local current_digest
current_digest="$(uv_lock_digest)"
if [ "$current_digest" = "$expected_digest" ]; then
return 0
fi
log_error "${action_label} 修改了 uv.lock新环境依赖安装不允许污染 lockfile"
log_note "请还原 uv.lock并只在明确升级依赖时手动运行 uv lock"
exit 1
}
install_system_package() {
@@ -1763,19 +1803,15 @@ ensure_frontend_deps() {
cd "$SCRIPT_DIR/frontend"
: > "$log_file"
set_wait_detail "检查 Vite Bun 入口是否已安装"
if [ ! -f "$FRONTEND_VITE_ENTRY" ]; then
log_warn "前端依赖缺失,正在执行 bun install (${FRONTEND_RUNTIME_SOURCE})"
set_wait_detail "执行 ${FRONTEND_RUNTIME_SOURCE} bun install"
if ! run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"前端依赖安装失败,已重试 ${DEPENDENCY_INSTALL_MAX_RETRIES}" \
"bun install" \
run_command_quiet_unless_verbose "$log_file" "$FRONTEND_RUNTIME_BIN" install; then
tail -20 "$log_file" 2>/dev/null || true
exit 1
fi
set_wait_detail "同步前端依赖"
if ! run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"前端依赖安装失败,已重试 ${DEPENDENCY_INSTALL_MAX_RETRIES}" \
"bun install" \
run_command_quiet_unless_verbose "$log_file" "$FRONTEND_RUNTIME_BIN" install; then
tail -20 "$log_file" 2>/dev/null || true
exit 1
fi
if [ ! -f "$FRONTEND_VITE_ENTRY" ]; then

View File

@@ -1,6 +1,6 @@
[project]
name = "planet"
version = "0.66.2"
version = "0.68.1"
description = "智能星球计划 - 态势感知系统"
requires-python = ">=3.14"
dependencies = [

2
uv.lock generated
View File

@@ -757,7 +757,7 @@ wheels = [
[[package]]
name = "planet"
version = "0.66.2"
version = "0.68.1"
source = { virtual = "." }
dependencies = [
{ name = "aiofiles" },