Compare commits

..

10 Commits
v0.31.3 ... zg

Author SHA1 Message Date
opencode
8f67d8d108 feat: add ue_scene websocket integration 2026-05-09 15:51:46 +08:00
linkong
61baee00f6 Merge pull request 'dev' (#8) from dev into main
Reviewed-on: #8
2026-04-21 21:30:45 +00:00
rayd1o
0082cf3fbd release: bump version to 0.33.0 2026-04-22 05:28:54 +08:00
rayd1o
3ae4acdff8 release: bump version to 0.32.0 2026-04-22 04:41:39 +08:00
linkong
620190819b Merge pull request 'dev' (#5) from dev into main
Reviewed-on: #5
2026-04-13 23:41:04 +00:00
linkong
f67d6bde60 Merge pull request 'codex/aiprovider-foundation' (#4) from codex/aiprovider-foundation into main
Reviewed-on: #4
2026-04-07 09:35:48 +00:00
linkong
36672e4c53 Merge pull request 'dev' (#3) from dev into main
Reviewed-on: #3
2026-03-25 09:25:38 +00:00
linkong
506402ce16 Merge pull request 'dev' (#2) from dev into main
Reviewed-on: #2
2026-03-20 21:17:30 +00:00
linkong
9d135bf2e1 revert 49a9c33836
revert feat(earth): toolbar and zoom improvements

- Add box-sizing/padding normalization to toolbar buttons
- Remove zoom slider, implement click/hold zoom behavior (+/- buttons)
- Add 10% step on click, 1% continuous on hold
- Fix satellite init: show satellite points immediately, delay trail visibility
- Fix breathing effect: faster pulse, wider opacity range
- Add toggle-cables functionality with visibility state
- Initialize satellites and cables as visible by default
2026-03-20 21:16:45 +00:00
linkong
49a9c33836 feat(earth): toolbar and zoom improvements
- Add box-sizing/padding normalization to toolbar buttons
- Remove zoom slider, implement click/hold zoom behavior (+/- buttons)
- Add 10% step on click, 1% continuous on hold
- Fix satellite init: show satellite points immediately, delay trail visibility
- Fix breathing effect: faster pulse, wider opacity range
- Add toggle-cables functionality with visibility state
- Initialize satellites and cables as visible by default
2026-03-20 17:13:02 +08:00
40 changed files with 3987 additions and 180 deletions

View File

@@ -22,3 +22,5 @@
- [ ] 可选优化(非必做):将 BGP incident/collector 标点改为 HTML marker参考 worldmonitor 的 `htmlElementsData` 思路),实现近乎固定屏幕尺寸与更高密度可点击性
- [ ] 保持 Earth 当前这批纯个人偏好设置继续走本地持久化:`旋转模式`、HUD 面板显示/隐藏、`地形透明度` 暂不升级到后端系统设置,避免把设备级偏好过早做成全局配置
- [ ] 如果后续明确需要“账号级同步 Earth 偏好”,再单独设计 `Earth user preferences`:优先按用户维度而不是全局系统设置保存,并规划 `localStorage -> backend` 的平滑迁移策略
- [ ] 把 Earth 态势新闻源从 [earth_news.py](/home/ray/dev/linkong/planet/backend/app/services/earth_news.py) 的硬编码列表抽成可配置目录,优先保持当前“实时聚合”链路不变,只先解决新闻源不可配置的问题
- [ ] 为 Earth 态势新闻设计后续采集器化方案:明确新闻数据模型、去重策略、区域映射、过期清理和 Earth/AI 复用方式,再决定何时把新闻从实时抓取升级成正式 collector

View File

@@ -1 +1 @@
0.31.3
0.33.0

View File

@@ -420,6 +420,7 @@ async def list_datasources(
collector_list.append(
{
"id": datasource.id,
"source": datasource.source,
"name": datasource.name,
"module": datasource.module,
"priority": datasource.priority,

View File

@@ -1,7 +1,6 @@
"""WebSocket API endpoints"""
import asyncio
import json
import logging
from datetime import UTC, datetime
from typing import Optional
@@ -12,9 +11,20 @@ from jose import jwt, JWTError
from app.core.config import settings
from app.core.time import to_iso8601_utc
from app.core.websocket.manager import manager
from app.core.websocket.ue_scene import expand_scene_payload_for_transport, ue_scene_state_store
from app.db.session import async_session_factory
logger = logging.getLogger(__name__)
router = APIRouter()
SUPPORTED_CHANNELS = [
"gpu_clusters",
"submarine_cables",
"ixp_nodes",
"alerts",
"dashboard",
"datasource_tasks",
"ue_scene",
]
async def authenticate_token(token: str) -> Optional[dict]:
@@ -50,18 +60,12 @@ async def websocket_endpoint(
await websocket.send_json(
{
"type": "connection_established",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": {
"connection_id": f"conn_{user_id}",
"server_version": settings.VERSION,
"heartbeat_interval": 30,
"supported_channels": [
"gpu_clusters",
"submarine_cables",
"ixp_nodes",
"alerts",
"dashboard",
"datasource_tasks",
],
"supported_channels": SUPPORTED_CHANNELS,
},
}
)
@@ -74,26 +78,79 @@ async def websocket_endpoint(
await websocket.send_json(
{
"type": "heartbeat",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": {"action": "pong", "timestamp": to_iso8601_utc(datetime.now(UTC))},
}
)
elif data.get("type") == "subscribe":
channels = data.get("data", {}).get("channels", [])
requested_channels = data.get("data", {}).get("channels", [])
channels = manager.subscribe(websocket, requested_channels)
await websocket.send_json(
{
"type": "subscription_confirmed",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": {"action": "subscribe", "channels": channels},
}
)
elif data.get("type") == "sync_request":
sync_data = data.get("data", {})
channel = sync_data.get("channel")
if channel == "ue_scene":
async with async_session_factory() as session:
scene_payloads = await ue_scene_state_store.get_sync_payloads(
session,
last_sequence=sync_data.get("last_sequence"),
reason=sync_data.get("reason"),
)
for scene_payload in scene_payloads:
for transport_payload in expand_scene_payload_for_transport(scene_payload):
await websocket.send_json(
{
"type": "data_frame",
"channel": "ue_scene",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": transport_payload,
}
)
else:
await websocket.send_json(
{
"type": "error",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": {
"message": f"Unsupported sync channel: {channel}",
},
}
)
elif data.get("type") == "control_frame":
await websocket.send_json(
{"type": "control_acknowledged", "data": {"received": True}}
{
"type": "control_acknowledged",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": {
"target": data.get("data", {}).get("target"),
"command": data.get("data", {}).get("command"),
"accepted": True,
},
}
)
else:
await websocket.send_json({"type": "ack", "data": {"received": True}})
await websocket.send_json(
{
"type": "ack",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": {"received": True},
}
)
except asyncio.TimeoutError:
await websocket.send_json({"type": "heartbeat", "data": {"action": "ping"}})
await websocket.send_json(
{
"type": "heartbeat",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": {"action": "ping"},
}
)
except WebSocketDisconnect:
pass

View File

@@ -30,6 +30,7 @@ COLLECTOR_URL_KEYS = {
"iptoasn_prefix_geo": "iptoasn.combined_url",
"opengeofeed_prefix_geo": "opengeofeed.public_csv_url",
"nro_delegated_prefix_geo": "nro.delegated_stats_url",
"news_live_streams": "news_live_streams.channels_url",
}

View File

@@ -86,3 +86,11 @@ opengeofeed:
nro:
# NRO delegated stats 下载地址
delegated_stats_url: "https://ftp.ripe.net/pub/stats/ripencc/nro-stats/latest/nro-delegated-stats"
news_live_streams:
# IPTV-org 频道元数据 JSON
channels_url: "https://iptv-org.github.io/api/channels.json"
# IPTV-org 频道播放流 JSON
streams_url: "https://iptv-org.github.io/api/streams.json"
# IPTV-org 台标 JSON
logos_url: "https://iptv-org.github.io/api/logos.json"

View File

@@ -1,11 +1,12 @@
"""Data broadcaster for WebSocket connections"""
"""Data broadcaster for WebSocket connections."""
import asyncio
from datetime import UTC, datetime
from typing import Dict, Any, Optional
from typing import Any, Dict
from app.core.time import to_iso8601_utc
from app.core.websocket.manager import manager
from app.db.session import async_session_factory
@@ -45,6 +46,30 @@ class DataBroadcaster:
pass
await asyncio.sleep(interval)
async def broadcast_ue_scene(self, interval: int = 5) -> None:
"""Broadcast UE scene updates for nDisplay primary nodes."""
from app.core.websocket.ue_scene import expand_scene_payload_for_transport, ue_scene_state_store
while self.running:
try:
async with async_session_factory() as session:
scene_payloads = await ue_scene_state_store.get_broadcast_payloads(session)
for scene_data in scene_payloads:
for transport_payload in expand_scene_payload_for_transport(scene_data):
await manager.broadcast(
{
"type": "data_frame",
"channel": "ue_scene",
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"data": transport_payload,
},
channel="ue_scene",
)
except Exception:
pass
await asyncio.sleep(interval)
async def broadcast_alert(self, alert: Dict[str, Any]):
"""Broadcast an alert to all connected clients"""
await manager.broadcast(
@@ -75,7 +100,7 @@ class DataBroadcaster:
"timestamp": to_iso8601_utc(datetime.now(UTC)),
"payload": data,
},
channel=channel if channel in manager.active_connections else "all",
channel=channel,
)
async def broadcast_datasource_task_update(self, data: Dict[str, Any]):
@@ -95,6 +120,7 @@ class DataBroadcaster:
if not self.running:
self.running = True
self.tasks["dashboard"] = asyncio.create_task(self.broadcast_stats(5))
self.tasks["ue_scene"] = asyncio.create_task(self.broadcast_ue_scene(5))
def stop(self):
"""Stop all broadcasters"""

View File

@@ -1,9 +1,7 @@
"""WebSocket Connection Manager"""
"""WebSocket connection manager with channel subscriptions."""
from typing import Dict, Optional, Set
import json
import asyncio
from typing import Dict, Set, Optional
from datetime import datetime
from fastapi import WebSocket
import redis.asyncio as redis
@@ -11,17 +9,25 @@ from app.core.config import settings
class ConnectionManager:
"""Manages WebSocket connections"""
"""Manage user connections and channel subscriptions."""
def __init__(self):
self.active_connections: Dict[str, Set[WebSocket]] = {} # user_id -> connections
def __init__(self) -> None:
self.user_connections: Dict[str, Set[WebSocket]] = {}
self.socket_users: Dict[WebSocket, str] = {}
self.channel_connections: Dict[str, Set[WebSocket]] = {}
self.socket_channels: Dict[WebSocket, Set[str]] = {}
self.redis_client: Optional[redis.Redis] = None
async def connect(self, websocket: WebSocket, user_id: str):
@property
def active_connections(self) -> Dict[str, Set[WebSocket]]:
"""Compatibility alias for existing callers."""
return self.user_connections
async def connect(self, websocket: WebSocket, user_id: str) -> None:
await websocket.accept()
if user_id not in self.active_connections:
self.active_connections[user_id] = set()
self.active_connections[user_id].add(websocket)
self.user_connections.setdefault(user_id, set()).add(websocket)
self.socket_users[websocket] = user_id
self.socket_channels.setdefault(websocket, set())
if self.redis_client is None:
redis_url = settings.REDIS_URL
@@ -35,32 +41,69 @@ class ConnectionManager:
decode_responses=True,
)
def disconnect(self, websocket: WebSocket, user_id: str):
if user_id in self.active_connections:
self.active_connections[user_id].discard(websocket)
if not self.active_connections[user_id]:
del self.active_connections[user_id]
def disconnect(self, websocket: WebSocket, user_id: str) -> None:
self.unsubscribe(websocket, list(self.socket_channels.get(websocket, set())))
async def send_personal_message(self, message: dict, user_id: str):
if user_id in self.active_connections:
for connection in self.active_connections[user_id]:
try:
await connection.send_json(message)
except Exception:
pass
if user_id in self.user_connections:
self.user_connections[user_id].discard(websocket)
if not self.user_connections[user_id]:
del self.user_connections[user_id]
async def broadcast(self, message: dict, channel: str = "all"):
self.socket_users.pop(websocket, None)
self.socket_channels.pop(websocket, None)
def subscribe(self, websocket: WebSocket, channels: list[str]) -> list[str]:
subscribed_channels: list[str] = []
socket_channel_set = self.socket_channels.setdefault(websocket, set())
for channel in channels:
normalized_channel = channel.strip()
if not normalized_channel:
continue
self.channel_connections.setdefault(normalized_channel, set()).add(websocket)
socket_channel_set.add(normalized_channel)
subscribed_channels.append(normalized_channel)
return subscribed_channels
def unsubscribe(self, websocket: WebSocket, channels: list[str]) -> None:
socket_channel_set = self.socket_channels.setdefault(websocket, set())
for channel in channels:
normalized_channel = channel.strip()
if not normalized_channel:
continue
if normalized_channel in self.channel_connections:
self.channel_connections[normalized_channel].discard(websocket)
if not self.channel_connections[normalized_channel]:
del self.channel_connections[normalized_channel]
socket_channel_set.discard(normalized_channel)
async def send_personal_message(self, message: dict, user_id: str) -> None:
for connection in list(self.user_connections.get(user_id, set())):
try:
await connection.send_json(message)
except Exception:
self.disconnect(connection, user_id)
async def broadcast(self, message: dict, channel: str = "all") -> None:
if channel == "all":
for user_id in self.active_connections:
await self.send_personal_message(message, user_id)
targets = list(self.socket_users.keys())
else:
await self.send_personal_message(message, channel)
targets = list(self.channel_connections.get(channel, set()))
async def close_all(self):
for user_id in self.active_connections:
for connection in self.active_connections[user_id]:
await connection.close()
self.active_connections.clear()
for connection in targets:
try:
await connection.send_json(message)
except Exception:
user_id = self.socket_users.get(connection)
if user_id is not None:
self.disconnect(connection, user_id)
async def close_all(self) -> None:
for websocket, user_id in list(self.socket_users.items()):
await websocket.close()
self.disconnect(websocket, user_id)
manager = ConnectionManager()

View File

@@ -0,0 +1,562 @@
"""UE scene state built from visualization aggregate output."""
import asyncio
import json
import zlib
from datetime import UTC, datetime
from typing import TYPE_CHECKING, Any, Dict, List
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.time import to_iso8601_utc
from app.models.bgp_anomaly import BGPAnomaly
from app.models.bgp_incident import BGPIncident
if TYPE_CHECKING:
from app.models.collected_data import CollectedData
DEFAULT_LAYER_ORDER = (
"satellites",
"supercomputers",
"gpu_clusters",
"submarine_cables",
"landing_points",
"bgp_anomalies",
"bgp_incidents",
"bgp_collectors",
"alerts",
)
FULL_RESYNC_REASONS = {"initial_connect", "manual_resync", "profile_changed"}
UE_SCENE_LAYER_CHUNK_ITEM_LIMITS = {
"satellites": 500,
"supercomputers": 200,
"gpu_clusters": 100,
"submarine_cables": 25,
"landing_points": 250,
"bgp_anomalies": 100,
"bgp_incidents": 100,
"bgp_collectors": 100,
"alerts": 100,
}
def _default_display_profile() -> Dict[str, Any]:
return {
"profile_id": "polarized-wall-a",
"stereo_mode": "polarized",
"screen_width_m": 3.0,
"screen_height_m": 2.0,
"target_refresh_hz": 120,
}
def _default_camera_state() -> Dict[str, Any]:
return {
"mode": "auto_cruise",
"path_id": "global_overview",
"fov": 42.0,
}
def _stable_json(value: Any) -> str:
return json.dumps(value, ensure_ascii=True, sort_keys=True, separators=(",", ":"))
def _entity_type_for_layer(layer_name: str) -> str:
return {
"satellites": "satellite",
"supercomputers": "supercomputer",
"gpu_clusters": "gpu_cluster",
"submarine_cables": "submarine_cable",
"landing_points": "landing_point",
"bgp_anomalies": "bgp_anomaly",
"bgp_incidents": "bgp_incident",
"bgp_collectors": "bgp_collector",
"alerts": "alert",
}[layer_name]
def _default_visual_for_layer(layer_name: str) -> Dict[str, Any]:
visuals = {
"satellites": {"style": "satellite_marker", "size": 0.7, "color": "#9BDBFF"},
"supercomputers": {"style": "supercomputer_marker", "size": 1.2, "color": "#FF6B6B"},
"gpu_clusters": {"style": "pulse_marker", "size": 1.0, "color": "#FF8C42"},
"submarine_cables": {"style": "cable_arc", "width": 2.0, "color": "#4ECDC4"},
"landing_points": {"style": "landing_point_marker", "size": 0.8, "color": "#45B7D1"},
"bgp_anomalies": {"style": "anomaly_marker", "size": 1.0, "color": "#FFB703"},
"bgp_incidents": {"style": "incident_marker", "size": 1.2, "color": "#E63946"},
"bgp_collectors": {"style": "collector_marker", "size": 0.9, "color": "#7B9ACC"},
"alerts": {"style": "alert_marker", "size": 1.0, "color": "#FFD166"},
}
return visuals[layer_name]
def _empty_scene_layers() -> Dict[str, Dict[str, Any]]:
return {
layer_name: {"revision": 0, "items": {}}
for layer_name in DEFAULT_LAYER_ORDER
}
def _empty_changes() -> Dict[str, Dict[str, List[Any]]]:
return {
layer_name: {"added": [], "updated": [], "removed": []}
for layer_name in DEFAULT_LAYER_ORDER
}
def _point_geo(coordinates: List[Any]) -> Dict[str, Any]:
lng = coordinates[0] if len(coordinates) > 0 else None
lat = coordinates[1] if len(coordinates) > 1 else None
alt = coordinates[2] if len(coordinates) > 2 else 0.0
return {"lat": lat, "lng": lng, "alt": alt}
def _path_geo(geometry_type: str, coordinates: Any) -> Dict[str, Any]:
if geometry_type == "LineString":
return {
"path": [
{"lat": point[1], "lng": point[0], "alt": point[2] if len(point) > 2 else 0.0}
for point in coordinates
if isinstance(point, list) and len(point) >= 2
]
}
if geometry_type == "MultiLineString":
return {
"segments": [
[
{"lat": point[1], "lng": point[0], "alt": point[2] if len(point) > 2 else 0.0}
for point in line
if isinstance(point, list) and len(point) >= 2
]
for line in coordinates
if isinstance(line, list)
]
}
return {}
def _item_identifier(layer_name: str, feature: Dict[str, Any], index: int) -> str:
properties = feature.get("properties") or {}
feature_id = feature.get("id") or properties.get("id") or properties.get("source_id")
return f"{layer_name}:{feature_id or index}"
def _item_revision(item: Dict[str, Any]) -> int:
return zlib.crc32(_stable_json(item).encode("utf-8")) & 0xFFFFFFFF
def _layer_revision(items: Dict[str, Dict[str, Any]]) -> int:
if not items:
return 0
joined = "|".join(
f"{item_id}:{items[item_id]['revision']}"
for item_id in sorted(items)
)
return zlib.crc32(joined.encode("utf-8")) & 0xFFFFFFFF
def _serialize_feature(layer_name: str, feature: Dict[str, Any], index: int) -> Dict[str, Any]:
properties = dict(feature.get("properties") or {})
geometry = feature.get("geometry") or {}
geometry_type = geometry.get("type", "")
coordinates = geometry.get("coordinates") or []
item_id = _item_identifier(layer_name, feature, index)
geo: Dict[str, Any] = {}
if geometry_type == "Point":
geo = _point_geo(coordinates)
elif geometry_type in {"LineString", "MultiLineString"}:
geo = _path_geo(geometry_type, coordinates)
title = (
properties.get("name")
or properties.get("Name")
or properties.get("title")
or item_id
)
subtitle_parts = [
properties.get("city"),
properties.get("country"),
properties.get("region"),
]
item = {
"id": item_id,
"entity_type": _entity_type_for_layer(layer_name),
"geo": geo,
"visual": {
**_default_visual_for_layer(layer_name),
**({"color": properties["color"]} if properties.get("color") else {}),
},
"metrics": properties,
"labels": {
"title": title,
"subtitle": ", ".join([part for part in subtitle_parts if part]),
},
"status": {
"health": properties.get("status", "normal"),
"alert_level": properties.get("severity", "none"),
},
}
item["revision"] = _item_revision(item)
return item
async def build_visualization_scene_state(db: AsyncSession) -> Dict[str, Any]:
from app.api.v1.visualization import (
_build_landing_point_cable_maps,
_filter_known_records,
_load_current_collected_data_by_sources,
build_anomaly_geography_hints,
build_incident_geography_hints,
convert_bgp_anomalies_to_geojson,
convert_bgp_collectors_to_geojson,
convert_bgp_incidents_to_geojson,
convert_cable_to_geojson,
convert_gpu_cluster_to_geojson,
convert_landing_point_to_geojson,
convert_satellite_to_geojson,
convert_supercomputer_to_geojson,
)
from app.services.bgp_collectors import build_bgp_collector_coverage
records_by_source = await _load_current_collected_data_by_sources(
db,
[
"arcgis_cables",
"arcgis_landing_points",
"arcgis_cable_landing_relation",
"celestrak_tle",
"top500",
"epoch_ai_gpu",
],
)
cables_records = records_by_source.get("arcgis_cables", [])
landing_point_records = records_by_source.get("arcgis_landing_points", [])
relation_records = records_by_source.get("arcgis_cable_landing_relation", [])
satellites_records = _filter_known_records(records_by_source.get("celestrak_tle", []))
supercomputer_records = _filter_known_records(records_by_source.get("top500", []))
gpu_records = _filter_known_records(records_by_source.get("epoch_ai_gpu", []))
bgp_anomalies_result = await db.execute(
select(BGPAnomaly)
.where(BGPAnomaly.status == "active")
.order_by(BGPAnomaly.created_at.desc())
.limit(200)
)
bgp_anomalies = list(bgp_anomalies_result.scalars().all())
bgp_anomaly_geography_hints = await build_anomaly_geography_hints(db, bgp_anomalies)
bgp_incidents_result = await db.execute(
select(BGPIncident)
.where(BGPIncident.status == "active")
.order_by(BGPIncident.created_at.desc())
.limit(100)
)
bgp_incidents = list(bgp_incidents_result.scalars().all())
bgp_incident_geography_hints = await build_incident_geography_hints(db, bgp_incidents)
bgp_collector_coverage = await build_bgp_collector_coverage(
db,
source_filter=("ris_live_bgp", "bgpstream_bgp"),
)
bgp_coverage_by_collector = {
item["collector"]: item
for item in bgp_collector_coverage
if item.get("collector")
}
city_to_cable_ids_map, cable_id_to_name_map = _build_landing_point_cable_maps(
relation_records,
cables_records,
)
aggregate = {
"satellites": convert_satellite_to_geojson(satellites_records),
"supercomputers": convert_supercomputer_to_geojson(supercomputer_records),
"gpu_clusters": convert_gpu_cluster_to_geojson(gpu_records),
"submarine_cables": convert_cable_to_geojson(cables_records),
"landing_points": convert_landing_point_to_geojson(
landing_point_records,
city_to_cable_ids_map,
cable_id_to_name_map,
),
"bgp_anomalies": convert_bgp_anomalies_to_geojson(
bgp_anomalies,
bgp_anomaly_geography_hints,
),
"bgp_incidents": convert_bgp_incidents_to_geojson(
bgp_incidents,
bgp_incident_geography_hints,
),
"bgp_collectors": convert_bgp_collectors_to_geojson(bgp_coverage_by_collector),
"alerts": {"type": "FeatureCollection", "features": []},
}
layers = _empty_scene_layers()
total_records = 0
for layer_name in DEFAULT_LAYER_ORDER:
feature_collection = aggregate.get(layer_name) or {"features": []}
items = {
item["id"]: item
for index, feature in enumerate(feature_collection.get("features", []), start=1)
for item in [_serialize_feature(layer_name, feature, index)]
}
layers[layer_name]["items"] = items
layers[layer_name]["revision"] = _layer_revision(items)
total_records += len(items)
timestamp = to_iso8601_utc(datetime.now(UTC))
state_hash = zlib.crc32(
_stable_json(
{
layer_name: {
item_id: item["revision"]
for item_id, item in layers[layer_name]["items"].items()
}
for layer_name in DEFAULT_LAYER_ORDER
}
).encode("utf-8")
) & 0xFFFFFFFF
return {
"generated_at": timestamp,
"state_hash": state_hash,
"total_records": total_records,
"layers": layers,
}
def _changes_exist(changes: Dict[str, Dict[str, List[Any]]]) -> bool:
return any(
layer_changes["added"] or layer_changes["updated"] or layer_changes["removed"]
for layer_changes in changes.values()
)
def build_incremental_changes(
previous_state: Dict[str, Any],
current_state: Dict[str, Any],
) -> Dict[str, Dict[str, List[Any]]]:
changes = _empty_changes()
for layer_name in DEFAULT_LAYER_ORDER:
previous_items = previous_state["layers"][layer_name]["items"]
current_items = current_state["layers"][layer_name]["items"]
previous_ids = set(previous_items)
current_ids = set(current_items)
for added_id in sorted(current_ids - previous_ids):
changes[layer_name]["added"].append(current_items[added_id])
for removed_id in sorted(previous_ids - current_ids):
changes[layer_name]["removed"].append(removed_id)
for common_id in sorted(previous_ids & current_ids):
if _stable_json(previous_items[common_id]) != _stable_json(current_items[common_id]):
changes[layer_name]["updated"].append(current_items[common_id])
return changes
def _full_payload_from_state(state: Dict[str, Any], sequence: int) -> Dict[str, Any]:
return {
"update_type": "full",
"sequence": sequence,
"cluster_time": state["generated_at"],
"display_profile": _default_display_profile(),
"camera_state": _default_camera_state(),
"payload": {
"meta": {
"generated_at": state["generated_at"],
"total_records": state["total_records"],
"state_hash": state["state_hash"],
},
"layers": {
layer_name: {
"revision": layer_data["revision"],
"items": [
layer_data["items"][item_id]
for item_id in sorted(layer_data["items"])
],
}
for layer_name, layer_data in state["layers"].items()
},
},
}
def _incremental_payload(
*,
current_state: Dict[str, Any],
sequence: int,
base_sequence: int,
changes: Dict[str, Dict[str, List[Any]]],
) -> Dict[str, Any]:
return {
"update_type": "incremental",
"sequence": sequence,
"base_sequence": base_sequence,
"cluster_time": current_state["generated_at"],
"changes": changes,
"meta": {
"generated_at": current_state["generated_at"],
"total_records": current_state["total_records"],
"state_hash": current_state["state_hash"],
},
}
def _noop_incremental_payload(state: Dict[str, Any], sequence: int) -> Dict[str, Any]:
return _incremental_payload(
current_state=state,
sequence=sequence,
base_sequence=sequence,
changes=_empty_changes(),
)
def expand_scene_payload_for_transport(scene_payload: Dict[str, Any]) -> List[Dict[str, Any]]:
"""Split oversized scene payloads into smaller transport-safe chunks."""
update_type = scene_payload.get("update_type")
if update_type != "full":
return [scene_payload]
payload = scene_payload.get("payload") or {}
layers = payload.get("layers") or {}
if not layers:
return [scene_payload]
chunks: List[Dict[str, Any]] = []
for layer_name in DEFAULT_LAYER_ORDER:
layer_data = layers.get(layer_name) or {}
items = list(layer_data.get("items") or [])
if not items:
continue
chunk_size = UE_SCENE_LAYER_CHUNK_ITEM_LIMITS.get(layer_name, 100)
total_layer_chunks = max(1, (len(items) + chunk_size - 1) // chunk_size)
for chunk_index, offset in enumerate(range(0, len(items), chunk_size), start=1):
chunk_items = items[offset : offset + chunk_size]
chunks.append(
{
**scene_payload,
"payload": {
"meta": {
**(payload.get("meta") or {}),
"chunked": True,
"layer_name": layer_name,
"layer_chunk_index": chunk_index,
"layer_chunk_count": total_layer_chunks,
},
"layers": {
layer_name: {
"revision": layer_data.get("revision", 0),
"items": chunk_items,
}
},
},
}
)
if not chunks:
return [scene_payload]
total_chunks = len(chunks)
for transport_index, chunk in enumerate(chunks, start=1):
chunk_payload = chunk.setdefault("payload", {})
chunk_meta = chunk_payload.setdefault("meta", {})
chunk_meta["transport_chunk_index"] = transport_index
chunk_meta["transport_chunk_count"] = total_chunks
return chunks
class UeSceneStateStore:
"""Maintain cached ue_scene state and a bounded incremental replay history."""
def __init__(self, history_limit: int = 20) -> None:
self.history_limit = history_limit
self.sequence = 0
self.state: Dict[str, Any] | None = None
self.history: List[Dict[str, Any]] = []
self.lock = asyncio.Lock()
async def _refresh_locked(self, db: AsyncSession) -> List[Dict[str, Any]]:
current_state = await build_visualization_scene_state(db)
if self.state is None:
self.sequence = 1
self.state = current_state
return [_full_payload_from_state(self.state, self.sequence)]
if current_state["state_hash"] == self.state["state_hash"]:
self.state = current_state
return []
previous_sequence = self.sequence
previous_state = self.state
self.sequence += 1
self.state = current_state
changes = build_incremental_changes(previous_state, current_state)
incremental = _incremental_payload(
current_state=current_state,
sequence=self.sequence,
base_sequence=previous_sequence,
changes=changes,
)
self.history.append(incremental)
if len(self.history) > self.history_limit:
self.history = self.history[-self.history_limit :]
return [incremental]
async def get_broadcast_payloads(self, db: AsyncSession) -> List[Dict[str, Any]]:
async with self.lock:
payloads = await self._refresh_locked(db)
return [payload for payload in payloads if payload["update_type"] == "full" or _changes_exist(payload.get("changes", {}))]
async def get_sync_payloads(
self,
db: AsyncSession,
*,
last_sequence: int | None,
reason: str | None,
) -> List[Dict[str, Any]]:
async with self.lock:
await self._refresh_locked(db)
if self.state is None:
return []
normalized_reason = (reason or "").strip()
if last_sequence is None or normalized_reason in FULL_RESYNC_REASONS:
return [_full_payload_from_state(self.state, self.sequence)]
if last_sequence == self.sequence:
return [_noop_incremental_payload(self.state, self.sequence)]
if last_sequence > self.sequence:
return [_full_payload_from_state(self.state, self.sequence)]
replay_payloads = [
payload
for payload in self.history
if payload["base_sequence"] >= last_sequence
and payload["sequence"] > last_sequence
]
if replay_payloads:
expected_base = last_sequence
ordered_payloads: List[Dict[str, Any]] = []
for payload in replay_payloads:
if payload["base_sequence"] != expected_base:
return [_full_payload_from_state(self.state, self.sequence)]
ordered_payloads.append(payload)
expected_base = payload["sequence"]
if ordered_payloads and ordered_payloads[-1]["sequence"] == self.sequence:
return ordered_payloads
return [_full_payload_from_state(self.state, self.sequence)]
ue_scene_state_store = UeSceneStateStore()

View File

@@ -1,10 +1,16 @@
from __future__ import annotations
import asyncio
import base64
from datetime import UTC, datetime
from typing import Any
from urllib.parse import urlparse
import httpx
from sqlalchemy import select
from app.core.data_sources import get_data_sources_config
from app.models.datasource_config import DataSourceConfig
from app.services.collectors.base import BaseCollector
@@ -18,52 +24,537 @@ class NewsLiveStreamsCollector(BaseCollector):
data_type = "news_live_stream"
fail_on_empty = False
DEFAULT_TIMEOUT = 45.0
DEFAULT_HEADERS = {
"User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
"Accept": "application/json",
}
RESPONSE_CANDIDATE_KEYS = ("sources", "streams", "channels", "items", "results", "data")
DEFAULT_ADAPTER = "iptv_org"
DEFAULT_IPTV_ORG_STREAMS_URL = "https://iptv-org.github.io/api/streams.json"
DEFAULT_IPTV_ORG_LOGOS_URL = "https://iptv-org.github.io/api/logos.json"
DEFAULT_IPTV_ORG_NEWS_CATEGORIES = ("news", "business", "weather")
DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES = ("music", "sports", "kids", "entertainment")
DEFAULT_IPTV_ORG_MAX_SOURCES = 120
async def fetch(self) -> list[dict[str, Any]]:
request_url = (self._resolved_url or "").strip()
if not request_url:
return []
async with httpx.AsyncClient(timeout=45.0, follow_redirects=True) as client:
response = await client.get(
datasource_config = await self._load_datasource_config()
effective_config = self._get_effective_config(datasource_config)
adapter = str(effective_config.get("adapter") or "").strip().lower()
if adapter == "iptv_org":
return await self._fetch_iptv_org(request_url, effective_config)
request_headers = self._build_request_headers(datasource_config)
request_config = self._get_request_config(datasource_config)
request_params = self._build_request_params(datasource_config)
request_json = self._build_request_json_body(datasource_config)
request_data = self._build_request_form_body(datasource_config)
timeout = self._get_timeout(datasource_config)
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
response = await client.request(
request_config["method"],
request_url,
headers={
"User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)",
"Accept": "application/json",
},
headers=request_headers,
params=request_params or None,
json=request_json,
data=request_data,
)
response.raise_for_status()
return self.parse_response(response.json())
return self.parse_response(
response.json(),
response_path=request_config["response_path"],
)
def parse_response(self, response: Any) -> list[dict[str, Any]]:
if isinstance(response, dict):
candidates = response.get("sources") or response.get("streams") or response.get("data") or []
elif isinstance(response, list):
candidates = response
async def _load_datasource_config(self) -> DataSourceConfig | None:
if not self._db_session:
return None
result = await self._db_session.execute(
select(DataSourceConfig)
.where(DataSourceConfig.name == self.name)
.where(DataSourceConfig.is_active.is_(True))
.limit(1)
)
return result.scalar_one_or_none()
def _get_effective_config(self, datasource_config: DataSourceConfig | None) -> dict[str, Any]:
payload = dict(datasource_config.config or {}) if datasource_config else {}
if payload:
return payload
yaml_config = get_data_sources_config()
return {
"adapter": self.DEFAULT_ADAPTER,
"streams_url": yaml_config.get_yaml_value("news_live_streams.streams_url")
or self.DEFAULT_IPTV_ORG_STREAMS_URL,
"logos_url": yaml_config.get_yaml_value("news_live_streams.logos_url")
or self.DEFAULT_IPTV_ORG_LOGOS_URL,
"news_categories": list(self.DEFAULT_IPTV_ORG_NEWS_CATEGORIES),
"exclude_categories": list(self.DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES),
"max_sources": self.DEFAULT_IPTV_ORG_MAX_SOURCES,
}
def _get_request_config(self, datasource_config: DataSourceConfig | None) -> dict[str, Any]:
payload = self._get_effective_config(datasource_config)
raw_method = payload.get("method") or payload.get("request_method") or "GET"
method = str(raw_method).strip().upper() or "GET"
if method not in {"GET", "POST"}:
method = "GET"
response_path = payload.get("response_path") or payload.get("payload_path") or payload.get("items_path")
if isinstance(response_path, str):
response_path = response_path.strip()
else:
candidates = []
response_path = None
return {
"method": method,
"response_path": response_path or None,
}
def _get_timeout(self, datasource_config: DataSourceConfig | None) -> float:
payload = self._get_effective_config(datasource_config)
try:
return float(payload.get("timeout", self.DEFAULT_TIMEOUT))
except (TypeError, ValueError):
return self.DEFAULT_TIMEOUT
def _build_request_headers(self, datasource_config: DataSourceConfig | None) -> dict[str, str]:
headers = dict(self.DEFAULT_HEADERS)
if datasource_config:
headers.update(self._normalize_headers(datasource_config.headers))
headers.update(self._build_auth_headers(datasource_config))
return headers
def _build_request_params(self, datasource_config: DataSourceConfig | None) -> dict[str, Any]:
params: dict[str, Any] = {}
if not datasource_config:
return params
payload = datasource_config.config or {}
candidate = payload.get("params") or payload.get("query_params")
if isinstance(candidate, dict):
params.update(candidate)
if datasource_config.auth_type == "api_key":
auth_config = datasource_config.auth_config or {}
if str(auth_config.get("in") or auth_config.get("location") or "header").lower() == "query":
api_key = auth_config.get("api_key")
key_name = auth_config.get("key_name") or auth_config.get("param_name") or "api_key"
if api_key and key_name:
params[str(key_name)] = api_key
return params
def _build_request_json_body(self, datasource_config: DataSourceConfig | None) -> Any:
if not datasource_config:
return None
payload = datasource_config.config or {}
body = payload.get("json_body")
if body is None and str(payload.get("body_type") or "").lower() in {"json", ""}:
candidate = payload.get("body")
if isinstance(candidate, (dict, list)):
body = candidate
return body
def _build_request_form_body(self, datasource_config: DataSourceConfig | None) -> Any:
if not datasource_config:
return None
payload = datasource_config.config or {}
form_body = payload.get("form_body")
if form_body is not None:
return form_body
if str(payload.get("body_type") or "").lower() == "form":
candidate = payload.get("body")
if isinstance(candidate, dict):
return candidate
return None
def _normalize_headers(self, headers: Any) -> dict[str, str]:
if not isinstance(headers, dict):
return {}
normalized: dict[str, str] = {}
for key, value in headers.items():
header_name = str(key).strip()
if not header_name or value is None:
continue
normalized[header_name] = str(value)
return normalized
def _build_auth_headers(self, datasource_config: DataSourceConfig | None) -> dict[str, str]:
if not datasource_config:
return {}
auth_type = str(datasource_config.auth_type or "none").lower()
auth_config = datasource_config.auth_config or {}
if auth_type == "bearer" and auth_config.get("token"):
return {"Authorization": f"Bearer {auth_config['token']}"}
if auth_type == "api_key" and auth_config.get("api_key"):
location = str(auth_config.get("in") or auth_config.get("location") or "header").lower()
if location == "query":
return {}
key_name = auth_config.get("key_name") or "X-API-Key"
return {str(key_name): str(auth_config["api_key"])}
if auth_type == "basic":
username = str(auth_config.get("username") or "")
password = str(auth_config.get("password") or "")
encoded = base64.b64encode(f"{username}:{password}".encode()).decode()
return {"Authorization": f"Basic {encoded}"}
return {}
def _extract_candidates(self, response: Any, response_path: str | None) -> list[Any]:
if response_path:
extracted = self._extract_from_path(response, response_path)
if isinstance(extracted, list):
return extracted
if isinstance(extracted, dict):
for key in self.RESPONSE_CANDIDATE_KEYS:
nested = extracted.get(key)
if isinstance(nested, list):
return nested
return [extracted]
if isinstance(response, dict):
for key in self.RESPONSE_CANDIDATE_KEYS:
nested = response.get(key)
if isinstance(nested, list):
return nested
return []
if isinstance(response, list):
return response
return []
def _extract_from_path(self, payload: Any, path: str) -> Any:
current = payload
for segment in (part.strip() for part in path.split(".") if part.strip()):
if isinstance(current, dict):
current = current.get(segment)
continue
if isinstance(current, list):
try:
current = current[int(segment)]
except (TypeError, ValueError, IndexError):
return None
continue
return None
return current
def _infer_source_type(self, item: dict[str, Any]) -> str:
explicit = str(item.get("source_type") or item.get("type") or "").strip().lower()
if explicit in {"iframe", "hls", "video", "external", "youtube"}:
return explicit
youtube_video_id = self._clean_text(
item.get("youtube_video_id")
or item.get("video_id")
or item.get("youtubeVideoId")
)
youtube_channel = self._clean_text(item.get("youtube_channel") or item.get("channel_handle"))
embed_url = self._clean_url(item.get("embed_url") or item.get("embed") or item.get("page_url"))
stream_url = self._clean_url(item.get("stream_url") or item.get("stream") or item.get("playback_url") or item.get("hls_url"))
homepage_url = self._clean_url(item.get("homepage_url") or item.get("source_url") or item.get("website"))
if youtube_video_id or youtube_channel:
return "youtube"
if stream_url.endswith(".m3u8"):
return "hls"
if stream_url:
return "video"
if embed_url:
parsed = urlparse(embed_url)
if "youtube.com" in (parsed.netloc or "") or "youtu.be" in (parsed.netloc or ""):
return "youtube"
return "iframe"
if homepage_url:
return "external"
return "iframe"
def _parse_enabled(self, item: dict[str, Any]) -> bool:
if "is_enabled" in item:
return self._to_bool(item.get("is_enabled"), default=True)
if "enabled" in item:
return self._to_bool(item.get("enabled"), default=True)
if "active" in item:
return self._to_bool(item.get("active"), default=True)
if "status" in item:
status = str(item.get("status") or "").strip().lower()
if status in {"disabled", "inactive", "offline"}:
return False
if status in {"enabled", "active", "online", "live"}:
return True
return True
def _to_bool(self, value: Any, *, default: bool) -> bool:
if isinstance(value, bool):
return value
if value in (None, ""):
return default
if isinstance(value, str):
lowered = value.strip().lower()
if lowered in {"1", "true", "yes", "on", "enabled", "active", "online", "live"}:
return True
if lowered in {"0", "false", "no", "off", "disabled", "inactive", "offline"}:
return False
return bool(value)
def _clean_text(self, value: Any) -> str:
if value is None:
return ""
return str(value).strip()
def _clean_url(self, value: Any) -> str:
text = self._clean_text(value)
if not text:
return ""
parsed = urlparse(text)
if parsed.scheme and parsed.scheme not in {"http", "https"}:
return ""
if parsed.scheme and not parsed.netloc:
return ""
return text
async def _fetch_iptv_org(self, channels_url: str, collector_config: dict[str, Any]) -> list[dict[str, Any]]:
streams_url = self._clean_url(collector_config.get("streams_url")) or self.DEFAULT_IPTV_ORG_STREAMS_URL
logos_url = self._clean_url(collector_config.get("logos_url")) or self.DEFAULT_IPTV_ORG_LOGOS_URL
news_categories = {
self._clean_text(value).lower()
for value in (collector_config.get("news_categories") or self.DEFAULT_IPTV_ORG_NEWS_CATEGORIES)
if self._clean_text(value)
}
exclude_categories = {
self._clean_text(value).lower()
for value in (collector_config.get("exclude_categories") or self.DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES)
if self._clean_text(value)
}
try:
max_sources = int(collector_config.get("max_sources", self.DEFAULT_IPTV_ORG_MAX_SOURCES))
except (TypeError, ValueError):
max_sources = self.DEFAULT_IPTV_ORG_MAX_SOURCES
timeout = self.DEFAULT_TIMEOUT
try:
timeout = float(collector_config.get("timeout", self.DEFAULT_TIMEOUT))
except (TypeError, ValueError):
timeout = self.DEFAULT_TIMEOUT
async with httpx.AsyncClient(timeout=timeout, follow_redirects=True) as client:
channels_payload, streams_payload, logos_payload = await self._gather_iptv_org_payloads(
client,
channels_url,
streams_url,
logos_url,
)
channels = channels_payload if isinstance(channels_payload, list) else []
streams = streams_payload if isinstance(streams_payload, list) else []
logos = logos_payload if isinstance(logos_payload, list) else []
logo_by_channel = {
self._clean_text(item.get("channel")): self._clean_url(item.get("url"))
for item in logos
if isinstance(item, dict) and self._clean_text(item.get("channel")) and self._clean_url(item.get("url"))
}
streams_by_channel: dict[str, list[dict[str, Any]]] = {}
for stream in streams:
if not isinstance(stream, dict):
continue
channel_id = self._clean_text(stream.get("channel"))
if not channel_id:
continue
streams_by_channel.setdefault(channel_id, []).append(stream)
normalized: list[dict[str, Any]] = []
for channel in channels:
if not isinstance(channel, dict):
continue
categories = [
self._clean_text(value).lower()
for value in (channel.get("categories") or [])
if self._clean_text(value)
]
if news_categories and not any(category in news_categories for category in categories):
continue
if exclude_categories and any(category in exclude_categories for category in categories):
continue
if channel.get("is_nsfw") is True:
continue
if channel.get("closed"):
continue
channel_id = self._clean_text(channel.get("id"))
if not channel_id:
continue
stream = self._pick_iptv_org_stream(streams_by_channel.get(channel_id) or [])
if not stream:
continue
stream_url = self._clean_url(stream.get("url"))
if not stream_url:
continue
name = self._clean_text(channel.get("name")) or channel_id
notes_parts = [
f"Imported from IPTV-org catalog ({channel_id})",
f"Categories: {', '.join(categories)}" if categories else "",
f"Quality: {self._clean_text(stream.get('quality'))}" if self._clean_text(stream.get("quality")) else "",
]
metadata = {
"provider": self._clean_text(channel.get("network")) or "IPTV-org",
"region": self._clean_text(channel.get("country")) or "Global",
"language": "und",
"source_type": "hls" if stream_url.endswith(".m3u8") else "video",
"embed_url": "",
"stream_url": stream_url,
"homepage_url": self._clean_url(channel.get("website")),
"poster_url": logo_by_channel.get(channel_id, ""),
"youtube_video_id": "",
"youtube_channel": "",
"sort_order": 400 + len(normalized),
"notes": "; ".join(part for part in notes_parts if part),
"is_enabled": True,
"collector_adapter": "iptv_org",
"channel_id": channel_id,
"categories": categories,
"quality": self._clean_text(stream.get("quality")),
"stream_label": self._clean_text(stream.get("label") or stream.get("title")),
"stream_referrer": self._clean_text(stream.get("referrer")),
"stream_user_agent": self._clean_text(stream.get("user_agent")),
}
normalized.append(
{
"source_id": channel_id,
"name": name,
"description": metadata["notes"],
"metadata": metadata,
"reference_date": datetime.now(UTC).isoformat(),
}
)
if len(normalized) >= max_sources:
break
return normalized
async def _gather_iptv_org_payloads(
self,
client: httpx.AsyncClient,
channels_url: str,
streams_url: str,
logos_url: str,
) -> tuple[Any, Any, Any]:
headers = dict(self.DEFAULT_HEADERS)
channels_payload, streams_payload, logos_payload = await asyncio.gather(
client.get(channels_url, headers=headers),
client.get(streams_url, headers=headers),
client.get(logos_url, headers=headers),
)
channels_payload.raise_for_status()
streams_payload.raise_for_status()
logos_payload.raise_for_status()
return channels_payload.json(), streams_payload.json(), logos_payload.json()
def _pick_iptv_org_stream(self, streams: list[dict[str, Any]]) -> dict[str, Any] | None:
if not streams:
return None
def score(stream: dict[str, Any]) -> tuple[int, int]:
url = self._clean_url(stream.get("url"))
quality = self._clean_text(stream.get("quality")).lower()
quality_score = 0
if quality.endswith("p"):
try:
quality_score = int(quality[:-1])
except ValueError:
quality_score = 0
stream_score = 1000 if url.endswith(".m3u8") else 0
return stream_score, quality_score
sorted_streams = sorted(streams, key=score, reverse=True)
return sorted_streams[0]
def parse_response(self, response: Any, *, response_path: str | None = None) -> list[dict[str, Any]]:
candidates = self._extract_candidates(response, response_path)
normalized: list[dict[str, Any]] = []
for index, item in enumerate(candidates):
if not isinstance(item, dict):
continue
stream_id = item.get("id") or item.get("source_id") or item.get("slug") or f"news-live-{index + 1}"
name = str(item.get("name") or item.get("title") or f"News Live {index + 1}").strip()
stream_id = (
item.get("id")
or item.get("source_id")
or item.get("slug")
or item.get("channel_id")
or item.get("code")
or f"news-live-{index + 1}"
)
name = self._clean_text(
item.get("name")
or item.get("title")
or item.get("channel")
or item.get("display_name")
or f"News Live {index + 1}"
)
if not name:
continue
source_type = self._infer_source_type(item)
stream_url = self._clean_url(
item.get("stream_url")
or item.get("stream")
or item.get("playback_url")
or item.get("hls_url")
or item.get("m3u8_url")
)
embed_url = self._clean_url(
item.get("embed_url")
or item.get("embed")
or item.get("page_url")
or (item.get("url") if source_type == "iframe" else "")
)
homepage_url = self._clean_url(
item.get("homepage_url")
or item.get("source_url")
or item.get("website")
or item.get("url")
)
metadata = {
"provider": item.get("provider") or item.get("publisher") or "Collector",
"region": item.get("region") or item.get("country") or "Global",
"language": item.get("language") or "und",
"source_type": item.get("source_type") or "iframe",
"embed_url": item.get("embed_url") or item.get("url") or "",
"stream_url": item.get("stream_url") or "",
"homepage_url": item.get("homepage_url") or item.get("source_url") or "",
"poster_url": item.get("poster_url") or "",
"provider": self._clean_text(item.get("provider") or item.get("publisher") or item.get("network")) or "Collector",
"region": self._clean_text(item.get("region") or item.get("country") or item.get("market")) or "Global",
"language": self._clean_text(item.get("language") or item.get("lang") or item.get("locale")) or "und",
"source_type": source_type,
"embed_url": embed_url,
"stream_url": stream_url,
"homepage_url": homepage_url,
"poster_url": self._clean_url(item.get("poster_url") or item.get("thumbnail_url") or item.get("logo_url")),
"youtube_video_id": self._clean_text(
item.get("youtube_video_id")
or item.get("video_id")
or item.get("youtubeVideoId")
),
"youtube_channel": self._clean_text(
item.get("youtube_channel")
or item.get("channel_handle")
or item.get("youtubeChannel")
),
"sort_order": item.get("sort_order", 200 + index),
"notes": item.get("notes") or item.get("description") or "",
"is_enabled": item.get("is_enabled", True),
"notes": self._clean_text(item.get("notes") or item.get("description") or item.get("summary")),
"is_enabled": self._parse_enabled(item),
}
normalized.append(
@@ -72,7 +563,7 @@ class NewsLiveStreamsCollector(BaseCollector):
"name": name,
"description": metadata["notes"],
"metadata": metadata,
"reference_date": item.get("reference_date", datetime.now(UTC).isoformat()),
"reference_date": item.get("reference_date") or datetime.now(UTC).isoformat(),
}
)

View File

@@ -17,7 +17,7 @@ TV_LIVE_SOURCE_COLLECTOR = "news_live_streams"
TV_LIVE_SOURCE_DATA_TYPE = "news_live_stream"
DEFAULT_TV_SETTINGS = {
"default_source_id": DEFAULT_TV_SOURCE_ID,
"default_source_id": DEFAULT_TV_SOURCE_ID,
"auto_fallback": True,
"sources": [
{
@@ -362,7 +362,7 @@ def _build_collected_tv_source(record: CollectedData, index: int) -> dict[str, A
"sort_order": metadata.get("sort_order", 200 + index),
"collector_source": record.source,
"notes": record.description or metadata.get("notes") or "",
"updated_at": to_iso8601_utc(record.updated_at or record.reference_date or datetime.now(UTC)),
"updated_at": to_iso8601_utc(record.collected_at or record.reference_date or datetime.now(UTC)),
},
index=index,
)

View File

@@ -0,0 +1,150 @@
import pytest
from app.core.websocket.manager import ConnectionManager
from app.core.websocket.ue_scene import (
UeSceneStateStore,
_item_revision,
build_incremental_changes,
)
class _FakeSession:
pass
class _FakeWebSocket:
def __init__(self):
self.accepted = False
self.messages = []
async def accept(self):
self.accepted = True
async def send_json(self, message):
self.messages.append(message)
async def close(self):
return None
def _scene_item(item_id: str, title: str, *, lat: float = 0.0, lng: float = 0.0):
item = {
"id": item_id,
"entity_type": "gpu_cluster",
"geo": {"lat": lat, "lng": lng, "alt": 0.0},
"visual": {"style": "pulse_marker", "size": 1.0, "color": "#FF8C42"},
"metrics": {"name": title},
"labels": {"title": title, "subtitle": ""},
"status": {"health": "normal", "alert_level": "none"},
}
item["revision"] = _item_revision(item)
return item
def _scene_state(state_hash: int, items_by_layer: dict[str, dict[str, dict]]):
base_layers = {
"satellites": {"revision": 0, "items": {}},
"supercomputers": {"revision": 0, "items": {}},
"gpu_clusters": {"revision": 0, "items": {}},
"submarine_cables": {"revision": 0, "items": {}},
"landing_points": {"revision": 0, "items": {}},
"alerts": {"revision": 0, "items": {}},
}
for layer_name, items in items_by_layer.items():
base_layers[layer_name]["items"] = items
base_layers[layer_name]["revision"] = len(items)
return {
"generated_at": "2026-04-17T00:00:00Z",
"state_hash": state_hash,
"total_records": sum(len(items) for items in items_by_layer.values()),
"layers": base_layers,
}
def test_build_incremental_changes_detects_add_update_remove():
previous_state = _scene_state(
1,
{
"gpu_clusters": {
"gpu:a": _scene_item("gpu:a", "A"),
"gpu:b": _scene_item("gpu:b", "B"),
}
},
)
current_state = _scene_state(
2,
{
"gpu_clusters": {
"gpu:b": _scene_item("gpu:b", "B Updated"),
"gpu:c": _scene_item("gpu:c", "C"),
}
},
)
changes = build_incremental_changes(previous_state, current_state)
assert changes["gpu_clusters"]["added"][0]["id"] == "gpu:c"
assert changes["gpu_clusters"]["updated"][0]["id"] == "gpu:b"
assert changes["gpu_clusters"]["removed"] == ["gpu:a"]
@pytest.mark.asyncio
async def test_state_store_replays_incremental_history(monkeypatch):
state_list = [
_scene_state(1, {"gpu_clusters": {"gpu:a": _scene_item("gpu:a", "A")}}),
_scene_state(
2,
{"gpu_clusters": {"gpu:a": _scene_item("gpu:a", "A"), "gpu:b": _scene_item("gpu:b", "B")}},
),
_scene_state(
3,
{"gpu_clusters": {"gpu:a": _scene_item("gpu:a", "A Updated"), "gpu:b": _scene_item("gpu:b", "B")}},
),
]
state_index = {"value": 0}
async def _fake_build_visualization_scene_state(_db):
current_index = state_index["value"]
if current_index < len(state_list) - 1:
state_index["value"] += 1
return state_list[current_index]
monkeypatch.setattr(
"app.core.websocket.ue_scene.build_visualization_scene_state",
_fake_build_visualization_scene_state,
)
store = UeSceneStateStore(history_limit=5)
session = _FakeSession()
first_payloads = await store.get_broadcast_payloads(session)
second_payloads = await store.get_broadcast_payloads(session)
third_payloads = await store.get_broadcast_payloads(session)
replay_payloads = await store.get_sync_payloads(
session,
last_sequence=1,
reason="sequence_gap",
)
assert first_payloads[0]["update_type"] == "full"
assert second_payloads[0]["update_type"] == "incremental"
assert third_payloads[0]["sequence"] == 3
assert [payload["sequence"] for payload in replay_payloads] == [2, 3]
@pytest.mark.asyncio
async def test_connection_manager_broadcasts_to_subscribed_channel_only():
manager = ConnectionManager()
ws_dashboard = _FakeWebSocket()
ws_scene = _FakeWebSocket()
await manager.connect(ws_dashboard, "user-dashboard")
await manager.connect(ws_scene, "user-scene")
manager.subscribe(ws_dashboard, ["dashboard"])
manager.subscribe(ws_scene, ["ue_scene"])
await manager.broadcast({"type": "data_frame", "channel": "ue_scene"}, channel="ue_scene")
assert ws_dashboard.messages == []
assert ws_scene.messages == [{"type": "data_frame", "channel": "ue_scene"}]

View File

@@ -8,6 +8,42 @@ This project follows the repository versioning rule:
- `improvement` -> `+0.0.1`bugfix + 小功能混合)
- `bugfix` -> `+0.0.1`
## [0.33.0] — 2026-04-22
### ✨ Highlights
- `news_live_streams` 采集器默认接入 `iptv-org` 频道目录,并将采集结果稳定并入 Earth TV 直播源列表
- 数据源页支持直接编辑内置数据源 override并为内置源提供一键恢复默认配置入口
### 🔧 Improvements
- `News Live Streams` 现在作为可直接触发的内置默认数据源提供,无需先手工补 override 才能采集
- TV 播放源菜单会直接区分 `[内置]``[采集]` 来源,频道来源信息也会同步展示
- 新增 [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md),正式规划 Earth 态势新闻源配置化与后续采集器化路线
### 🐛 Fixes
- 修复 `news_live_streams` 采集完成后 `/api/v1/tv/streams` 因读取不存在的 `updated_at` 字段而导致默认频道全部消失的问题
- 修复内置数据源操作列按钮显示不全,以及编辑抽屉中多个 `Collapse` 紧贴的问题
---
## [0.32.0] — 2026-04-22
### ✨ Highlights
- Earth 设置新增“地球默认大小”持久化项,重置视角、缩放百分比重置和 BGP 巡航视图现在统一复用这一份默认 zoom
- 卫星焦点层次继续收口:巡航进入 presentation 前不再过早 dim非焦点卫星改成“降亮度/尾迹/背板”而不是去饱和度
### 🔧 Improvements
- Earth 设置面板区块和左右留白进一步收紧,整体更贴近 HUD 面板的密度
- toolbar 展开边界缓存改为按需刷新,减少 document 级 mousemove 期间的重复布局读取
- Scrollbar 和 ScrollbarOverlay 收窄 observer 范围,减少大表格和动态菜单下的额外刷新成本
- 更新 [earth-frontend-context.md](/home/ray/dev/linkong/planet/docs/technical/earth-frontend-context.md),补充默认视图大小已进入 Earth 设置持久化真源
### 🐛 Fixes
- 修复开启巡航后,尚未进入连线/presentation 时卫星已经整体变暗的问题
- 修复默认大小重置链路分散在多个入口、实际 reset/cruise/缩放提示不一致的问题
- 修复开启地形后卫星反馈层与地球背面可见性之间的一组表现问题,保留正面反馈同时恢复背面轨道遮挡
---
## [0.31.3] — 2026-04-22
### ✨ Highlights

View File

@@ -20,6 +20,7 @@
- [earth-predicted-orbit-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-predicted-orbit-plan.md)
- [earth-webgl-instancing-satellites-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-webgl-instancing-satellites-plan.md)
- [earth-real-terrain-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-real-terrain-plan.md)
- [earth-news-source-configuration-and-collector-plan.md](/home/ray/dev/linkong/planet/docs/plans/earth-news-source-configuration-and-collector-plan.md)
- [frontend-ai-playground-development-plan.md](/home/ray/dev/linkong/planet/docs/plans/frontend-ai-playground-development-plan.md)
- [ue5-mvp-fused-plan.md](/home/ray/dev/linkong/planet/docs/plans/ue5-mvp-fused-plan.md)

View File

@@ -0,0 +1,156 @@
# Earth News Source Configuration And Collector Plan
## Why
当前 Earth 的“态势新闻”由 [earth_news.py](/home/ray/dev/linkong/planet/backend/app/services/earth_news.py) 直接在请求时抓取 RSS / Google News feed再按当前地球视角中心区域聚合返回。
这条链已经可用,但存在两个明显限制:
- 新闻源写死在代码里,不能像 TV 直播源一样从后台维护
- 新闻并未进入统一采集体系,没有采集状态、失败监控、历史数据和后续 AI 复用能力
因此这块更合理的路线不是一步到位重写,而是分阶段推进:
1. 先做“新闻源配置化”
2. 再做“新闻采集器化”
## Current State
当前实现分布在:
- 新闻接口
- [news.py](/home/ray/dev/linkong/planet/backend/app/api/v1/news.py)
- 实时聚合逻辑
- [earth_news.py](/home/ray/dev/linkong/planet/backend/app/services/earth_news.py)
- 前端消费
- [news.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/news.js)
当前新闻源包含:
- `BBC World` RSS
- `DW Top Stories` RSS
- 按区域关键词拼出来的 `Google News RSS`
- `Global`
- `Americas`
- `Europe`
- `Middle East / Africa`
- `Asia Pacific`
当前不是采集器,也不落库,只做内存缓存。
## Phase 1: Source Configuration
### Goal
`NEWS_FEED_SOURCES` 从硬编码列表升级成可配置新闻源目录,但继续保留当前“实时聚合”的工作方式。
### Scope
- 为 Earth news 建立独立配置结构
- 支持后台维护 feed 源
- 支持启用/禁用、优先级、区域、源类型
- 保持现有 `/api/v1/news/earth-feed` 输出协议不变
### Proposed Shape
建议配置字段至少包括:
- `id`
- `name`
- `region`
- `feed_url`
- `homepage_url`
- `source_type`
- `priority`
- `is_enabled`
- 可选 `query_profile`
- 可选 `language`
- 可选 `notes`
### Suggested Storage
优先走系统设置或单独的 news source settings payload而不是先建复杂新表。
推荐原因:
- 改动小
- 易上线
- 和当前 TV settings 维护体验更接近
- 先解决“写死在代码里”的问题
### Non-goals
这一阶段不做:
- 新闻入库
- 新闻历史回看
- 新闻采集任务监控
- 新闻去重流水线
## Phase 2: News Collectorization
### Goal
把“态势新闻”升级为真正的采集器链路,使其进入采集系统和数据层。
### Scope
- 新增专用 news collector
- 按配置源定时采集 RSS / feed
- 做标题/链接级去重
- 建立统一新闻记录模型
- 为 Earth、控制台、AI 研判复用同一份新闻数据
### Benefits
- 有采集状态
- 有失败监控
- 有历史缓存
- 可以做时间轴 / 区域新闻基线
- 可以作为 AI 引用证据
### Required Design Work
需要提前明确:
- 新闻数据模型
- 去重策略
- 过期清理策略
- 区域映射策略
- 聚合排序策略
- 新闻与 Earth 当前视角/区域的关联方式
### Candidate Output Model
至少应包含:
- `source_id`
- `headline`
- `summary`
- `url`
- `publisher`
- `region`
- `published_at`
- `language`
- `tags`
- `raw_feed_source`
- `reference_date`
## Recommended Order
推荐执行顺序:
1. 先完成 Phase 1 配置化
2. 保持 Earth 继续实时聚合,但改为读取配置源
3. 等新闻源稳定后,再设计 Phase 2 的 collector / storage / dedupe
## Decision
当前结论:
- TV 直播源:优先采集器化
- 态势新闻:优先配置化,再采集器化
## Source Note
This plan is newly created for the Planet repo to separate the short-term "configurable source directory" work from the longer-term "collectorized news pipeline" work.

View File

@@ -258,6 +258,7 @@ Earth 设置面板当前由 [controls.js](/home/ray/dev/linkong/planet/frontend/
当前持久化的范围是:
- 旋转模式
- 地球默认大小(作为重置视角、缩放重置和巡航视图的默认 zoom 真源)
- HUD 面板显示/隐藏
- 图层控制开关:`地形 / 卫星 / 轨迹 / 海缆 / BGP`
- 地形透明度

View File

@@ -95,3 +95,93 @@
- 手工配置源
- `news_live_streams` 采集器采集源
- 当前默认兜底源为 `CCTV-4 中文国际`
- `news_live_streams` 在未配置 override 时,默认使用 `iptv-org`
- `channels.json`
- `streams.json`
- `logos.json`
并自动筛出新闻类频道目录
## 采集器配置方式
`news_live_streams` 不需要单独新页面,直接复用现有数据源配置:
- `endpoint`
- 频道目录 JSON API 地址
- `auth_type`
- `none` / `bearer` / `api_key` / `basic`
- `headers`
- 额外请求头
- `config`
- 采集器请求与解析行为
### 支持的 `config` 字段
```json
{
"timeout": 30,
"method": "GET",
"params": {
"region": "global"
},
"body_type": "json",
"body": {
"include_disabled": false
},
"response_path": "payload.channels"
}
```
- `timeout`
- 请求超时秒数
- `method`
- `GET``POST`
- `params`
- 查询参数对象
- `body_type`
- `json``form`
- `body`
- 配合 `POST` 使用的请求体
- `json_body`
- 显式 JSON 请求体,优先级高于 `body`
- `form_body`
- 显式表单请求体,优先级高于 `body`
- `response_path`
- 返回 JSON 中频道数组所在路径,支持点路径,例如:
- `payload.channels`
- `data.items`
- `result.streams`
### 认证补充
- `bearer`
- 使用 `Authorization: Bearer <token>`
- `api_key`
- 默认作为请求头发送
- 如果 `auth_config.in = "query"`,则作为 query param 发送
- `basic`
- 使用 HTTP Basic Authorization
## 兼容的响应结构
采集器会优先读取:
- 顶层数组
- 或这些常见字段下的数组:
- `sources`
- `streams`
- `channels`
- `items`
- `results`
- `data`
同时会兼容这些字段别名:
- `id` / `source_id` / `slug` / `channel_id` / `code`
- `name` / `title` / `channel` / `display_name`
- `provider` / `publisher` / `network`
- `stream_url` / `stream` / `playback_url` / `hls_url` / `m3u8_url`
- `embed_url` / `embed` / `page_url`
- `homepage_url` / `source_url` / `website`
- `language` / `lang` / `locale`
- `youtube_video_id` / `video_id`
- `youtube_channel` / `channel_handle`

View File

@@ -0,0 +1,352 @@
# UE nDisplay WebSocket 协议草案
## 目的
本协议定义后端到 UE `nDisplay` 集群运行时之间的场景同步契约。
设计目标:
- 由 UE 主节点单点接入后端
- 使用全量快照重建完整场景
- 使用增量更新降低同步成本
- 支持控制帧和显示配置
- 支持序列号、重放和重同步
## 适用范围
该协议建立在通用 WebSocket 协议之上,参考 [websocket_protocol.md](/D:/work/planet/planet/websocket_protocol.md:1)。
新增的核心内容是 `ue_scene` 频道。
## 连接模型
### 客户端
- UE 主节点连接 `ws://<backend-host>:8000/ws?token=<access_token>`
- UE 渲染节点不直接连接后端场景通道
### 订阅
UE 主节点建议订阅:
- `ue_scene`
- `alerts`
- `dashboard`
## 协议规则
- 每次场景更新都带单调递增的 `sequence`
- 增量更新必须包含 `base_sequence`
- UE 如果发现序列断裂,应先发 `sync_request`
- 服务端优先返回可重放的增量历史;无法重放时返回全量快照
- 后端始终发送地理坐标UE 本地负责转成球体世界坐标
- 控制消息与场景消息分离
## 消息类型
### `connection_established`
服务端 -> 客户端
```json
{
"type": "connection_established",
"timestamp": "2026-04-17T02:30:00.000Z",
"data": {
"connection_id": "conn_primary_node_01",
"server_version": "0.19.0",
"heartbeat_interval": 30,
"supported_channels": [
"ue_scene",
"alerts",
"dashboard",
"datasource_tasks"
]
}
}
```
### `subscribe`
客户端 -> 服务端
```json
{
"type": "subscribe",
"timestamp": "2026-04-17T02:30:01.000Z",
"data": {
"channels": ["ue_scene", "alerts"]
}
}
```
### `sync_request`
客户端 -> 服务端
```json
{
"type": "sync_request",
"timestamp": "2026-04-17T02:30:05.000Z",
"data": {
"channel": "ue_scene",
"reason": "initial_connect",
"last_sequence": null
}
}
```
`reason` 常见值:
- `initial_connect`
- `sequence_gap`
- `manual_resync`
- `profile_changed`
### `data_frame` 全量快照
服务端 -> 客户端
```json
{
"type": "data_frame",
"channel": "ue_scene",
"timestamp": "2026-04-17T02:30:06.000Z",
"data": {
"update_type": "full",
"sequence": 12,
"cluster_time": "2026-04-17T02:30:06.000Z",
"display_profile": {
"profile_id": "polarized-wall-a",
"stereo_mode": "polarized",
"screen_width_m": 3.0,
"screen_height_m": 2.0,
"target_refresh_hz": 120
},
"camera_state": {
"mode": "auto_cruise",
"path_id": "global_overview",
"fov": 42.0
},
"payload": {
"meta": {
"generated_at": "2026-04-17T02:30:06.000Z",
"total_records": 20800,
"state_hash": 123456789
},
"layers": {
"satellites": {
"revision": 111,
"items": []
},
"supercomputers": {
"revision": 222,
"items": []
},
"gpu_clusters": {
"revision": 333,
"items": []
},
"submarine_cables": {
"revision": 444,
"items": []
},
"landing_points": {
"revision": 555,
"items": []
},
"alerts": {
"revision": 0,
"items": []
}
}
}
}
}
```
### `data_frame` 增量更新
服务端 -> 客户端
```json
{
"type": "data_frame",
"channel": "ue_scene",
"timestamp": "2026-04-17T02:31:00.000Z",
"data": {
"update_type": "incremental",
"sequence": 13,
"base_sequence": 12,
"cluster_time": "2026-04-17T02:31:00.000Z",
"meta": {
"generated_at": "2026-04-17T02:31:00.000Z",
"total_records": 20805,
"state_hash": 123456790
},
"changes": {
"gpu_clusters": {
"added": [],
"updated": [],
"removed": []
},
"submarine_cables": {
"added": [],
"updated": [],
"removed": []
}
}
}
}
```
### `control_frame`
客户端 -> 服务端
```json
{
"type": "control_frame",
"timestamp": "2026-04-17T02:32:00.000Z",
"data": {
"target": "ue_scene",
"command": "set_camera_mode",
"arguments": {
"mode": "manual",
"camera_id": "operator_cam_1"
}
}
}
```
常见命令:
- `set_camera_mode`
- `set_camera_path`
- `focus_entity`
- `set_layer_visibility`
- `set_alert_filter`
- `set_display_profile`
- `pause_auto_cruise`
- `resume_auto_cruise`
### `control_acknowledged`
服务端 -> 客户端
```json
{
"type": "control_acknowledged",
"timestamp": "2026-04-17T02:32:00.050Z",
"data": {
"target": "ue_scene",
"command": "set_camera_mode",
"accepted": true
}
}
```
### `heartbeat`
双向
```json
{
"type": "heartbeat",
"timestamp": "2026-04-17T02:32:30.000Z",
"data": {
"action": "ping"
}
}
```
服务端返回:
```json
{
"type": "heartbeat",
"timestamp": "2026-04-17T02:32:30.020Z",
"data": {
"action": "pong"
}
}
```
## 场景实体结构
每个场景实体建议包含:
- `id`
- `entity_type`
- `revision`
- `geo`
- `visual`
- `metrics`
- `labels`
- `status`
示例:
```json
{
"id": "gpu_clusters:123",
"entity_type": "gpu_cluster",
"revision": 987654321,
"geo": {
"lat": 35.9327,
"lng": -84.3107,
"alt": 0.0
},
"visual": {
"style": "pulse_marker",
"size": 1.0,
"color": "#FF8C42"
},
"metrics": {
"name": "Frontier"
},
"labels": {
"title": "Frontier",
"subtitle": "Oak Ridge, US"
},
"status": {
"health": "normal",
"alert_level": "none"
}
}
```
## 同步策略
UE 主节点建议这样处理:
1. 保存本地最后一次成功应用的 `sequence`
2. 收到增量帧时检查 `base_sequence`
3.`base_sequence` 不等于本地序列,则立即发 `sync_request`
4. 如果服务端能重放缺失增量,则按顺序依次应用
5. 如果服务端无法重放,则接收并应用新的全量快照
6. 场景更新应在主节点原子提交,再交给集群同步显示
## 服务端重同步规则
服务端建议行为:
- `initial_connect`:直接返回全量快照
- `manual_resync`:直接返回全量快照
- `profile_changed`:直接返回全量快照
- `sequence_gap`:优先尝试从历史缓存中回放增量
- 如果历史缓存不完整:回退到全量快照
- 如果客户端已是最新序列:返回空变化增量或保持静默
## 集群职责约束
- 只有 UE 主节点接收后端场景同步消息
- 渲染节点不直接重放后端消息
- 服务端表达的是场景意图,而不是每个节点的底层渲染状态
## 推荐的后端扩展
-`ue_scene` 增加更多真实可视化层
- 将告警/BGP 事件并入统一场景层
- 为控制命令增加真正的状态持久化与回执
- 为主节点断线重连补状态恢复策略

View File

@@ -16,12 +16,14 @@
## Current Version
- `main` 当前主线历史推导到:`0.16.5`
- `dev` 当前开发分支历史推导到:`0.31.3`
- `dev` 当前开发分支历史推导到:`0.33.0`
## Timeline
| Version | Type | Branch | Commit | Summary |
| --- | --- | --- | --- | --- |
| `0.33.0` | feature | `dev` | `pending` | `news_live_streams` 默认接入 iptv-org 频道目录,内置数据源支持直接编辑 override并修复 TV 合并采集源后默认频道消失的问题 |
| `0.32.0` | feature | `dev` | `pending` | Earth 设置新增默认地球大小真源并继续收口卫星焦点层次、toolbar/scrollbar 性能与 HUD 设置面板细节 |
| `0.31.3` | bugfix | `dev` | `pending` | 收口 Earth 图层注册表与启动任务框架,修复旋转/巡航切换、卫星地形遮挡与日夜关闭照明回归 |
| `0.31.2` | bugfix | `dev` | `pending` | 将 Earth 巡航模式拆成通用 sequencer、通用连线和 BGP 巡航适配层,并修复空白点击推进与连线动画回归 |
| `0.31.1` | bugfix | `dev` | `pending` | Earth 图层开关统一 loading 状态机,卫星首次加载可见化,并将文档按 technical / plans / deprecated 重构归档 |

View File

@@ -1,6 +1,6 @@
{
"name": "planet-frontend",
"version": "0.31.3",
"version": "0.33.0",
"private": true,
"packageManager": "bun@1",
"dependencies": {

View File

@@ -566,7 +566,8 @@
.earth-settings-content {
overflow-y: auto;
padding-right: 4px;
padding-inline: 8px;
padding-right: 10px;
scrollbar-width: thin;
scrollbar-color: rgba(160, 186, 216, 0.34) transparent;
}
@@ -585,11 +586,11 @@
}
.earth-settings-section {
padding: 12px 0 20px;
padding: 6px 0 10px;
}
.earth-settings-section-title {
margin-bottom: 12px;
margin-bottom: 10px;
color: var(--hud-text-soft);
font-size: calc(0.62rem * var(--hud-scale));
letter-spacing: 0.16em;
@@ -599,7 +600,7 @@
.earth-settings-list {
display: flex;
flex-direction: column;
gap: 10px;
gap: 8px;
}
.earth-settings-item {
@@ -607,7 +608,7 @@
align-items: center;
justify-content: space-between;
gap: 12px;
padding: 11px 14px;
padding: 10px 13px;
border-radius: 14px;
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.04), transparent),
@@ -629,7 +630,7 @@
.earth-settings-item--stacked {
align-items: stretch;
flex-direction: column;
gap: 14px;
gap: 10px;
cursor: default;
}
@@ -644,7 +645,7 @@
.earth-settings-slider-row {
display: flex;
align-items: center;
gap: 14px;
gap: 12px;
}
.earth-settings-segmented {
@@ -783,7 +784,7 @@
.earth-settings-copy {
display: flex;
flex-direction: column;
gap: 4px;
gap: 3px;
}
.earth-settings-item-title {

View File

@@ -512,6 +512,30 @@
</label>
</div>
</section>
<section class="earth-settings-section">
<div class="earth-settings-section-title">视图</div>
<div class="earth-settings-list">
<div class="earth-settings-item earth-settings-item--stacked">
<div class="earth-settings-copy">
<span class="earth-settings-item-title">地球默认大小</span>
<span class="earth-settings-item-subtitle">用于重置视角、缩放重置和巡航视图的默认缩放比例</span>
</div>
<div class="earth-settings-slider-row">
<input
id="default-earth-size-slider"
class="earth-settings-slider"
type="range"
min="0.5"
max="5"
step="0.01"
value="1"
aria-label="调整地球默认大小"
>
<span id="default-earth-size-value" class="earth-settings-slider-value">100%</span>
</div>
</div>
</div>
</section>
<section class="earth-settings-section">
<div class="earth-settings-section-title">地形</div>
<div class="earth-settings-list">

View File

@@ -169,13 +169,11 @@ export function createBGPCruiseAdapter({
cardPlacement = getCardScreenCoords(marker);
setMarkerLocked(marker);
showMarkerOverlay(marker);
applySatelliteHighlights(marker);
await focusView({
lat: marker.userData?.latitude ?? 0,
lon: marker.userData?.longitude ?? 0,
rotLon: (marker.userData?.longitude ?? 0) - 270,
zoom: 1.0,
duration: interrupt
? Math.round(CRUISE_CONFIG.focusDurationMs * 0.78)
: CRUISE_CONFIG.focusDurationMs,
@@ -203,6 +201,8 @@ export function createBGPCruiseAdapter({
return false;
}
applySatelliteHighlights(marker);
const connectorDelayCompleted = await context.wait(CRUISE_CONNECTOR_DRAW_MS);
if (!connectorDelayCompleted || !context.isCurrent()) {
cardPlacement = null;

View File

@@ -3,6 +3,7 @@
// Scene configuration
export const CONFIG = {
defaultCameraZ: 300,
defaultViewZoom: 1.0,
minZoom: 0.5,
maxZoom: 5.0,
earthRadius: 100,

View File

@@ -44,6 +44,8 @@ export let showTerrain = false;
export let layoutExpanded = false;
export let rotationMode = ROTATION_MODE.ROTATE;
let dayNightEnabled = true;
let defaultEarthZoom = CONFIG.defaultViewZoom;
let activeCamera = null;
let earthObj = null;
let listeners = [];
@@ -73,6 +75,7 @@ const SETTINGS_SHEET_MIN_SCALE = 0.06;
const SETTINGS_SHEET_MAX_SCALE_X = 0.22;
const SETTINGS_SHEET_MAX_SCALE_Y = 0.18;
const EARTH_SETTINGS_STORAGE_KEY = "planet.earth.settings.v1";
const DEFAULT_EARTH_ZOOM_STEP = 0.01;
let settingsModalTimer = null;
let settingsSheetAnimation = null;
let terrainToggleToken = 0;
@@ -120,6 +123,26 @@ function shouldIncludeLayerInStartupLoad(definition) {
return Boolean(definition?.getVisible?.());
}
function clampEarthZoomLevel(nextZoom) {
const parsedZoom = Number.parseFloat(nextZoom);
if (!Number.isFinite(parsedZoom)) {
return CONFIG.defaultViewZoom;
}
return Math.min(CONFIG.maxZoom, Math.max(CONFIG.minZoom, parsedZoom));
}
function formatZoomPercent(zoom) {
return `${Math.round(zoom * 100)}%`;
}
function getZoomResetTooltipText(zoom) {
return `重置缩放到${formatZoomPercent(zoom)}`;
}
function getZoomResetStatusMessage(zoom) {
return `缩放已重置到${formatZoomPercent(zoom)}`;
}
function canUseLocalStorage() {
try {
return typeof window !== "undefined" && !!window.localStorage;
@@ -145,6 +168,7 @@ function getCurrentSettingsSnapshot() {
),
terrainOpacity: getTerrainOpacity(),
dayNightEnabled,
defaultEarthZoom,
};
}
@@ -160,6 +184,7 @@ function cloneEarthSettings(settings) {
rotationMode: settings.rotationMode,
terrainOpacity: settings.terrainOpacity,
dayNightEnabled: settings.dayNightEnabled,
defaultEarthZoom: settings.defaultEarthZoom,
panelVisibility: { ...(settings.panelVisibility || {}) },
layerVisibility: { ...(settings.layerVisibility || {}) },
};
@@ -197,6 +222,9 @@ function normalizeEarthSettings(rawSettings, defaults) {
const nextDayNightEnabled = typeof rawSettings?.dayNightEnabled === "boolean"
? rawSettings.dayNightEnabled
: defaults.dayNightEnabled;
const nextDefaultEarthZoom = clampEarthZoomLevel(
rawSettings?.defaultEarthZoom ?? defaults.defaultEarthZoom,
);
return {
rotationMode: nextRotationMode,
@@ -206,6 +234,7 @@ function normalizeEarthSettings(rawSettings, defaults) {
? nextTerrainOpacity
: defaults.terrainOpacity,
dayNightEnabled: nextDayNightEnabled,
defaultEarthZoom: nextDefaultEarthZoom,
};
}
@@ -251,6 +280,46 @@ function persistEarthSettings() {
}
}
function syncDefaultEarthZoomUi(nextZoom) {
const slider = document.getElementById("default-earth-size-slider");
const value = document.getElementById("default-earth-size-value");
const zoomValue = document.getElementById("zoom-value");
const tooltipText = getZoomResetTooltipText(nextZoom);
if (slider instanceof HTMLInputElement) {
slider.min = CONFIG.minZoom.toString();
slider.max = CONFIG.maxZoom.toString();
slider.step = DEFAULT_EARTH_ZOOM_STEP.toString();
slider.value = nextZoom.toFixed(2);
}
if (value) {
value.textContent = formatZoomPercent(nextZoom);
}
if (zoomValue instanceof HTMLElement) {
zoomValue.title = tooltipText;
const tooltip = zoomValue.querySelector(".tooltip");
if (tooltip) {
tooltip.textContent = tooltipText;
}
}
}
function setDefaultEarthZoom(nextZoom, { persist = true, applyToCurrentView = true } = {}) {
defaultEarthZoom = clampEarthZoomLevel(nextZoom);
syncDefaultEarthZoomUi(defaultEarthZoom);
if (applyToCurrentView && activeCamera) {
zoomLevel = defaultEarthZoom;
applyZoom(activeCamera);
}
if (persist) {
persistEarthSettings();
}
return defaultEarthZoom;
}
async function applyEarthSettings(settings) {
if (!settings) return;
@@ -277,6 +346,11 @@ async function applyEarthSettings(settings) {
applyDayNightEnabled(settings.dayNightEnabled, { persist: false });
}
setDefaultEarthZoom(settings.defaultEarthZoom, {
persist: false,
applyToCurrentView: true,
});
await applyLayerVisibilitySettings(settings.layerVisibility, {
persist: false,
silent: true,
@@ -660,7 +734,7 @@ export function applyImmediateView(targetEarthObj, camera, options = {}) {
const {
lat = EARTH_CONFIG.chinaLat,
rotLon = EARTH_CONFIG.chinaRotLon,
zoom = 1.0,
zoom = getDefaultEarthZoomLevel(),
} = options;
const nextRotation = getViewRotation(lat, rotLon);
@@ -956,6 +1030,7 @@ function setupSettingsControls() {
const terrainOpacitySlider = document.getElementById("terrain-opacity-slider");
const terrainOpacityValue = document.getElementById("terrain-opacity-value");
const defaultEarthSizeSlider = document.getElementById("default-earth-size-slider");
const rotationModeButtons = document.querySelectorAll("[data-rotation-mode]");
const syncTerrainOpacityUi = (nextOpacity) => {
const safeOpacity = Math.round(nextOpacity * 100);
@@ -968,6 +1043,7 @@ function setupSettingsControls() {
};
syncTerrainOpacityUi(getTerrainOpacity());
syncDefaultEarthZoomUi(defaultEarthZoom);
if (terrainOpacitySlider instanceof HTMLInputElement) {
bindListener(terrainOpacitySlider, "input", (event) => {
@@ -982,6 +1058,18 @@ function setupSettingsControls() {
});
}
if (defaultEarthSizeSlider instanceof HTMLInputElement) {
bindListener(defaultEarthSizeSlider, "input", (event) => {
const target = event.currentTarget;
if (!(target instanceof HTMLInputElement)) return;
const nextZoom = Number.parseFloat(target.value);
setDefaultEarthZoom(
Number.isFinite(nextZoom) ? nextZoom : defaultEarthZoom,
{ persist: true, applyToCurrentView: true },
);
});
}
rotationModeButtons.forEach((button) => {
bindListener(button, "click", (event) => {
const target = event.currentTarget;
@@ -1303,6 +1391,7 @@ function resetCleanup() {
export function setupControls(camera, renderer, scene, earth) {
resetCleanup();
activeCamera = camera;
earthObj = earth;
setupZoomControls(camera);
setupWheelZoom(camera, renderer);
@@ -1407,7 +1496,7 @@ function setupZoomControls(camera) {
bindListener(zoomValue, "click", () => {
const startZoomVal = zoomLevel;
const targetZoom = 1.0;
const targetZoom = getDefaultEarthZoomLevel();
const startDistance = CONFIG.defaultCameraZ / startZoomVal;
const targetDistance = CONFIG.defaultCameraZ / targetZoom;
@@ -1424,8 +1513,8 @@ function setupZoomControls(camera) {
updateZoomDisplay(zoomLevel, distance.toFixed(0));
},
() => {
zoomLevel = 1.0;
showStatusMessage("缩放已重置到100%", "info");
zoomLevel = targetZoom;
showStatusMessage(getZoomResetStatusMessage(targetZoom), "info");
},
);
});
@@ -1482,6 +1571,7 @@ function animateValue(start, end, duration, onUpdate, onComplete) {
export function resetView(camera) {
if (!earthObj) return;
const defaultZoom = getDefaultEarthZoomLevel();
if (navigator.geolocation) {
navigator.geolocation.getCurrentPosition(
@@ -1490,7 +1580,7 @@ export function resetView(camera) {
lat: pos.coords.latitude,
lon: pos.coords.longitude,
rotLon: pos.coords.longitude - 270,
zoom: 1.0,
zoom: defaultZoom,
duration: 800,
suppressStatus: false,
}),
@@ -1499,7 +1589,7 @@ export function resetView(camera) {
lat: EARTH_CONFIG.chinaLat,
lon: EARTH_CONFIG.chinaLon,
rotLon: EARTH_CONFIG.chinaRotLon,
zoom: 1.0,
zoom: defaultZoom,
duration: 800,
suppressStatus: false,
}),
@@ -1510,7 +1600,7 @@ export function resetView(camera) {
lat: EARTH_CONFIG.chinaLat,
lon: EARTH_CONFIG.chinaLon,
rotLon: EARTH_CONFIG.chinaRotLon,
zoom: 1.0,
zoom: defaultZoom,
duration: 800,
suppressStatus: false,
});
@@ -1877,6 +1967,8 @@ function setupToolbarHubCluster() {
}
let collapseTimer = null;
let expandedToolbarBounds = null;
let refreshBoundsFrameId = 0;
const layoutToolbarOrbs = () => {
const orbs = Array.from(cluster.querySelectorAll(".earth-toolbar-orb"));
@@ -1942,11 +2034,24 @@ function setupToolbarHubCluster() {
orb.style.setProperty("--orb-x", `${x.toFixed(2)}px`);
orb.style.setProperty("--orb-y", `${y.toFixed(2)}px`);
});
if (cluster.classList.contains("is-expanded")) {
scheduleExpandedToolbarBoundsRefresh();
}
};
const setExpanded = (expanded) => {
cluster.classList.toggle("is-expanded", expanded);
cluster.classList.toggle("is-collapsed", !expanded);
if (!expanded) {
expandedToolbarBounds = null;
if (refreshBoundsFrameId) {
cancelAnimationFrame(refreshBoundsFrameId);
refreshBoundsFrameId = 0;
}
return;
}
scheduleExpandedToolbarBoundsRefresh();
};
const scheduleCollapse = () => {
@@ -1966,6 +2071,10 @@ function setupToolbarHubCluster() {
cleanupFns.push(() => {
if (collapseTimer) clearTimeout(collapseTimer);
if (refreshBoundsFrameId) {
cancelAnimationFrame(refreshBoundsFrameId);
refreshBoundsFrameId = 0;
}
});
// Start collapsed — hub acts as the hover target to reveal the arc
@@ -2034,9 +2143,23 @@ function setupToolbarHubCluster() {
};
};
const scheduleExpandedToolbarBoundsRefresh = () => {
if (refreshBoundsFrameId) return;
refreshBoundsFrameId = window.requestAnimationFrame(() => {
expandedToolbarBounds = collectExpandedToolbarBounds();
refreshBoundsFrameId = 0;
});
};
bindListener(document, "mousemove", (event) => {
if (!cluster.classList.contains("is-expanded")) return;
const activeBounds = collectExpandedToolbarBounds();
if (
event.target instanceof Element &&
event.target.closest("#toolbar-cluster, .earth-stack-toolbar")
) {
scheduleExpandedToolbarBoundsRefresh();
}
const activeBounds = expandedToolbarBounds;
if (!activeBounds) {
scheduleCollapse();
return;
@@ -2052,10 +2175,23 @@ function setupToolbarHubCluster() {
scheduleCollapse();
}
});
bindListener(cluster, "mouseenter", () => {
if (cluster.classList.contains("is-expanded")) {
scheduleExpandedToolbarBoundsRefresh();
}
});
bindListener(cluster, "focusin", () => {
if (cluster.classList.contains("is-expanded")) {
scheduleExpandedToolbarBoundsRefresh();
}
});
}
export function teardownControls() {
resetCleanup();
activeCamera = null;
}
export function getAutoRotate() {
@@ -2143,7 +2279,7 @@ export function focusEarthView(camera, options = {}) {
lat = EARTH_CONFIG.chinaLat,
lon = EARTH_CONFIG.chinaLon,
rotLon = lon - 270,
zoom = 1.0,
zoom = getDefaultEarthZoomLevel(),
duration = 800,
suppressStatus = true,
} = options;
@@ -2181,6 +2317,10 @@ export function getZoomLevel() {
return zoomLevel;
}
export function getDefaultEarthZoomLevel() {
return defaultEarthZoom;
}
export function getShowTerrain() {
return showTerrain;
}

View File

@@ -31,6 +31,10 @@ let satelliteSatrecCache = new Map();
const TRAIL_LENGTH = SATELLITE_CONFIG.trailLength;
const DOT_TEXTURE_SIZE = 32;
const POSITION_UPDATE_INTERVAL_MS = 250;
const DIMMED_SATELLITE_BRIGHTNESS = 0.42;
const DIMMED_SATELLITE_TRAIL_BRIGHTNESS = 0.24;
const DIMMED_SATELLITE_POINT_OPACITY = 0.62;
const DIMMED_SATELLITE_BACKDROP_OPACITY = 0.1;
const scratchWorldSatellitePosition = new THREE.Vector3();
const scratchToCamera = new THREE.Vector3();
@@ -746,16 +750,16 @@ export function updateSatellitePositions(deltaTime = 0, force = false) {
const rule = getSatelliteLegendRule(props);
const { r, g, b } = getSatelliteRuleColor(rule);
if (highlightedSatelliteIndices !== null && !highlightedSatelliteIndices.has(i)) {
const lum = r * 0.299 + g * 0.587 + b * 0.114;
colors[i * 3] = lum * 0.75 + r * 0.25;
colors[i * 3 + 1] = lum * 0.75 + g * 0.25;
colors[i * 3 + 2] = lum * 0.75 + b * 0.25;
} else {
colors[i * 3] = r;
colors[i * 3 + 1] = g;
colors[i * 3 + 2] = b;
}
const isNonFocusDimmed =
highlightedSatelliteIndices !== null && !highlightedSatelliteIndices.has(i);
const pointBrightness = isNonFocusDimmed ? DIMMED_SATELLITE_BRIGHTNESS : 1;
const trailBrightness = isNonFocusDimmed
? DIMMED_SATELLITE_TRAIL_BRIGHTNESS
: 1;
colors[i * 3] = r * pointBrightness;
colors[i * 3 + 1] = g * pointBrightness;
colors[i * 3 + 2] = b * pointBrightness;
const satPosition = satellitePositions[i];
for (let j = 0; j < TRAIL_LENGTH; j++) {
@@ -771,9 +775,9 @@ export function updateSatellitePositions(deltaTime = 0, force = false) {
trailPositions[trailIdx + 1] = trailPoint.y;
trailPositions[trailIdx + 2] = trailPoint.z;
const alpha = (j + 1) / satPosition.trailCount;
trailColors[trailIdx] = r * alpha;
trailColors[trailIdx + 1] = g * alpha;
trailColors[trailIdx + 2] = b * alpha;
trailColors[trailIdx] = r * alpha * trailBrightness;
trailColors[trailIdx + 1] = g * alpha * trailBrightness;
trailColors[trailIdx + 2] = b * alpha * trailBrightness;
continue;
}
}
@@ -1097,10 +1101,10 @@ export function setSatelliteRingState(index, state, position) {
function applyDimMaterialState(isDimmed) {
if (satellitePoints) {
satellitePoints.material.opacity = isDimmed ? 0.32 : 0.9;
satellitePoints.material.opacity = isDimmed ? DIMMED_SATELLITE_POINT_OPACITY : 0.9;
}
if (satelliteBackdropPoints) {
satelliteBackdropPoints.material.opacity = isDimmed ? 0.12 : 0.42;
satelliteBackdropPoints.material.opacity = isDimmed ? DIMMED_SATELLITE_BACKDROP_OPACITY : 0.42;
}
}

View File

@@ -877,11 +877,12 @@ function renderSourceOptions() {
const fragment = document.createDocumentFragment();
sources.forEach((source) => {
const sourceOriginLabel = source.collector_source ? "[采集]" : "[内置]";
const defaultMark = source.id === tvPayload?.default_source_id ? " · 默认" : "";
const failMark = failedSourceIds.has(source.id) ? " ⚠" : "";
const option = document.createElement("option");
option.value = source.id;
option.textContent = `${source.name}${defaultMark}${failMark}`;
option.textContent = `${sourceOriginLabel} ${source.name}${defaultMark}${failMark}`;
fragment.appendChild(option);
});
@@ -913,8 +914,12 @@ function renderSource(source) {
const latestLabel = latestUpdatedAt
? `最近同步 ${new Date(latestUpdatedAt).toLocaleString("zh-CN", { hour12: false })}`
: "尚未同步";
const collectorLabel = source?.collector_source ? ` · 采集器 ${source.collector_source}` : "";
catalog.textContent = ` ${sourceCount} 个频道 · ${latestLabel}${collectorLabel}`;
const sourceOriginLabel = source?.collector_source
? `采集源 ${source.collector_source}`
: source
? "内置源"
: "";
catalog.textContent = `${sourceCount} 个频道 · ${latestLabel}${sourceOriginLabel ? ` · ${sourceOriginLabel}` : ""}`;
}
if (notes) {
notes.textContent = source?.notes || "支持后台配置默认源与采集器补充源。";

View File

@@ -155,7 +155,15 @@ export function updateZoomDisplay(zoomLevel, distance) {
const slider = getElement("zoom-slider");
const cameraDistanceEl = getElement("camera-distance");
if (zoomValueEl) zoomValueEl.textContent = percent + "%";
if (zoomValueEl) {
const tooltip = zoomValueEl.querySelector(".tooltip");
const label = `${percent}%`;
if (zoomValueEl.firstChild?.nodeType === Node.TEXT_NODE) {
zoomValueEl.firstChild.nodeValue = label;
} else {
zoomValueEl.insertBefore(document.createTextNode(label), tooltip || null);
}
}
if (zoomLevelEl) zoomLevelEl.textContent = "缩放: " + percent + "%";
if (slider) slider.value = zoomLevel;
if (cameraDistanceEl) cameraDistanceEl.textContent = distance + " km";

View File

@@ -132,13 +132,10 @@ function Scrollbar({
resizeObserver.observe(viewport)
resizeObserver.observe(trackX)
resizeObserver.observe(trackY)
Array.from(viewport.children).forEach((child) => resizeObserver.observe(child))
mutationObserver.observe(viewport, {
childList: true,
subtree: true,
attributes: true,
characterData: true,
})
viewport.addEventListener('scroll', scheduleUpdate, { passive: true })

View File

@@ -156,7 +156,6 @@ function ScrollbarOverlay({
scheduleUpdate()
})
resizeObserver.observe(target)
Array.from(target.children).forEach((child) => resizeObserver?.observe(child))
scheduleUpdate()
}
@@ -169,7 +168,6 @@ function ScrollbarOverlay({
mutationObserver.observe(container, {
childList: true,
subtree: true,
attributes: true,
})
window.addEventListener('resize', scheduleUpdate)

View File

@@ -1562,6 +1562,14 @@ body {
border-radius: 12px;
}
.data-source-drawer-collapse {
margin-bottom: 12px;
}
.data-source-drawer-collapse:last-of-type {
margin-bottom: 0;
}
.stat-card {
background: white;
padding: 24px;

View File

@@ -19,6 +19,7 @@ import { useWebSocket } from '../../hooks/useWebSocket'
interface BuiltInDataSource {
id: number
source: string
name: string
module: string
priority: string
@@ -180,6 +181,19 @@ interface CustomDataSource {
updated_at: string | null
}
interface EditableDataSourceConfig {
id: number
name: string
description: string | null
source_type: string
endpoint: string
auth_type: string
auth_config: Record<string, any>
headers: Record<string, string>
config: Record<string, any>
is_active?: boolean
}
interface ViewDataSource {
id: number
name: string
@@ -205,6 +219,7 @@ function DataSources() {
const [drawerVisible, setDrawerVisible] = useState(false)
const [viewDrawerVisible, setViewDrawerVisible] = useState(false)
const [editingConfig, setEditingConfig] = useState<CustomDataSource | null>(null)
const [builtinEditingSource, setBuiltinEditingSource] = useState<BuiltInDataSource | null>(null)
const [viewingSource, setViewingSource] = useState<ViewDataSource | null>(null)
const [recordCount, setRecordCount] = useState<number>(0)
const [testing, setTesting] = useState(false)
@@ -219,6 +234,81 @@ function DataSources() {
const [customActionsCollapsed, customContainerRef] = useCollapsedActions()
const [form] = Form.useForm()
const headersMapToList = useCallback((headers?: Record<string, string> | null) => {
return Object.entries(headers || {})
.filter(([key, value]) => key && value !== undefined && value !== null && String(value).trim() !== '')
.map(([key, value]) => ({ key, value }))
}, [])
const headersListToMap = useCallback((headers?: Array<{ key?: string; value?: string }> | Record<string, string>) => {
if (!headers) return {}
if (!Array.isArray(headers)) return headers
return headers.reduce<Record<string, string>>((acc, item) => {
const key = item?.key?.trim()
const value = item?.value?.trim()
if (!key || value === undefined) return acc
acc[key] = value
return acc
}, {})
}, [])
const applyConfigToForm = useCallback((config?: Partial<EditableDataSourceConfig> | null) => {
form.setFieldsValue({
name: config?.name || '',
description: config?.description || '',
source_type: config?.source_type || 'http',
endpoint: config?.endpoint || '',
auth_type: config?.auth_type || 'none',
auth_config: config?.auth_config || {},
headers: headersMapToList(config?.headers || {}),
config: config?.config || { timeout: 30, retry: 3 },
})
}, [form, headersMapToList])
const loadConfigDetail = useCallback(async (configId: number) => {
const res = await axios.get<EditableDataSourceConfig>(`/api/v1/datasources/configs/${configId}`)
return res.data
}, [])
const createDefaultConfigDraft = useCallback((overrides?: Partial<EditableDataSourceConfig>) => ({
source_type: 'http',
auth_type: 'none',
headers: {},
config: { timeout: 30, retry: 3 },
...overrides,
}), [])
const getBuiltinOverrideDescription = useCallback(
(source?: Pick<BuiltInDataSource, 'name'> | null) =>
source ? `Built-in datasource override for ${source.name}` : undefined,
[],
)
const createFormPayload = useCallback((values: any) => ({
...values,
name: builtinEditingSource ? builtinEditingSource.source : values.name,
description:
values.description ||
getBuiltinOverrideDescription(builtinEditingSource),
source_type: builtinEditingSource ? 'http' : values.source_type,
headers: headersListToMap(values.headers),
}), [builtinEditingSource, getBuiltinOverrideDescription, headersListToMap])
const closeDrawerAfterLoadError = useCallback((
errorMessage: string,
options?: { clearBuiltin?: boolean; clearEditingConfig?: boolean },
) => {
messageApi.error(errorMessage)
setDrawerVisible(false)
if (options?.clearBuiltin) {
setBuiltinEditingSource(null)
}
if (options?.clearEditingConfig) {
setEditingConfig(null)
}
}, [messageApi])
const fetchData = useCallback(async () => {
setLoading(true)
try {
@@ -711,9 +801,11 @@ function DataSources() {
const handleViewSource = async (source: BuiltInDataSource) => {
try {
const [res, statsRes] = await Promise.all([
const existingOverride = customSources.find((item) => item.name === source.source)
const [res, statsRes, overrideDetail] = await Promise.all([
axios.get(`/api/v1/datasources/${source.id}`),
axios.get(`/api/v1/datasources/${source.id}/stats`)
axios.get(`/api/v1/datasources/${source.id}/stats`),
existingOverride ? loadConfigDetail(existingOverride.id) : Promise.resolve(null),
])
const data = res.data
setViewingSource({
@@ -721,10 +813,10 @@ function DataSources() {
name: data.name,
description: null,
source_type: data.collector_class,
endpoint: data.endpoint || '',
auth_type: 'none',
headers: {},
config: {},
endpoint: overrideDetail?.endpoint || data.endpoint || '',
auth_type: overrideDetail?.auth_type || 'none',
headers: overrideDetail?.headers || {},
config: overrideDetail?.config || {},
collector_class: data.collector_class,
module: data.module,
priority: data.priority,
@@ -753,7 +845,8 @@ function DataSources() {
const values = await form.validateFields()
setTesting(true)
setTestResult(null)
const res = await axios.post('/api/v1/datasources/configs/test', values)
const payload = createFormPayload(values)
const res = await axios.post('/api/v1/datasources/configs/test', payload)
setTestResult(res.data)
if (res.data.success) {
messageApi.success('连接测试成功')
@@ -771,16 +864,18 @@ function DataSources() {
const handleSave = async () => {
try {
const values = await form.validateFields()
const payload = createFormPayload(values)
if (editingConfig) {
await axios.put(`/api/v1/datasources/configs/${editingConfig.id}`, values)
await axios.put(`/api/v1/datasources/configs/${editingConfig.id}`, payload)
messageApi.success('配置已更新')
} else {
await axios.post('/api/v1/datasources/configs', values)
await axios.post('/api/v1/datasources/configs', payload)
messageApi.success('配置已创建')
}
setDrawerVisible(false)
form.resetFields()
setEditingConfig(null)
setBuiltinEditingSource(null)
setTestResult(null)
fetchData()
} catch (error: unknown) {
@@ -800,6 +895,23 @@ function DataSources() {
}
}
const handleResetBuiltinOverride = async () => {
if (!builtinEditingSource || !editingConfig) return
try {
await axios.delete(`/api/v1/datasources/configs/${editingConfig.id}`)
messageApi.success(`已恢复 ${builtinEditingSource.name} 的默认配置`)
setDrawerVisible(false)
form.resetFields()
setEditingConfig(null)
setBuiltinEditingSource(null)
setTestResult(null)
fetchData()
} catch (error: unknown) {
const err = error as { response?: { data?: { detail?: string } } }
messageApi.error(err.response?.data?.detail || '恢复默认失败')
}
}
const handleToggleCustom = async (id: number, current: boolean) => {
try {
await axios.put(`/api/v1/datasources/configs/${id}`, { is_active: !current })
@@ -811,24 +923,53 @@ function DataSources() {
}
}
const openDrawer = (config?: CustomDataSource) => {
const openDrawer = async (config?: CustomDataSource) => {
setBuiltinEditingSource(null)
setEditingConfig(config || null)
if (config) {
form.setFieldsValue({
...config,
auth_config: {},
})
} else {
form.resetFields()
form.setFieldsValue({
source_type: 'http',
auth_type: 'none',
config: { timeout: 30, retry: 3 },
headers: {},
})
}
setDrawerVisible(true)
setTestResult(null)
setDrawerVisible(true)
if (config) {
try {
const detail = await loadConfigDetail(config.id)
applyConfigToForm(detail)
} catch {
closeDrawerAfterLoadError('获取配置详情失败', { clearEditingConfig: true })
}
return
}
form.resetFields()
applyConfigToForm(createDefaultConfigDraft())
}
const openBuiltinConfigDrawer = async (source: BuiltInDataSource) => {
setBuiltinEditingSource(source)
setTestResult(null)
setDrawerVisible(true)
const existingOverride = customSources.find((item) => item.name === source.source)
setEditingConfig(existingOverride || null)
if (existingOverride) {
try {
const detail = await loadConfigDetail(existingOverride.id)
applyConfigToForm(detail)
} catch {
closeDrawerAfterLoadError('获取内置数据源配置失败', {
clearBuiltin: true,
clearEditingConfig: true,
})
}
return
}
form.resetFields()
applyConfigToForm(createDefaultConfigDraft({
name: source.source,
description: getBuiltinOverrideDescription(source),
endpoint: source.endpoint || '',
}))
}
const handleCopyLink = async (value: string, successText: string) => {
@@ -945,12 +1086,18 @@ function DataSources() {
title: '操作',
key: 'action',
fixed: 'right' as const,
width: builtinActionsCollapsed ? 40 : 164,
width: builtinActionsCollapsed ? 40 : 228,
onCell: () => actionCellProps,
render: (_: unknown, record: BuiltInDataSource) => (
<TableActions
collapsed={builtinActionsCollapsed}
items={[
{
key: 'edit',
label: '编辑',
icon: <EditOutlined />,
onClick: () => { void openBuiltinConfigDrawer(record) },
},
{
key: 'trigger',
label: '触发',
@@ -967,6 +1114,14 @@ function DataSources() {
},
]}
>
<Button
type="link"
size="small"
icon={<EditOutlined />}
onClick={() => { void openBuiltinConfigDrawer(record) }}
>
</Button>
<Button
type="link"
size="small"
@@ -1035,7 +1190,7 @@ function DataSources() {
key: 'edit',
label: '编辑',
icon: <EditOutlined />,
onClick: () => openDrawer(record),
onClick: () => { void openDrawer(record) },
},
{
key: 'toggle',
@@ -1059,7 +1214,7 @@ function DataSources() {
},
]}
>
<Button type="link" size="small" icon={<EditOutlined />} onClick={() => openDrawer(record)}></Button>
<Button type="link" size="small" icon={<EditOutlined />} onClick={() => { void openDrawer(record) }}></Button>
<Button
type="link"
size="small"
@@ -1171,7 +1326,7 @@ function DataSources() {
children: (
<div className="page-shell__body data-source-custom-tab" ref={customContainerRef}>
<div className="data-source-custom-toolbar">
<Button type="primary" icon={<PlusOutlined />} onClick={() => openDrawer()}>
<Button type="primary" icon={<PlusOutlined />} onClick={() => { void openDrawer() }}>
</Button>
</div>
@@ -1215,24 +1370,40 @@ function DataSources() {
</div>
<Drawer
title={editingConfig ? '编辑数据源' : '添加数据源'}
title={builtinEditingSource ? `编辑内置数据源配置 · ${builtinEditingSource.name}` : editingConfig ? '编辑数据源' : '添加数据源'}
width={600}
open={drawerVisible}
onClose={() => {
setDrawerVisible(false)
form.resetFields()
setEditingConfig(null)
setBuiltinEditingSource(null)
setTestResult(null)
}}
footer={
<div style={{ display: 'flex', justifyContent: 'space-between' }}>
<Button
icon={<ExperimentOutlined />}
loading={testing}
onClick={handleTest}
>
</Button>
<Space>
{builtinEditingSource && editingConfig ? (
<Popconfirm
title="恢复内置默认配置?"
description="这会删除当前 override并重新使用代码内置默认配置。"
okText="恢复默认"
cancelText="取消"
onConfirm={handleResetBuiltinOverride}
>
<Button danger icon={<ClearOutlined />}>
</Button>
</Popconfirm>
) : null}
<Button
icon={<ExperimentOutlined />}
loading={testing}
onClick={handleTest}
>
</Button>
</Space>
<Space>
<Button onClick={() => setDrawerVisible(false)}></Button>
<Button type="primary" onClick={handleSave}>
@@ -1243,29 +1414,46 @@ function DataSources() {
}
>
<Form form={form} layout="vertical">
<Form.Item
name="name"
label="名称"
rules={[{ required: true, message: '请输入名称' }]}
>
<Input placeholder="My API Data Source" />
</Form.Item>
{builtinEditingSource ? (
<Card size="small" bordered={false} style={{ marginBottom: 16, background: '#fafafa' }}>
<Row gutter={[12, 12]}>
<Col span={12}>
<div style={{ marginBottom: 4, color: '#8c8c8c', fontSize: 12 }}></div>
<Input value={builtinEditingSource.name} disabled />
</Col>
<Col span={12}>
<div style={{ marginBottom: 4, color: '#8c8c8c', fontSize: 12 }}>Collector Key</div>
<Input value={builtinEditingSource.source} disabled />
</Col>
</Row>
</Card>
) : (
<Form.Item
name="name"
label="名称"
rules={[{ required: true, message: '请输入名称' }]}
>
<Input placeholder="My API Data Source" />
</Form.Item>
)}
<Form.Item name="description" label="描述">
<Input.TextArea rows={2} placeholder="数据源描述" />
</Form.Item>
<Form.Item
name="source_type"
label="数据源类型"
rules={[{ required: true, message: '请选择类型' }]}
>
<Select>
<Select.Option value="http">HTTP API</Select.Option>
<Select.Option value="api">REST API</Select.Option>
<Select.Option value="database"></Select.Option>
</Select>
</Form.Item>
{builtinEditingSource ? null : (
<Form.Item
name="source_type"
label="数据源类型"
rules={[{ required: true, message: '请选择类型' }]}
>
<Select>
<Select.Option value="http">HTTP API</Select.Option>
<Select.Option value="api">REST API</Select.Option>
<Select.Option value="database"></Select.Option>
</Select>
</Form.Item>
)}
<Form.Item
name="endpoint"
@@ -1276,6 +1464,7 @@ function DataSources() {
</Form.Item>
<Collapse
className="data-source-drawer-collapse"
items={[
{
key: 'auth',
@@ -1311,6 +1500,12 @@ function DataSources() {
<Form.Item name={['auth_config', 'key_name']} label="Header名称" initialValue="X-API-Key">
<Input placeholder="X-API-Key" />
</Form.Item>
<Form.Item name={['auth_config', 'in']} label="传递位置" initialValue="header">
<Select>
<Select.Option value="header">Header</Select.Option>
<Select.Option value="query">Query Param</Select.Option>
</Select>
</Form.Item>
<Form.Item name={['auth_config', 'api_key']} label="API Key">
<Input.Password placeholder="API Key" />
</Form.Item>
@@ -1345,6 +1540,7 @@ function DataSources() {
/>
<Collapse
className="data-source-drawer-collapse"
items={[
{
key: 'headers',
@@ -1376,6 +1572,7 @@ function DataSources() {
/>
<Collapse
className="data-source-drawer-collapse"
items={[
{
key: 'config',

View File

@@ -1,6 +1,6 @@
[project]
name = "planet"
version = "0.31.3"
version = "0.33.0"
description = "智能星球计划 - 态势感知系统"
requires-python = ">=3.14"
dependencies = [

View File

@@ -0,0 +1,134 @@
param(
[string]$BaseUrl = "http://localhost:8000",
[string]$Username,
[string]$Password,
[int]$ReadMessageCount = 5
)
Set-StrictMode -Version Latest
$ErrorActionPreference = "Stop"
if ([string]::IsNullOrWhiteSpace($Username) -or [string]::IsNullOrWhiteSpace($Password)) {
throw "Missing username/password. Pass -Username and -Password."
}
Add-Type -AssemblyName System.Net.Http
function Read-WebSocketMessage {
param(
[System.Net.WebSockets.ClientWebSocket]$Socket
)
$buffer = New-Object byte[] 8192
$segment = [System.ArraySegment[byte]]::new($buffer)
$builder = New-Object System.Text.StringBuilder
do {
$result = $Socket.ReceiveAsync($segment, [Threading.CancellationToken]::None).GetAwaiter().GetResult()
if ($result.MessageType -eq [System.Net.WebSockets.WebSocketMessageType]::Close) {
return $null
}
$builder.Append([System.Text.Encoding]::UTF8.GetString($buffer, 0, $result.Count)) | Out-Null
} while (-not $result.EndOfMessage)
return $builder.ToString()
}
$loginUri = "$BaseUrl/api/v1/auth/login"
$loginBody = "username=$([uri]::EscapeDataString($Username))&password=$([uri]::EscapeDataString($Password))"
Write-Host "Requesting access token from $loginUri ..."
$tokenResponse = Invoke-RestMethod `
-Method Post `
-Uri $loginUri `
-ContentType "application/x-www-form-urlencoded" `
-Body $loginBody
if (-not $tokenResponse.access_token) {
throw "Login succeeded but access_token missing."
}
$accessToken = [string]$tokenResponse.access_token
Write-Host "Token acquired. Expires in: $($tokenResponse.expires_in)"
$wsBase = $BaseUrl -replace "^http", "ws"
$wsUri = "$wsBase/ws?token=$accessToken"
$socket = [System.Net.WebSockets.ClientWebSocket]::new()
$socket.Options.KeepAliveInterval = [TimeSpan]::FromSeconds(20)
Write-Host "Connecting WebSocket: $wsUri"
$socket.ConnectAsync([Uri]$wsUri, [Threading.CancellationToken]::None).GetAwaiter().GetResult()
Write-Host "WebSocket connected."
try {
$initial = Read-WebSocketMessage -Socket $socket
if ($null -ne $initial) {
Write-Host ""
Write-Host "=== connection_established ==="
Write-Host $initial
}
$subscribeFrame = @{
type = "subscribe"
data = @{
channels = @("ue_scene")
}
} | ConvertTo-Json -Depth 8 -Compress
$subscribeBytes = [System.Text.Encoding]::UTF8.GetBytes($subscribeFrame)
$subscribeSegment = [System.ArraySegment[byte]]::new($subscribeBytes)
$socket.SendAsync(
$subscribeSegment,
[System.Net.WebSockets.WebSocketMessageType]::Text,
$true,
[Threading.CancellationToken]::None
).GetAwaiter().GetResult()
$subAck = Read-WebSocketMessage -Socket $socket
if ($null -ne $subAck) {
Write-Host ""
Write-Host "=== subscription_confirmed ==="
Write-Host $subAck
}
$syncFrame = @{
type = "sync_request"
data = @{
channel = "ue_scene"
reason = "initial_connect"
last_sequence = $null
}
} | ConvertTo-Json -Depth 8 -Compress
$syncBytes = [System.Text.Encoding]::UTF8.GetBytes($syncFrame)
$syncSegment = [System.ArraySegment[byte]]::new($syncBytes)
$socket.SendAsync(
$syncSegment,
[System.Net.WebSockets.WebSocketMessageType]::Text,
$true,
[Threading.CancellationToken]::None
).GetAwaiter().GetResult()
for ($i = 1; $i -le $ReadMessageCount; $i++) {
$message = Read-WebSocketMessage -Socket $socket
if ($null -eq $message) {
break
}
Write-Host ""
Write-Host "=== message $i ==="
Write-Host $message
}
}
finally {
if ($socket.State -eq [System.Net.WebSockets.WebSocketState]::Open) {
$socket.CloseAsync(
[System.Net.WebSockets.WebSocketCloseStatus]::NormalClosure,
"done",
[Threading.CancellationToken]::None
).GetAwaiter().GetResult()
}
$socket.Dispose()
}

View File

@@ -0,0 +1,264 @@
# BP_GlobeController 接 `ue_scene` 数据接口指南
## 这份文档解决什么问题
这份文档专门讲:
1. 后端 `ue_scene` 已经准备好之后,如何先在本机验证接口
2. UE 展示端接接口时,消息应该怎么走
3. `BP_GlobeController` 后续该预留哪些函数和状态
适合当前阶段:
- 你已经有 `Earth_Globe`
- 你已经有一个能展示地球的 Demo
- 现在要开始把后端的 `ue_scene` 数据接进来
## 当前后端入口
后端 WebSocket 入口在:
- [backend/app/api/v1/websocket.py](/D:/work/planet/planet/backend/app/api/v1/websocket.py:1)
`ue_scene` 数据构建在:
- [backend/app/core/websocket/ue_scene.py](/D:/work/planet/planet/backend/app/core/websocket/ue_scene.py:1)
协议草案在:
- [docs/ue-ndisplay-websocket-protocol.md](/D:/work/planet/planet/docs/ue-ndisplay-websocket-protocol.md:1)
## 先不要直接进 UE
建议先做一件事:
先在本机验证这条链路是通的:
1. 登录后端拿 `access_token`
2. 用这个 token 连接 `/ws`
3. 订阅 `ue_scene`
4. 发送 `sync_request`
5. 确认后端回了 `full` 快照
这一步跑通后,再把同样的消息流程搬到 UE。
## 本机验证脚本
仓库里已经补了测试脚本:
- [scripts/test_ue_scene_ws.ps1](/D:/work/planet/planet/scripts/test_ue_scene_ws.ps1)
### 用法
```powershell
powershell -ExecutionPolicy Bypass -File scripts/test_ue_scene_ws.ps1 `
-BaseUrl http://localhost:8000 `
-Username 你的用户名 `
-Password 你的密码 `
-ReadMessageCount 3
```
### 预期输出
你应该能看到这些消息:
1. `connection_established`
2. `subscription_confirmed`
3. 至少一条 `data_frame`
其中第一条 `data_frame` 里:
- `channel = ue_scene`
- `update_type = full`
如果这一步失败,不要先去 UE 里找问题。
## 连接顺序
UE 主节点后续接接口时,顺序就是下面这样:
1. 登录拿 token
2. 连接 `ws://<host>:8000/ws?token=<access_token>`
3. 收到 `connection_established`
4.`subscribe`
5.`sync_request`
6.`data_frame`
7. 应用 `full`
8. 后续继续应用 `incremental`
## WebSocket 地址
```text
ws://localhost:8000/ws?token=<access_token>
```
## `subscribe` 消息
```json
{
"type": "subscribe",
"data": {
"channels": ["ue_scene"]
}
}
```
## `sync_request` 消息
```json
{
"type": "sync_request",
"data": {
"channel": "ue_scene",
"reason": "initial_connect",
"last_sequence": null
}
}
```
## 你在 UE 里要接收的重点字段
### 1. `data.update_type`
可能是:
- `full`
- `incremental`
### 2. `data.sequence`
每次场景更新都会递增。
UE 主节点要保存自己最近一次成功应用的 `sequence`
### 3. `data.payload.layers`
`full` 快照时,场景实体都在这里。
重点层包括:
- `satellites`
- `supercomputers`
- `gpu_clusters`
- `submarine_cables`
- `landing_points`
- `alerts`
### 4. `data.changes`
`incremental` 更新时,变化都在这里。
每一层会包含:
- `added`
- `updated`
- `removed`
## `BP_GlobeController` 建议先预留的变量
### `LastSequence`
- 类型:整数
- 用途:记录最近一次成功应用的序列号
### `SceneConnected`
- 类型:布尔
- 用途:当前是否已连上后端 WebSocket
### `PendingFullSync`
- 类型:布尔
- 用途:是否正在等待全量快照
### `CurrentSceneHash`
- 类型:字符串
- 用途:记录当前已应用场景的 `state_hash`
## `BP_GlobeController` 建议先预留的函数
### `ConnectWebSocket`
负责:
- 建立连接
- 进入已连接状态
### `SendSubscribe`
负责:
- 订阅 `ue_scene`
### `SendSyncRequest`
负责:
- 请求 `full` 快照
### `HandleMessage`
负责:
- 解析服务端返回的原始 JSON
### `ApplyFullScene`
负责:
- 清理旧场景数据
-`layers` 重建当前场景
### `ApplyIncrementalScene`
负责:
- 处理 `added`
- 处理 `updated`
- 处理 `removed`
### `GeoToWorld`
负责:
-`lat/lng/alt` 转成球面世界坐标
## 第一个可落地目标
不要一上来就把所有层都接进来。
建议第一阶段只做:
1. 连接成功
2. 收到 `full`
3. 先只解析 `gpu_clusters`
4. 在地球上生成几个测试点位
这样最容易判断问题在哪:
- 是连接没通
- 还是 JSON 没解析对
- 还是地理坐标映射有问题
## 当前阶段建议
对于你现在这个 Demo建议下一步按这个顺序
1. 先跑 [scripts/test_ue_scene_ws.ps1](/D:/work/planet/planet/scripts/test_ue_scene_ws.ps1:1)
2. 确认后端确实回 `ue_scene`
3. 再在 UE 里接登录和 WebSocket
4. 先只落 `ApplyFullScene`
5. 最后再接 `ApplyIncrementalScene`
## 一个现实提醒
UE 原生 `WebSockets` 插件更适合 C++ 层接入,不是最友好的纯蓝图体验。
所以如果你下一步要我继续带着落地,我建议走这条路线:
1. 先把协议和测试跑通
2. 再补一个 UE 侧最小 C++ WebSocket 接收层
3. 最后把消息转给 `BP_GlobeController`
这样会比纯蓝图硬接稳定很多。

View File

@@ -0,0 +1,22 @@
using UnrealBuildTool;
public class Planet : ModuleRules
{
public Planet(ReadOnlyTargetRules Target) : base(Target)
{
PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;
PublicDependencyModuleNames.AddRange(
new[]
{
"Core",
"CoreUObject",
"Engine",
"HTTP",
"Json",
"JsonUtilities",
"WebSockets",
}
);
}
}

View File

@@ -0,0 +1,677 @@
#include "UeSceneWebSocketClient.h"
#include "Dom/JsonObject.h"
#include "Dom/JsonValue.h"
#include "HttpModule.h"
#include "JsonObjectConverter.h"
#include "Serialization/JsonReader.h"
#include "Serialization/JsonSerializer.h"
#include "WebSocketsModule.h"
namespace
{
constexpr double GeoPointMergeTolerance = 1e-4;
bool TryParseGeoPoint(const TSharedPtr<FJsonObject>& PointObject, FVector& OutGeoPoint)
{
if (!PointObject.IsValid())
{
return false;
}
double Latitude = 0.0;
double Longitude = 0.0;
double Altitude = 0.0;
if (!PointObject->TryGetNumberField(TEXT("lat"), Latitude)
|| !PointObject->TryGetNumberField(TEXT("lng"), Longitude))
{
return false;
}
PointObject->TryGetNumberField(TEXT("alt"), Altitude);
OutGeoPoint = FVector(Longitude, Latitude, Altitude);
return true;
}
FString ResolveCableName(const TSharedPtr<FJsonObject>& ItemObject)
{
FString CableName;
const TSharedPtr<FJsonObject>* LabelsObject = nullptr;
if (ItemObject->TryGetObjectField(TEXT("labels"), LabelsObject)
&& LabelsObject != nullptr
&& LabelsObject->IsValid())
{
(*LabelsObject)->TryGetStringField(TEXT("title"), CableName);
}
if (CableName.IsEmpty())
{
ItemObject->TryGetStringField(TEXT("id"), CableName);
}
return CableName;
}
bool AreGeoPointsConnected(const FVector& A, const FVector& B)
{
return FMath::Abs(A.X - B.X) <= GeoPointMergeTolerance
&& FMath::Abs(A.Y - B.Y) <= GeoPointMergeTolerance
&& FMath::Abs(A.Z - B.Z) <= GeoPointMergeTolerance;
}
void AppendMergedSegment(
TArray<TArray<FVector>>& MergedSegments,
const TArray<FVector>& IncomingSegment
)
{
if (IncomingSegment.Num() < 2)
{
return;
}
if (MergedSegments.Num() == 0)
{
MergedSegments.Add(IncomingSegment);
return;
}
TArray<FVector>& LastMergedSegment = MergedSegments.Last();
if (LastMergedSegment.Num() == 0)
{
LastMergedSegment = IncomingSegment;
return;
}
if (AreGeoPointsConnected(LastMergedSegment.Last(), IncomingSegment[0]))
{
for (int32 Index = 1; Index < IncomingSegment.Num(); ++Index)
{
LastMergedSegment.Add(IncomingSegment[Index]);
}
return;
}
MergedSegments.Add(IncomingSegment);
}
}
UUeSceneWebSocketClient::UUeSceneWebSocketClient()
{
PrimaryComponentTick.bCanEverTick = false;
}
void UUeSceneWebSocketClient::BeginPlay()
{
Super::BeginPlay();
}
void UUeSceneWebSocketClient::EndPlay(const EEndPlayReason::Type EndPlayReason)
{
DisconnectFromBackend();
Super::EndPlay(EndPlayReason);
}
void UUeSceneWebSocketClient::ConnectToBackend()
{
if (Username.IsEmpty() || Password.IsEmpty())
{
EmitError(TEXT("Username / Password 不能为空。"));
return;
}
RequestAccessToken();
}
void UUeSceneWebSocketClient::DisconnectFromBackend()
{
if (Socket.IsValid())
{
Socket->Close();
Socket.Reset();
}
PendingFullScene.Reset();
OnConnectedChanged.Broadcast(false);
}
void UUeSceneWebSocketClient::SendSubscribe()
{
TSharedRef<FJsonObject> Root = MakeShared<FJsonObject>();
Root->SetStringField(TEXT("type"), TEXT("subscribe"));
TSharedRef<FJsonObject> Data = MakeShared<FJsonObject>();
TArray<TSharedPtr<FJsonValue>> Channels;
for (const FString& Channel : DefaultChannels)
{
Channels.Add(MakeShared<FJsonValueString>(Channel));
}
Data->SetArrayField(TEXT("channels"), Channels);
Root->SetObjectField(TEXT("data"), Data);
FString Payload;
TSharedRef<TJsonWriter<>> Writer = TJsonWriterFactory<>::Create(&Payload);
FJsonSerializer::Serialize(Root, Writer);
SendJsonString(Payload);
}
void UUeSceneWebSocketClient::SendInitialSyncRequest()
{
SendSyncRequest(TEXT("initial_connect"), INDEX_NONE);
}
void UUeSceneWebSocketClient::SendSyncRequest(const FString& Reason, int32 InLastSequence)
{
TSharedRef<FJsonObject> Root = MakeShared<FJsonObject>();
Root->SetStringField(TEXT("type"), TEXT("sync_request"));
TSharedRef<FJsonObject> Data = MakeShared<FJsonObject>();
Data->SetStringField(TEXT("channel"), TEXT("ue_scene"));
Data->SetStringField(TEXT("reason"), Reason);
if (InLastSequence >= 0)
{
Data->SetNumberField(TEXT("last_sequence"), InLastSequence);
}
Root->SetObjectField(TEXT("data"), Data);
FString Payload;
TSharedRef<TJsonWriter<>> Writer = TJsonWriterFactory<>::Create(&Payload);
FJsonSerializer::Serialize(Root, Writer);
SendJsonString(Payload);
}
void UUeSceneWebSocketClient::RequestAccessToken()
{
TSharedRef<IHttpRequest, ESPMode::ThreadSafe> Request = FHttpModule::Get().CreateRequest();
Request->SetURL(BuildHttpUrl(TEXT("/api/v1/auth/login")));
Request->SetVerb(TEXT("POST"));
Request->SetHeader(TEXT("Content-Type"), TEXT("application/x-www-form-urlencoded"));
Request->SetContentAsString(
FString::Printf(TEXT("username=%s&password=%s"), *Username, *Password)
);
Request->OnProcessRequestComplete().BindUObject(
this, &UUeSceneWebSocketClient::HandleLoginResponse
);
Request->ProcessRequest();
}
void UUeSceneWebSocketClient::HandleLoginResponse(
FHttpRequestPtr Request,
FHttpResponsePtr Response,
bool bSucceeded
)
{
if (!bSucceeded || !Response.IsValid())
{
EmitError(TEXT("登录请求失败。"));
return;
}
if (Response->GetResponseCode() < 200 || Response->GetResponseCode() >= 300)
{
EmitError(FString::Printf(TEXT("登录失败HTTP %d"), Response->GetResponseCode()));
return;
}
TSharedPtr<FJsonObject> Root;
TSharedRef<TJsonReader<>> Reader = TJsonReaderFactory<>::Create(Response->GetContentAsString());
if (!FJsonSerializer::Deserialize(Reader, Root) || !Root.IsValid())
{
EmitError(TEXT("登录响应 JSON 解析失败。"));
return;
}
AccessToken = Root->GetStringField(TEXT("access_token"));
if (AccessToken.IsEmpty())
{
EmitError(TEXT("登录响应里没有 access_token。"));
return;
}
OpenWebSocket();
}
void UUeSceneWebSocketClient::OpenWebSocket()
{
if (!FModuleManager::Get().IsModuleLoaded(TEXT("WebSockets")))
{
FModuleManager::Get().LoadModule(TEXT("WebSockets"));
}
Socket = FWebSocketsModule::Get().CreateWebSocket(BuildWsUrl());
BindSocketEvents();
Socket->Connect();
}
void UUeSceneWebSocketClient::BindSocketEvents()
{
if (!Socket.IsValid())
{
return;
}
Socket->OnConnected().AddLambda([this]()
{
OnConnectedChanged.Broadcast(true);
});
Socket->OnConnectionError().AddLambda([this](const FString& Error)
{
EmitError(FString::Printf(TEXT("WebSocket 连接失败: %s"), *Error));
});
Socket->OnClosed().AddLambda([this](int32 StatusCode, const FString& Reason, bool bWasClean)
{
OnConnectedChanged.Broadcast(false);
});
Socket->OnMessage().AddLambda([this](const FString& Message)
{
OnRawMessage.Broadcast(Message);
TryBroadcastSceneFrame(Message);
});
}
void UUeSceneWebSocketClient::EmitError(const FString& ErrorMessage)
{
OnError.Broadcast(ErrorMessage);
}
void UUeSceneWebSocketClient::TryBroadcastSceneFrame(const FString& Message)
{
TSharedPtr<FJsonObject> Root;
TSharedRef<TJsonReader<>> Reader = TJsonReaderFactory<>::Create(Message);
if (!FJsonSerializer::Deserialize(Reader, Root) || !Root.IsValid())
{
return;
}
FString Type;
if (!Root->TryGetStringField(TEXT("type"), Type) || Type != TEXT("data_frame"))
{
return;
}
FString Channel;
if (!Root->TryGetStringField(TEXT("channel"), Channel) || Channel != TEXT("ue_scene"))
{
return;
}
const TSharedPtr<FJsonObject>* DataObject = nullptr;
if (!Root->TryGetObjectField(TEXT("data"), DataObject) || DataObject == nullptr || !DataObject->IsValid())
{
return;
}
TSharedPtr<FJsonObject> EffectiveDataObject = *DataObject;
FString UpdateType = TEXT("unknown");
EffectiveDataObject->TryGetStringField(TEXT("update_type"), UpdateType);
int32 Sequence = 0;
EffectiveDataObject->TryGetNumberField(TEXT("sequence"), Sequence);
int32 TotalRecords = 0;
const TSharedPtr<FJsonObject>* PayloadObject = nullptr;
if (EffectiveDataObject->TryGetObjectField(TEXT("payload"), PayloadObject) && PayloadObject != nullptr && PayloadObject->IsValid())
{
const TSharedPtr<FJsonObject>* MetaObject = nullptr;
if ((*PayloadObject)->TryGetObjectField(TEXT("meta"), MetaObject) && MetaObject != nullptr && MetaObject->IsValid())
{
(*MetaObject)->TryGetNumberField(TEXT("total_records"), TotalRecords);
}
}
if (UpdateType == TEXT("full"))
{
TSharedPtr<FJsonObject> AssembledDataObject;
if (!TryAssembleFullScene(EffectiveDataObject, Sequence, TotalRecords, AssembledDataObject))
{
return;
}
if (AssembledDataObject.IsValid())
{
EffectiveDataObject = AssembledDataObject;
}
}
LastSequence = Sequence;
OnSceneFrame.Broadcast(UpdateType, Sequence, TotalRecords);
BroadcastScenePayload(UpdateType, Sequence, EffectiveDataObject);
}
bool UUeSceneWebSocketClient::TryAssembleFullScene(
const TSharedPtr<FJsonObject>& DataObject,
int32 Sequence,
int32 TotalRecords,
TSharedPtr<FJsonObject>& OutAssembledDataObject
)
{
OutAssembledDataObject = nullptr;
const TSharedPtr<FJsonObject>* PayloadObject = nullptr;
if (!DataObject->TryGetObjectField(TEXT("payload"), PayloadObject) || PayloadObject == nullptr || !PayloadObject->IsValid())
{
EmitError(TEXT("Full scene payload is missing."));
return false;
}
const TSharedPtr<FJsonObject>* MetaObject = nullptr;
if (!(*PayloadObject)->TryGetObjectField(TEXT("meta"), MetaObject) || MetaObject == nullptr || !MetaObject->IsValid())
{
EmitError(TEXT("Full scene meta is missing."));
return false;
}
bool bChunked = false;
(*MetaObject)->TryGetBoolField(TEXT("chunked"), bChunked);
if (!bChunked)
{
OutAssembledDataObject = DataObject;
PendingFullScene.Reset();
return true;
}
int32 TransportChunkIndex = 0;
int32 TransportChunkCount = 0;
int32 LayerChunkIndex = 0;
int32 LayerChunkCount = 0;
FString LayerName;
(*MetaObject)->TryGetNumberField(TEXT("transport_chunk_index"), TransportChunkIndex);
(*MetaObject)->TryGetNumberField(TEXT("transport_chunk_count"), TransportChunkCount);
(*MetaObject)->TryGetNumberField(TEXT("layer_chunk_index"), LayerChunkIndex);
(*MetaObject)->TryGetNumberField(TEXT("layer_chunk_count"), LayerChunkCount);
(*MetaObject)->TryGetStringField(TEXT("layer_name"), LayerName);
if (TransportChunkIndex <= 0 || TransportChunkCount <= 0 || LayerChunkIndex <= 0 || LayerChunkCount <= 0 || LayerName.IsEmpty())
{
EmitError(TEXT("Chunked full scene metadata is incomplete."));
PendingFullScene.Reset();
return false;
}
if (PendingFullScene.Sequence != Sequence)
{
PendingFullScene.Reset();
PendingFullScene.Sequence = Sequence;
PendingFullScene.ExpectedTransportChunkCount = TransportChunkCount;
PendingFullScene.TotalRecords = TotalRecords;
PendingFullScene.DisplayProfile = DataObject->GetObjectField(TEXT("display_profile"));
PendingFullScene.CameraState = DataObject->GetObjectField(TEXT("camera_state"));
PendingFullScene.Meta = MakeShared<FJsonObject>(*MetaObject->Get());
DataObject->TryGetStringField(TEXT("cluster_time"), PendingFullScene.ClusterTime);
}
if (PendingFullScene.Sequence != Sequence || PendingFullScene.ExpectedTransportChunkCount != TransportChunkCount)
{
EmitError(TEXT("Chunked full scene sequence mismatch."));
PendingFullScene.Reset();
return false;
}
if (PendingFullScene.ReceivedTransportChunkIndices.Contains(TransportChunkIndex))
{
return false;
}
PendingFullScene.ReceivedTransportChunkIndices.Add(TransportChunkIndex);
const TSharedPtr<FJsonObject>* LayersObject = nullptr;
if (!(*PayloadObject)->TryGetObjectField(TEXT("layers"), LayersObject) || LayersObject == nullptr || !LayersObject->IsValid())
{
EmitError(TEXT("Chunked full scene layers are missing."));
PendingFullScene.Reset();
return false;
}
const TSharedPtr<FJsonObject>* LayerDataObject = nullptr;
if (!(*LayersObject)->TryGetObjectField(LayerName, LayerDataObject) || LayerDataObject == nullptr || !LayerDataObject->IsValid())
{
EmitError(TEXT("Chunked full scene layer payload is missing."));
PendingFullScene.Reset();
return false;
}
FLayerChunkAccumulator& LayerAccumulator = PendingFullScene.Layers.FindOrAdd(LayerName);
if (LayerAccumulator.ExpectedChunkCount == 0)
{
LayerAccumulator.ExpectedChunkCount = LayerChunkCount;
LayerDataObject->Get()->TryGetNumberField(TEXT("revision"), LayerAccumulator.Revision);
}
else if (LayerAccumulator.ExpectedChunkCount != LayerChunkCount)
{
EmitError(TEXT("Layer chunk count changed while assembling full scene."));
PendingFullScene.Reset();
return false;
}
if (LayerAccumulator.ReceivedChunkIndices.Contains(LayerChunkIndex))
{
return false;
}
const TArray<TSharedPtr<FJsonValue>>* LayerItems = nullptr;
if (!LayerDataObject->Get()->TryGetArrayField(TEXT("items"), LayerItems) || LayerItems == nullptr)
{
EmitError(TEXT("Chunked full scene items are missing."));
PendingFullScene.Reset();
return false;
}
LayerAccumulator.ReceivedChunkIndices.Add(LayerChunkIndex);
LayerAccumulator.Items.Append(*LayerItems);
if (PendingFullScene.ReceivedTransportChunkIndices.Num() != PendingFullScene.ExpectedTransportChunkCount)
{
return false;
}
for (const TPair<FString, FLayerChunkAccumulator>& LayerPair : PendingFullScene.Layers)
{
if (LayerPair.Value.ReceivedChunkIndices.Num() != LayerPair.Value.ExpectedChunkCount)
{
return false;
}
}
TSharedRef<FJsonObject> AssembledDataObject = MakeShared<FJsonObject>();
AssembledDataObject->SetStringField(TEXT("update_type"), TEXT("full"));
AssembledDataObject->SetNumberField(TEXT("sequence"), Sequence);
AssembledDataObject->SetStringField(TEXT("cluster_time"), PendingFullScene.ClusterTime);
if (PendingFullScene.DisplayProfile.IsValid())
{
AssembledDataObject->SetObjectField(TEXT("display_profile"), PendingFullScene.DisplayProfile);
}
if (PendingFullScene.CameraState.IsValid())
{
AssembledDataObject->SetObjectField(TEXT("camera_state"), PendingFullScene.CameraState);
}
TSharedRef<FJsonObject> AssembledPayloadObject = MakeShared<FJsonObject>();
TSharedRef<FJsonObject> AssembledMetaObject = PendingFullScene.Meta.IsValid()
? MakeShared<FJsonObject>(*PendingFullScene.Meta.Get())
: MakeShared<FJsonObject>();
AssembledMetaObject->SetBoolField(TEXT("chunked"), false);
AssembledMetaObject->RemoveField(TEXT("layer_name"));
AssembledMetaObject->RemoveField(TEXT("layer_chunk_index"));
AssembledMetaObject->RemoveField(TEXT("layer_chunk_count"));
AssembledMetaObject->RemoveField(TEXT("transport_chunk_index"));
AssembledMetaObject->RemoveField(TEXT("transport_chunk_count"));
AssembledPayloadObject->SetObjectField(TEXT("meta"), AssembledMetaObject);
TSharedRef<FJsonObject> AssembledLayersObject = MakeShared<FJsonObject>();
for (TPair<FString, FLayerChunkAccumulator>& LayerPair : PendingFullScene.Layers)
{
TSharedRef<FJsonObject> LayerObject = MakeShared<FJsonObject>();
LayerObject->SetNumberField(TEXT("revision"), LayerPair.Value.Revision);
LayerObject->SetArrayField(TEXT("items"), LayerPair.Value.Items);
AssembledLayersObject->SetObjectField(LayerPair.Key, LayerObject);
}
AssembledPayloadObject->SetObjectField(TEXT("layers"), AssembledLayersObject);
AssembledDataObject->SetObjectField(TEXT("payload"), AssembledPayloadObject);
OutAssembledDataObject = AssembledDataObject;
PendingFullScene.Reset();
return true;
}
void UUeSceneWebSocketClient::BroadcastScenePayload(
const FString& UpdateType,
int32 Sequence,
const TSharedPtr<FJsonObject>& DataObject
)
{
FString PayloadJson;
TSharedRef<TJsonWriter<>> Writer = TJsonWriterFactory<>::Create(&PayloadJson);
if (FJsonSerializer::Serialize(DataObject.ToSharedRef(), Writer))
{
OnScenePayload.Broadcast(UpdateType, Sequence, PayloadJson);
}
BroadcastLegacySubmarineCableEvents(DataObject);
}
void UUeSceneWebSocketClient::BroadcastLegacySubmarineCableEvents(
const TSharedPtr<FJsonObject>& DataObject
)
{
if (!DataObject.IsValid())
{
return;
}
const TSharedPtr<FJsonObject>* PayloadObject = nullptr;
if (!DataObject->TryGetObjectField(TEXT("payload"), PayloadObject)
|| PayloadObject == nullptr
|| !PayloadObject->IsValid())
{
return;
}
const TSharedPtr<FJsonObject>* LayersObject = nullptr;
if (!(*PayloadObject)->TryGetObjectField(TEXT("layers"), LayersObject)
|| LayersObject == nullptr
|| !LayersObject->IsValid())
{
return;
}
const TSharedPtr<FJsonObject>* CableLayerObject = nullptr;
if (!(*LayersObject)->TryGetObjectField(TEXT("submarine_cables"), CableLayerObject)
|| CableLayerObject == nullptr
|| !CableLayerObject->IsValid())
{
return;
}
const TArray<TSharedPtr<FJsonValue>>* CableItems = nullptr;
if (!(*CableLayerObject)->TryGetArrayField(TEXT("items"), CableItems) || CableItems == nullptr)
{
return;
}
for (const TSharedPtr<FJsonValue>& CableItemValue : *CableItems)
{
const TSharedPtr<FJsonObject> CableItemObject = CableItemValue.IsValid()
? CableItemValue->AsObject()
: nullptr;
if (!CableItemObject.IsValid())
{
continue;
}
const FString CableName = ResolveCableName(CableItemObject);
const TSharedPtr<FJsonObject>* GeoObject = nullptr;
if (!CableItemObject->TryGetObjectField(TEXT("geo"), GeoObject)
|| GeoObject == nullptr
|| !GeoObject->IsValid())
{
continue;
}
const TArray<TSharedPtr<FJsonValue>>* SegmentArray = nullptr;
if ((*GeoObject)->TryGetArrayField(TEXT("segments"), SegmentArray) && SegmentArray != nullptr)
{
TArray<TArray<FVector>> MergedSegments;
for (const TSharedPtr<FJsonValue>& SegmentValue : *SegmentArray)
{
const TArray<TSharedPtr<FJsonValue>>* SegmentPoints = nullptr;
if (!SegmentValue.IsValid() || !SegmentValue->TryGetArray(SegmentPoints) || SegmentPoints == nullptr)
{
continue;
}
TArray<FVector> GeoPoints;
GeoPoints.Reserve(SegmentPoints->Num());
for (const TSharedPtr<FJsonValue>& PointValue : *SegmentPoints)
{
FVector GeoPoint = FVector::ZeroVector;
if (PointValue.IsValid() && TryParseGeoPoint(PointValue->AsObject(), GeoPoint))
{
GeoPoints.Add(GeoPoint);
}
}
AppendMergedSegment(MergedSegments, GeoPoints);
}
for (const TArray<FVector>& MergedSegment : MergedSegments)
{
if (MergedSegment.Num() >= 2)
{
OnSubmarineCableReceived.Broadcast(CableName, MergedSegment);
}
}
continue;
}
const TArray<TSharedPtr<FJsonValue>>* PathArray = nullptr;
if (!(*GeoObject)->TryGetArrayField(TEXT("path"), PathArray) || PathArray == nullptr)
{
continue;
}
TArray<FVector> GeoPoints;
GeoPoints.Reserve(PathArray->Num());
for (const TSharedPtr<FJsonValue>& PointValue : *PathArray)
{
FVector GeoPoint = FVector::ZeroVector;
if (PointValue.IsValid() && TryParseGeoPoint(PointValue->AsObject(), GeoPoint))
{
GeoPoints.Add(GeoPoint);
}
}
if (GeoPoints.Num() >= 2)
{
OnSubmarineCableReceived.Broadcast(CableName, GeoPoints);
}
}
}
FString UUeSceneWebSocketClient::BuildWsUrl() const
{
FString WsBase = BackendBaseUrl;
WsBase = WsBase.Replace(TEXT("http://"), TEXT("ws://"));
WsBase = WsBase.Replace(TEXT("https://"), TEXT("wss://"));
return FString::Printf(TEXT("%s/ws?token=%s"), *WsBase, *AccessToken);
}
FString UUeSceneWebSocketClient::BuildHttpUrl(const FString& Path) const
{
return FString::Printf(TEXT("%s%s"), *BackendBaseUrl, *Path);
}
void UUeSceneWebSocketClient::SendJsonString(const FString& JsonPayload)
{
if (!Socket.IsValid() || !Socket->IsConnected())
{
EmitError(TEXT("WebSocket 尚未连接。"));
return;
}
Socket->Send(JsonPayload);
}

View File

@@ -0,0 +1,159 @@
#pragma once
#include "CoreMinimal.h"
#include "Components/ActorComponent.h"
#include "Interfaces/IHttpRequest.h"
#include "Interfaces/IHttpResponse.h"
#include "IWebSocket.h"
#include "UeSceneWebSocketClient.generated.h"
DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam(FUeSceneRawMessageEvent, const FString&, Message);
DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam(FUeSceneConnectedEvent, bool, bConnected);
DECLARE_DYNAMIC_MULTICAST_DELEGATE_OneParam(FUeSceneErrorEvent, const FString&, ErrorMessage);
DECLARE_DYNAMIC_MULTICAST_DELEGATE_ThreeParams(
FUeSceneFrameEvent,
const FString&,
UpdateType,
int32,
Sequence,
int32,
TotalRecords
);
DECLARE_DYNAMIC_MULTICAST_DELEGATE_ThreeParams(
FUeScenePayloadEvent,
const FString&,
UpdateType,
int32,
Sequence,
const FString&,
PayloadJson
);
DECLARE_DYNAMIC_MULTICAST_DELEGATE_TwoParams(
FUeSceneSubmarineCableEvent,
const FString&,
CableName,
const TArray<FVector>&,
GeoPoints
);
UCLASS(ClassGroup=(Planet), meta=(BlueprintSpawnableComponent))
class PLANET_API UUeSceneWebSocketClient : public UActorComponent
{
GENERATED_BODY()
public:
UUeSceneWebSocketClient();
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Planet|Backend")
FString BackendBaseUrl = TEXT("http://127.0.0.1:8000");
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Planet|Backend")
FString Username;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Planet|Backend")
FString Password;
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Planet|Backend")
TArray<FString> DefaultChannels = {TEXT("ue_scene")};
UPROPERTY(EditAnywhere, BlueprintReadWrite, Category="Planet|Backend")
int32 LastSequence = INDEX_NONE;
UPROPERTY(BlueprintAssignable, Category="Planet|Backend")
FUeSceneConnectedEvent OnConnectedChanged;
UPROPERTY(BlueprintAssignable, Category="Planet|Backend")
FUeSceneRawMessageEvent OnRawMessage;
UPROPERTY(BlueprintAssignable, Category="Planet|Backend")
FUeSceneErrorEvent OnError;
UPROPERTY(BlueprintAssignable, Category="Planet|Backend")
FUeSceneFrameEvent OnSceneFrame;
UPROPERTY(BlueprintAssignable, Category="Planet|Backend")
FUeScenePayloadEvent OnScenePayload;
UPROPERTY(BlueprintAssignable, Category="Planet|Backend|Legacy")
FUeSceneSubmarineCableEvent OnSubmarineCableReceived;
UFUNCTION(BlueprintCallable, Category="Planet|Backend")
void ConnectToBackend();
UFUNCTION(BlueprintCallable, Category="Planet|Backend")
void DisconnectFromBackend();
UFUNCTION(BlueprintCallable, Category="Planet|Backend")
void SendSubscribe();
UFUNCTION(BlueprintCallable, Category="Planet|Backend")
void SendInitialSyncRequest();
UFUNCTION(BlueprintCallable, Category="Planet|Backend")
void SendSyncRequest(const FString& Reason, int32 InLastSequence);
protected:
virtual void BeginPlay() override;
virtual void EndPlay(const EEndPlayReason::Type EndPlayReason) override;
private:
struct FLayerChunkAccumulator
{
int32 Revision = 0;
int32 ExpectedChunkCount = 0;
TSet<int32> ReceivedChunkIndices;
TArray<TSharedPtr<FJsonValue>> Items;
};
struct FFullSceneAssemblyState
{
int32 Sequence = INDEX_NONE;
int32 ExpectedTransportChunkCount = 0;
int32 TotalRecords = 0;
FString ClusterTime;
TSharedPtr<FJsonObject> DisplayProfile;
TSharedPtr<FJsonObject> CameraState;
TSharedPtr<FJsonObject> Meta;
TMap<FString, FLayerChunkAccumulator> Layers;
TSet<int32> ReceivedTransportChunkIndices;
void Reset()
{
Sequence = INDEX_NONE;
ExpectedTransportChunkCount = 0;
TotalRecords = 0;
ClusterTime.Reset();
DisplayProfile.Reset();
CameraState.Reset();
Meta.Reset();
Layers.Reset();
ReceivedTransportChunkIndices.Reset();
}
};
FString AccessToken;
TSharedPtr<IWebSocket> Socket;
FFullSceneAssemblyState PendingFullScene;
void RequestAccessToken();
void HandleLoginResponse(FHttpRequestPtr Request, FHttpResponsePtr Response, bool bSucceeded);
void OpenWebSocket();
void BindSocketEvents();
void EmitError(const FString& ErrorMessage);
void TryBroadcastSceneFrame(const FString& Message);
bool TryAssembleFullScene(
const TSharedPtr<FJsonObject>& DataObject,
int32 Sequence,
int32 TotalRecords,
TSharedPtr<FJsonObject>& OutAssembledDataObject
);
void BroadcastScenePayload(
const FString& UpdateType,
int32 Sequence,
const TSharedPtr<FJsonObject>& DataObject
);
void BroadcastLegacySubmarineCableEvents(const TSharedPtr<FJsonObject>& DataObject);
FString BuildWsUrl() const;
FString BuildHttpUrl(const FString& Path) const;
void SendJsonString(const FString& JsonPayload);
};

View File

@@ -0,0 +1,189 @@
# UE5.7 连接 `ue_scene` 数据接口实操
## 目标
把当前后端的 `ue_scene` WebSocket 数据接到 UE5.7 工程里。
当前后端入口:
- [backend/app/api/v1/websocket.py](/D:/work/planet/planet/backend/app/api/v1/websocket.py:1)
- [backend/app/core/websocket/ue_scene.py](/D:/work/planet/planet/backend/app/core/websocket/ue_scene.py:1)
本次补的 UE 侧最小 C++ 文件:
- [Planet.Build.cs](/D:/work/planet/planet/unreal/Source/Planet/Planet.Build.cs:1)
- [UeSceneWebSocketClient.h](/D:/work/planet/planet/unreal/Source/Planet/Public/UeSceneWebSocketClient.h:1)
- [UeSceneWebSocketClient.cpp](/D:/work/planet/planet/unreal/Source/Planet/Private/UeSceneWebSocketClient.cpp:1)
## 这一步先做什么
先做最小闭环:
1. UE 登录后端拿 token
2. UE 连接 `/ws`
3. UE 订阅 `ue_scene`
4. UE 发送 `sync_request`
5. UE 收到原始 JSON
先不急着做完整场景生成。
## 第 1 步:把代码放进你的 UE5.7 工程
你的真实 UE 工程里应该有:
```text
YourProject/
Source/
YourProject/
```
把这里的示例代码内容按模块名改进去:
- `Planet.Build.cs`
- `UeSceneWebSocketClient.h`
- `UeSceneWebSocketClient.cpp`
如果你的工程模块名不是 `Planet`,要把:
- `PLANET_API`
改成你自己的模块导出宏。
## 第 2 步:启用插件
在 UE5.7 编辑器里启用:
- `WebSockets`
- `HTTP`
- `Json Blueprint Utilities`(可选)
启用后重启编辑器。
## 第 3 步:重新生成并编译工程
如果你是 Visual Studio 工作流:
1. 右键 `.uproject`
2. 选择“生成 Visual Studio 项目文件”
3. 打开 `.sln`
4. 编译 `Development Editor`
## 第 4 步:把组件挂到 `BP_GlobeController`
编译成功后:
1. 打开 `BP_GlobeController`
2. 点击 `添加`
3. 搜索:
- `Ue Scene Web Socket Client`
4. 把这个组件挂进去
## 第 5 步:填写后端参数
选中这个组件,在 `细节` 面板里填写:
- `Backend Base Url``http://127.0.0.1:8000`
- `Username`:你的后端用户名
- `Password`:你的后端密码
默认频道已经是:
- `ue_scene`
## 第 6 步:在蓝图里接事件
`BP_GlobeController` 里接这几个事件:
### `On Connected Changed`
连接成功后:
1.`Send Subscribe`
2. 再调 `Send Initial Sync Request`
### `On Raw Message`
先不要急着解析全部结构。
第一步只做:
1. 把原始 JSON 打印出来
2. 或保存到字符串变量
这样先确认 UE 确实收到后端消息。
### `On Error`
把错误字符串打印到屏幕和日志里。
## 蓝图推荐连法
### 开始游戏时
1. `开始游戏时`
2. 调组件的 `Connect To Backend`
### 连接成功后
1. `On Connected Changed`
2. `分支`
3. 如果 `bConnected = true`
4.`Send Subscribe`
5.`Send Initial Sync Request`
### 收消息后
1. `On Raw Message`
2. `打印字符串`
## 第 7 步:先验证消息类型
你应该先在屏幕或输出日志里确认这些消息已经进来了:
1. `connection_established`
2. `subscription_confirmed`
3. `data_frame`
其中 `data_frame` 的第一条应是:
- `channel = ue_scene`
- `update_type = full`
## 第 8 步:再接 JSON 解析
当前这一步不建议纯蓝图手搓整个 JSON 结构。
更稳的是:
1. 先在 C++ 里加一个简单解析层
2. 只抽出 `type`
3. 只抽出 `data.update_type`
4. 只抽出 `payload.layers.gpu_clusters.items`
先把 `gpu_clusters` 画出来,再扩到别的层。
## 当前阶段建议
先完成这 3 件事就够了:
1. UE 能成功连接后端
2. UE 能收到 `full` 快照
3. UE 能把原始 JSON 打印出来
做到这一步,我们再继续补:
- JSON 结构体
- `ApplyFullScene`
- `ApplyIncrementalScene`
- `GeoToWorld`
## 一句最重要的提醒
现在先别在 UE 里同时做:
- WebSocket 连接
- 全量 JSON 解析
- 点位生成
- 海缆生成
一步一步来,先确认“连通”,再做“可视化”。这样最不容易卡死。

2
uv.lock generated
View File

@@ -475,7 +475,7 @@ wheels = [
[[package]]
name = "planet"
version = "0.31.3"
version = "0.33.0"
source = { virtual = "." }
dependencies = [
{ name = "aiofiles" },