Compare commits

...

4 Commits

Author SHA1 Message Date
linkong
acbbfdf9e2 release: bump version to 0.69.0
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / delivery (push) Has been cancelled
release / images (push) Has been cancelled
2026-06-03 17:27:00 +08:00
linkong
06aca980d0 release: bump version to 0.68.1
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
release / images (push) Has been cancelled
ci / delivery (push) Has been cancelled
2026-05-28 18:26:15 +08:00
linkong
f3f1ceb833 release: bump version to 0.68.0
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / delivery (push) Has been cancelled
release / images (push) Has been cancelled
2026-05-28 17:10:05 +08:00
rayd1o
b18ffa0b0a release: bump version to 0.67.0
Some checks failed
ci / backend (push) Has been cancelled
ci / frontend (push) Has been cancelled
ci / delivery (push) Has been cancelled
release / images (push) Has been cancelled
2026-05-27 13:50:16 +08:00
95 changed files with 9233 additions and 1069 deletions

6
.gitignore vendored
View File

@@ -28,10 +28,8 @@ dist/
downloads/
eggs/
.eggs/
lib/
!frontend/src/admin/lib/
!frontend/src/admin/lib/**
lib64/
/lib/
/lib64/
parts/
sdist/
var/

15
TODO.md
View File

@@ -11,10 +11,8 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
- [ ] Import authoritative China POV / coastline / claim-line source packages through the three standard Earth boundary source collectors, then rebuild a versioned PMTiles artifact so highest zoom `8-10` preserves trusted source geometry instead of seed data.
- [ ] Earth boundary data: acquire or generate auditable China POV geometry for Zangnan, Aksai Chin, Taiwan/Penghu, Diaoyu Dao and affiliated islands, Chiwei Yu, South China Sea islands, Kosovo, Gaza, and the official dashed maritime claim line before implementing final visual changes.
- [ ] Earth high-resolution basemap tiles: implement the viewport-loaded imagery layer described in [Earth High Resolution Basemap Tiles Plan](/home/ray/dev/linkong/planet/docs/plans/earth-high-resolution-basemap-tiles-plan.md), using high-precision coastline as the alignment reference instead of replacing the globe with one huge texture.
- [ ] Presentation controller ownership: replace the singleton card fallback in [presentation-controller.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/presentation-controller.js) with a presentation/card token check before BGP/News migrate onto the shared controller, so connectors only attach to their owning card.
- [ ] BGP frontend maintainability: split [bgp.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp.js) by responsibility into data loading, marker rendering, overlays, and animation once the current interaction behavior is stable.
- [ ] Optional BGP marker experiment: evaluate HTML markers for BGP incident/collector points if WebGL marker density or fixed screen-size clickability becomes a real blocker.
- [ ] Earth news cruise: connect Earth news to the generic cruise queue via a news adapter rather than coupling news-specific sequencing into `main.js`.
## Compute Centers And Location
@@ -45,13 +43,6 @@ This file is the active backlog only. Completed history belongs in `docs/CHANGEL
- [ ] Compatibility schema: cover adapter type, base URL pattern, auth header, thinking/reasoning defaults, stream path, tool-call capability, multimodal capability, and provider-specific request patches.
- [ ] BGP geography fallback: evaluate `inetnum` / `inet6num` whois as a finer fallback layer after `prefix_geography`, `OpenGeoFeed`, and RIR delegated data.
## Platform
- [ ] Earth preferences scope: keep current device-local Earth preferences in `localStorage`; only design backend user preferences if account-level synchronization becomes a real product requirement.
- [ ] System logs: finish a usable Planet log viewing flow that covers backend, frontend, AI Provider, and collector/task logs, with filtering and tailing.
- [ ] Console UI modernization: gradually replace Ant Design with Planet-owned components and a consistent Tabler Icons based icon system.
- [ ] Earth live sync: design a unified realtime invalidation path for summary/BGP/satellite updates if polling and current WebSocket channels become insufficient.
## Archive
Archived items stay here so old context is not lost. Completed items remain checked; obsolete, invalid, or superseded items stay unchecked and include the reason.
@@ -75,6 +66,11 @@ Archived items stay here so old context is not lost. Completed items remain chec
- [x] Added OpenGeoFeed as a high-quality prefix geography override source.
- [x] Made RIR delegated data a prefix geography fallback rather than the primary source.
- [x] Added route leak and path instability / flap detectors after the activity layer work.
- [x] Console UI modernization. Admin is now the only console, legacy Ant Design / Admin Next code paths and dependencies have been removed, and current console UI uses Planet-owned components.
- [x] Earth news cruise adapter. News cruise now uses `news-cruise-adapter.js` and is wired from `main.js` instead of keeping news-specific sequencing directly in the main Earth loop.
- [x] Presentation controller ownership. `PresentationController` now guards async ownership through active request identity checks, and current callers pass per-request card targets so stale connector/card work cannot overwrite the active presentation.
- [x] Earth live sync. Database writes now flow through `earth_data_change_events`, `earth_db_change_listener`, layer adapters, cache invalidation, and the `earth_updates` WebSocket channel; the Earth frontend debounces updates and refreshes BGP, cables, compute centers, satellites, vessels, news, and interactables by layer.
- [x] System logs. Log sources now normalize into `LogEvent`, Admin supports snapshot filtering plus WebSocket tail/follow, task/detail views deep-link into prefiltered logs, and Admin runtime errors report through the `admin-client` log source.
### Obsolete Or Superseded
@@ -85,3 +81,4 @@ Archived items stay here so old context is not lost. Completed items remain chec
- [ ] Earth surface material overlay for boundary calibration. Superseded by the high-precision boundary tile plan; future work must use source-faithful boundary/coastline data rather than overlay calibration against the coarse base map.
- [ ] Hardcoded Earth news source extraction as a standalone task. Superseded by the broader Earth news source configuration and collector plans.
- [ ] Country-level compute-center fallback placement as a standalone task. Superseded by the shared location pipeline and registry/manual-review backlog.
- [ ] Earth preferences backend sync scope. Superseded by the current product decision to keep Earth preferences device-local in `localStorage` until account-level synchronization becomes a real requirement.

View File

@@ -1 +1 @@
0.66.3
0.69.0

View File

@@ -19,7 +19,6 @@ from app.models.datasource_config import DataSourceConfig
from app.models.task import CollectionTask
from app.models.user import User
from app.models.vessel import AISRawObservation
from app.services.vessel_ais_aggregation import VESSEL_AIS_SCHEMA
from app.services.scheduler import (
sync_datasource_job,
)
@@ -165,6 +164,8 @@ async def _load_latest_tasks(
async def _load_collected_record_counts(
db: AsyncSession,
sources: list[str],
*,
exact_vessel_counts: bool = False,
) -> dict[str, int]:
if not sources:
return {}
@@ -185,14 +186,46 @@ async def _load_collected_record_counts(
or "ais" in source
]
if vessel_sources:
raw_result = await db.execute(
select(AISRawObservation.source, func.count(AISRawObservation.id))
.where(AISRawObservation.target_schema == VESSEL_AIS_SCHEMA)
.where(AISRawObservation.source.in_(vessel_sources))
.group_by(AISRawObservation.source)
if exact_vessel_counts:
exact_result = await db.execute(
select(AISRawObservation.source, func.count(AISRawObservation.id))
.where(AISRawObservation.source.in_(vessel_sources))
.group_by(AISRawObservation.source)
)
for source, count in exact_result.all():
counts[source] = max(counts.get(source, 0), int(count or 0))
return counts
# AIS raw observations can be tens of millions of rows. Use planner
# statistics for the datasource list instead of blocking page load on
# source-level count(*) scans.
stats_result = await db.execute(
text(
"""
SELECT
COALESCE(pg_class.reltuples, 0)::bigint AS total_rows,
pg_stats.most_common_vals::text AS source_values,
pg_stats.most_common_freqs::text AS source_freqs
FROM pg_class
LEFT JOIN pg_stats
ON pg_stats.schemaname = 'public'
AND pg_stats.tablename = 'ais_raw_observations'
AND pg_stats.attname = 'source'
WHERE pg_class.relname = 'ais_raw_observations'
LIMIT 1
"""
)
)
for source, count in raw_result.all():
counts[source] = max(counts.get(source, 0), int(count or 0))
stats = stats_result.mappings().first()
if stats:
total_rows = int(stats["total_rows"] or 0)
values = str(stats["source_values"] or "").strip("{}")
freqs = str(stats["source_freqs"] or "").strip("{}")
source_values = [value.strip('"') for value in values.split(",") if value]
source_freqs = [float(value) for value in freqs.split(",") if value]
for source, freq in zip(source_values, source_freqs):
if source in vessel_sources:
counts[source] = max(counts.get(source, 0), int(round(total_rows * freq)))
return counts
@@ -929,7 +962,7 @@ async def get_datasource_row(
[datasource],
include_endpoint=include_endpoint,
)
record_counts = await _load_collected_record_counts(db, [datasource.source])
record_counts = await _load_collected_record_counts(db, [datasource.source], exact_vessel_counts=True)
return {
"data": serialize_datasource_row(
datasource,

View File

@@ -21,6 +21,12 @@ from app.models.datasource_config import DataSourceConfig
from app.models.system_setting import SystemSetting
from app.models.user import User
from app.services.tv_streams import get_tv_settings_payload
from app.services.earth_news import (
get_earth_news_sources_payload,
reset_earth_news_sources_payload,
save_earth_news_sources_payload,
test_news_source_config,
)
from app.services.earth_boundaries import (
EarthBoundaryBuildError,
get_boundary_build_status,
@@ -100,6 +106,19 @@ class EarthAboutPayload(BaseModel):
meta: list[EarthAboutMetaItem] = Field(default_factory=list)
class EarthNewsSourcesPayload(BaseModel):
cache_version: int | None = None
source_tags: list[dict[str, Any]] = Field(default_factory=list)
categories: list[dict[str, Any]] = Field(default_factory=list)
item_tag_rules: list[dict[str, Any]] = Field(default_factory=list)
sources: list[dict[str, Any]] = Field(default_factory=list)
health: dict[str, Any] = Field(default_factory=dict)
class EarthNewsSourceTestPayload(BaseModel):
source: dict[str, Any] = Field(default_factory=dict)
def _normalize_earth_brand_payload(payload: dict[str, Any] | None) -> dict[str, str]:
merged = DEFAULT_EARTH_BRAND.copy()
if payload:
@@ -324,6 +343,38 @@ async def reset_earth_about(
return {"status": "reset", "about": _normalize_earth_about_payload(None), "is_default": True}
@router.get("/news-sources")
async def get_earth_news_sources(db: AsyncSession = Depends(get_db)):
return await get_earth_news_sources_payload(db)
@router.put("/news-sources")
async def update_earth_news_sources(
payload: EarthNewsSourcesPayload,
_current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
return await save_earth_news_sources_payload(db, payload.model_dump())
@router.delete("/news-sources")
@router.post("/news-sources/reset")
async def reset_earth_news_sources(
_current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
return await reset_earth_news_sources_payload(db)
@router.post("/news-sources/test")
async def test_earth_news_source(
payload: EarthNewsSourceTestPayload,
_current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
return await test_news_source_config(payload.source, db=db)
@router.get("/oobe-status")
async def get_earth_oobe_status(
current_user: User | None = Depends(_get_optional_current_user),

View File

@@ -90,7 +90,7 @@ async def get_interactables_geojson(
return interactables_to_geojson(items)
payload = await get_or_build_layer_payload(
key=earth_layer_cache.key("interactables", layer=layer or "all"),
key=earth_layer_cache.key("interactables", interactable_layer=layer or "all"),
policy=INTERACTABLE_CACHE_POLICY,
builder=build_payload,
response=response,

View File

@@ -1,16 +1,92 @@
from fastapi import APIRouter, Depends, Query
from fastapi import APIRouter, Depends, HTTPException, Query
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.session import get_db
from app.services.earth_news import get_earth_news_payload
from app.services.earth_news import (
ALLOWED_NEWS_CATEGORY_KEYS,
SUPPORTED_NEWS_LOCALES,
REGION_ANCHORS,
get_earth_news_payload,
)
router = APIRouter()
def _parse_categories(raw: str | None) -> set[str] | None:
if raw is None or not raw.strip():
return None
requested = {item.strip().lower() for item in raw.split(",") if item.strip()}
invalid = sorted(requested - set(ALLOWED_NEWS_CATEGORY_KEYS))
if invalid:
raise HTTPException(
status_code=422,
detail={
"message": "Unsupported news categories.",
"invalid_categories": invalid,
"allowed_categories": list(ALLOWED_NEWS_CATEGORY_KEYS),
},
)
return requested or None
def _parse_source_ids(raw: str | None) -> set[str] | None:
if raw is None or not raw.strip():
return None
return {item.strip() for item in raw.split(",") if item.strip()} or None
def _parse_limit(raw: int | None) -> int:
if raw is None:
return 12
if raw < 1:
raise HTTPException(status_code=422, detail={"message": "News limit must be greater than 0."})
return min(raw, 100)
def _parse_locale(raw: str | None) -> str:
if raw is None or not raw.strip():
return "zh-CN"
requested = raw.strip()
if requested not in SUPPORTED_NEWS_LOCALES:
raise HTTPException(
status_code=422,
detail={
"message": "Unsupported news locale.",
"invalid_locale": requested,
"allowed_locales": sorted(SUPPORTED_NEWS_LOCALES),
},
)
return requested
@router.get("/earth-feed")
async def get_earth_feed(
lat: float | None = Query(None, description="Current Earth view center latitude"),
lon: float | None = Query(None, description="Current Earth view center longitude"),
region: str | None = Query(None, description="Explicit Earth news region for UE/client integrations"),
categories: str | None = Query(None, description="Comma-separated news category keys"),
sources: str | None = Query(None, description="Comma-separated news source ids"),
limit: int | None = Query(None, description="Maximum news items to return, capped at 100"),
locale: str | None = Query(None, description="Display locale, zh-CN or en-US"),
db: AsyncSession = Depends(get_db),
):
return await get_earth_news_payload(lat=lat, lon=lon, db=db)
normalized_region = region.strip().lower() if isinstance(region, str) and region.strip() else None
if normalized_region is not None and normalized_region not in REGION_ANCHORS:
raise HTTPException(
status_code=422,
detail={
"message": "Unsupported news region.",
"invalid_region": normalized_region,
"allowed_regions": list(REGION_ANCHORS.keys()),
},
)
return await get_earth_news_payload(
lat=lat,
lon=lon,
region=normalized_region,
categories=_parse_categories(categories),
source_ids=_parse_source_ids(sources),
limit=_parse_limit(limit),
locale=_parse_locale(locale),
db=db,
)

View File

@@ -1,21 +1,19 @@
from __future__ import annotations
import os
import json
import secrets
import subprocess
import sys
from datetime import datetime
from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
from fastapi import APIRouter, Depends, Header, HTTPException, Query, Request, status
from pydantic import BaseModel
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import ROOT_DIR
from app.core.config import ROOT_DIR, settings
from app.core.security import get_current_user
from app.db.session import get_db
from app.models.system_log import AuditLog, SystemLog
from app.models.user import User
from app.services.persistent_logs import record_audit_log, record_system_log
from app.services.system_control import (
@@ -38,49 +36,17 @@ from app.services.system_logs import (
append_buffer_log,
list_log_sources,
normalize_log_level,
read_database_log_snapshot,
read_log_snapshot,
read_observability_group_events,
read_observability_groups,
read_observability_raw_events,
)
from app.services.earth_layer_cache import earth_layer_cache
router = APIRouter()
def _compact_log_context(context: dict | None) -> str:
if not context:
return ""
allowed = {
key: value
for key, value in (context or {}).items()
if key
in {
"status",
"duration_ms",
"provider",
"model",
"result_provider",
"result_model",
"collector_name",
"datasource_id",
"task_id",
"snapshot_id",
"raw_count",
"transformed_count",
"saved_count",
"created",
"updated",
"unchanged",
"deleted",
"result_count",
"status_code",
"error_type",
"error",
}
}
if not allowed:
return ""
return json.dumps(allowed, ensure_ascii=False, sort_keys=True)
class RestartTaskCreate(BaseModel):
action: str
@@ -146,12 +112,104 @@ class EarthClientLogEventCreate(BaseModel):
url: str | None = None
module: str | None = None
detail: str | None = None
fingerprint: str | None = None
occurrence_count: int = 1
metadata: dict[str, object] | None = None
class EarthClientLogEventResponse(BaseModel):
accepted: bool
source_id: str
level: str
fingerprint: str | None = None
class ServiceLogEventCreate(BaseModel):
source: str = "ai-provider"
service: str = "ai-provider"
module: str | None = None
category: str | None = None
event: str = "service.runtime_log"
level: str = "error"
message: str
fingerprint: str | None = None
occurrence_count: int = 1
request_id: str | None = None
trace_id: str | None = None
task_id: str | None = None
source_id: int | str | None = None
provider: str | None = None
context: dict[str, object] | None = None
async def ingest_client_log_event(
source_id: str,
*,
service: str,
event: str,
default_module: str,
default_category: str,
payload: EarthClientLogEventCreate,
request: Request,
) -> EarthClientLogEventResponse:
normalized_level = normalize_log_level(payload.level)
append_buffer_log(
source_id,
level=normalized_level,
message=payload.message,
context={
"category": payload.category or "",
"url": payload.url or "",
"module": payload.module or "",
"detail": payload.detail or "",
"fingerprint": payload.fingerprint or "",
"occurrence_count": max(1, int(payload.occurrence_count or 1)),
"metadata": payload.metadata or {},
},
)
await record_system_log(
source=source_id,
service=service,
module=payload.module or default_module,
event=event,
level=normalized_level,
message=payload.message,
category=payload.category or default_category,
context={
"url": payload.url or "",
"detail": payload.detail or "",
"module": payload.module or "",
"client_ip": request.client.host if request.client else "",
"metadata": payload.metadata or {},
},
fingerprint=payload.fingerprint,
occurrence_count=max(1, int(payload.occurrence_count or 1)),
)
return EarthClientLogEventResponse(accepted=True, source_id=source_id, level=normalized_level, fingerprint=payload.fingerprint)
def require_observability_ingest_token(
authorization: str | None,
ingest_token: str | None,
) -> None:
expected_token = settings.OBSERVABILITY_INGEST_TOKEN.strip()
if not expected_token:
raise HTTPException(
status_code=status.HTTP_503_SERVICE_UNAVAILABLE,
detail="Observability service ingestion is not configured",
)
provided = ""
if ingest_token:
provided = ingest_token.strip()
elif authorization:
scheme, _, token = authorization.partition(" ")
if scheme.lower() == "bearer":
provided = token.strip()
if not provided or not secrets.compare_digest(provided, expected_token):
raise HTTPException(
status_code=status.HTTP_403_FORBIDDEN,
detail="Invalid observability ingestion token",
)
class EarthLayerCacheStatusResponse(BaseModel):
@@ -376,98 +434,116 @@ async def get_system_log_sources(
}
async def read_database_log_snapshot(
source_id: str,
*,
limit: int,
level: str,
levels: str | None,
start_date: str | None,
end_date: str | None,
search: str | None,
db: AsyncSession,
) -> dict | None:
selected_levels = set(normalize_log_level(item) for item in (levels or level).split(",") if item.strip())
selected_levels.discard("all")
search_query = (search or "").strip().lower()
lines: list[str] = []
@router.get("/logs/observability/groups")
async def get_observability_log_groups(
limit: int = DEFAULT_LOG_LINE_LIMIT,
level: str = "all",
levels: str | None = Query(None, description="Comma-separated log levels"),
start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
search: str | None = Query(None, description="Case-insensitive substring search"),
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
ensure_super_admin(current_user)
if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
normalized_start_date = validate_log_date(start_date, "start_date")
normalized_end_date = validate_log_date(end_date, "end_date")
if normalized_start_date and normalized_end_date and normalized_start_date > normalized_end_date:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail="start_date must be earlier than or equal to end_date")
return await read_observability_groups(
limit=limit,
level=level,
levels=levels,
start_date=normalized_start_date,
end_date=normalized_end_date,
search=search,
db=db,
)
if source_id == "system-db":
query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(limit * 5)
result = await db.execute(query)
records = result.scalars().all()
for record in records:
record_level = normalize_log_level(record.level)
if selected_levels and record_level not in selected_levels:
continue
occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
if start_date and occurred_at and occurred_at < start_date:
continue
if end_date and occurred_at and occurred_at > end_date:
continue
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
record_level.upper(),
record.source,
record.category or "",
record.event or "",
f"request_id={record.request_id}" if record.request_id else "",
record.message,
_compact_log_context(record.context),
]
if part
)
if search_query and search_query not in line.lower():
continue
lines.append(line)
elif source_id == "audit-db":
query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(limit * 5)
result = await db.execute(query)
records = result.scalars().all()
for record in records:
occurred_at = record.occurred_at.date().isoformat() if record.occurred_at else ""
if start_date and occurred_at and occurred_at < start_date:
continue
if end_date and occurred_at and occurred_at > end_date:
continue
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
"INFO",
record.action,
record.target_type or "",
record.target_id or "",
record.result or "",
]
if part
)
if search_query and search_query not in line.lower():
continue
lines.append(line)
else:
return None
lines = list(reversed(lines[:limit]))
return {
"source_id": source_id,
"name": "系统事件" if source_id == "system-db" else "审计事件",
"kind": "database",
"location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
"description": "数据库持久化日志",
"category": "database" if source_id == "system-db" else "audit",
"status": "ok" if lines else "empty",
"level": level,
"selected_levels": sorted(selected_levels),
"search_query": search or "",
"available_levels": ["all", "error", "warning", "info", "debug"],
"daily_markers": [],
"line_limit": limit,
"line_count": len(lines),
"lines": lines,
}
@router.get("/logs/observability/groups/{fingerprint}/events")
async def get_observability_group_events(
fingerprint: str,
limit: int = DEFAULT_LOG_LINE_LIMIT,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
ensure_super_admin(current_user)
if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
payload = await read_observability_group_events(fingerprint, limit=limit, db=db)
if payload is None:
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Observability group not found")
return payload
@router.get("/logs/observability/raw")
async def get_observability_raw_events(
limit: int = DEFAULT_LOG_LINE_LIMIT,
level: str = "all",
levels: str | None = Query(None, description="Comma-separated log levels"),
start_date: str | None = Query(None, description="Filter logs from this date (YYYY-MM-DD)"),
end_date: str | None = Query(None, description="Filter logs until this date (YYYY-MM-DD)"),
search: str | None = Query(None, description="Case-insensitive substring search"),
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
ensure_super_admin(current_user)
if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
raise HTTPException(status_code=status.HTTP_400_BAD_REQUEST, detail=f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
normalized_start_date = validate_log_date(start_date, "start_date")
normalized_end_date = validate_log_date(end_date, "end_date")
return await read_observability_raw_events(
limit=limit,
level=level,
levels=levels,
start_date=normalized_start_date,
end_date=normalized_end_date,
search=search,
db=db,
)
@router.post("/logs/service", response_model=EarthClientLogEventResponse)
async def ingest_service_log(
payload: ServiceLogEventCreate,
authorization: str | None = Header(default=None),
ingest_token: str | None = Header(default=None, alias="X-Planet-Observability-Token"),
):
require_observability_ingest_token(authorization, ingest_token)
normalized_level = normalize_log_level(payload.level)
source = (payload.source or "ai-provider").strip() or "ai-provider"
context = dict(payload.context or {})
if payload.request_id:
context["request_id"] = payload.request_id
if payload.trace_id:
context["trace_id"] = payload.trace_id
if payload.task_id:
context["task_id"] = payload.task_id
if payload.source_id is not None:
context["source_id"] = payload.source_id
if payload.provider:
context["provider"] = payload.provider
await record_system_log(
source=source,
service=(payload.service or source).strip() or source,
module=payload.module or source,
event=(payload.event or "service.runtime_log").strip() or "service.runtime_log",
level=normalized_level,
message=payload.message,
category=payload.category or "service-runtime",
context=context,
fingerprint=payload.fingerprint,
occurrence_count=max(1, int(payload.occurrence_count or 1)),
)
return EarthClientLogEventResponse(
accepted=True,
source_id=source,
level=normalized_level,
fingerprint=payload.fingerprint,
)
@router.get("/logs/{source_id}", response_model=SystemLogSnapshotResponse)
@@ -533,31 +609,28 @@ async def ingest_earth_client_log(
payload: EarthClientLogEventCreate,
request: Request,
):
normalized_level = normalize_log_level(payload.level)
append_buffer_log(
return await ingest_client_log_event(
"earth-client",
level=normalized_level,
message=payload.message,
context={
"category": payload.category or "",
"url": payload.url or "",
"module": payload.module or "",
"detail": payload.detail or "",
},
)
await record_system_log(
source="earth-client",
service="earth",
module=payload.module or "earth-client",
event="earth.client.runtime_log",
level=normalized_level,
message=payload.message,
category=payload.category or "client-runtime",
context={
"url": payload.url or "",
"detail": payload.detail or "",
"module": payload.module or "",
"client_ip": request.client.host if request.client else "",
},
default_module="earth-client",
default_category="client-runtime",
payload=payload,
request=request,
)
@router.post("/logs/admin-client", response_model=EarthClientLogEventResponse)
async def ingest_admin_client_log(
payload: EarthClientLogEventCreate,
request: Request,
):
return await ingest_client_log_event(
"admin-client",
service="admin",
event="admin.client.runtime_log",
default_module="admin-client",
default_category="client-runtime",
payload=payload,
request=request,
)
return {"accepted": True, "source_id": "earth-client", "level": normalized_level}

View File

@@ -29,7 +29,8 @@ async def list_tasks(
SELECT ct.id, ct.datasource_id, ds.name as datasource_name, ct.status,
ct.started_at, ct.completed_at, ct.records_processed, ct.error_message,
ct.phase, ct.phase_progress, ct.phase_message, ct.phase_current,
ct.phase_total, ct.phase_unit, ct.total_records, ct.progress
ct.phase_total, ct.phase_unit, ct.total_records, ct.progress,
ct.task_type, ct.source, ds.source as datasource_source
FROM collection_tasks ct
JOIN data_sources ds ON ct.datasource_id = ds.id
WHERE 1=1
@@ -39,12 +40,19 @@ async def list_tasks(
if datasource_id:
query += " AND ct.datasource_id = :datasource_id"
count_query += " WHERE ct.datasource_id = :datasource_id"
count_query += " AND ct.datasource_id = :datasource_id"
params["datasource_id"] = datasource_id
if status:
query += " AND ct.status = :status"
count_query += " AND ct.status = :status"
params["status"] = status
statuses = [item.strip() for item in status.split(",") if item.strip()]
if len(statuses) > 1:
placeholders = ", ".join(f":status_{index}" for index, _item in enumerate(statuses))
query += f" AND ct.status IN ({placeholders})"
count_query += f" AND ct.status IN ({placeholders})"
params.update({f"status_{index}": item for index, item in enumerate(statuses)})
else:
query += " AND ct.status = :status"
count_query += " AND ct.status = :status"
params["status"] = statuses[0] if statuses else status
query += f" ORDER BY ct.created_at DESC LIMIT {page_size} OFFSET {offset}"
@@ -76,6 +84,9 @@ async def list_tasks(
"phase_unit": t[13],
"total_records": t[14],
"progress": t[15],
"task_type": t[16],
"source": t[17] or t[18],
"datasource_source": t[18],
}
for t in tasks
],

View File

@@ -1,3 +1,4 @@
import re
from urllib.parse import quote, urljoin
import httpx
@@ -10,6 +11,26 @@ from app.services.tv_streams import get_public_tv_payload, is_allowed_tv_proxy_u
router = APIRouter()
_HLS_URI_ATTRIBUTE_RE = re.compile(r'URI="([^"]+)"')
def _proxied_tv_url(url: str) -> str:
return f"/api/v1/tv/proxy?url={quote(url, safe='')}"
def _rewrite_hls_uri_attributes(line: str, *, base_url: str) -> str:
def replace(match: re.Match[str]) -> str:
uri = match.group(1)
absolute_url = urljoin(base_url, uri)
return f'URI="{_proxied_tv_url(absolute_url)}"'
return _HLS_URI_ATTRIBUTE_RE.sub(replace, line)
def _should_strip_hls_metadata_line(line: str) -> bool:
normalized = line.strip().upper()
return normalized.startswith("#EXT-X-MEDIA:") and "TYPE=SUBTITLES" in normalized
@router.get("/streams")
async def list_public_tv_streams(
@@ -56,11 +77,16 @@ async def proxy_tv_stream(
rewritten_lines: list[str] = []
for line in manifest_text.splitlines():
stripped = line.strip()
if not stripped or stripped.startswith("#"):
if not stripped:
rewritten_lines.append(line)
continue
if stripped.startswith("#"):
if _should_strip_hls_metadata_line(line):
continue
rewritten_lines.append(_rewrite_hls_uri_attributes(line, base_url=response_url))
continue
absolute_url = urljoin(response_url, stripped)
rewritten_lines.append(f"/api/v1/tv/proxy?url={quote(absolute_url, safe='')}")
rewritten_lines.append(_proxied_tv_url(absolute_url))
return Response(
content="\n".join(rewritten_lines),
media_type="application/vnd.apple.mpegurl",

View File

@@ -6,11 +6,14 @@ from typing import Optional
from fastapi import APIRouter, WebSocket, WebSocketDisconnect, Query
from jose import jwt, JWTError
from sqlalchemy import text
from app.core.config import settings
from app.core.logging import get_logger
from app.core.time import to_iso8601_utc
from app.core.websocket.manager import manager
from app.db.session import async_session_factory
from app.services.log_tail import LOG_TAIL_CHANNEL, log_tail_manager
logger = get_logger(__name__, service="api")
router = APIRouter()
@@ -37,6 +40,28 @@ async def authenticate_token(token: str) -> Optional[dict]:
return None
async def load_websocket_user_role(user_id: str | None) -> str | None:
if not user_id:
return None
try:
async with async_session_factory() as db:
result = await db.execute(
text("SELECT role, is_active FROM users WHERE id = :id"),
{"id": int(user_id)},
)
row = result.fetchone()
except Exception as exc:
logger.warning_event(
"WebSocket user role lookup failed",
event="auth.websocket.role_lookup_failed",
context={"user_id": user_id, "error": str(exc)},
)
return None
if row is None or not row[1]:
return None
return str(row[0] or "")
@router.websocket("/ws")
async def websocket_endpoint(
websocket: WebSocket,
@@ -59,6 +84,7 @@ async def websocket_endpoint(
is_anonymous = payload is None
user_id = str(payload.get("sub")) if payload else f"anonymous:{id(websocket)}"
user_role = await load_websocket_user_role(user_id) if payload else None
supported_channels = ["vessels", "earth_news", EARTH_UPDATES_CHANNEL] if is_anonymous else [
"gpu_clusters",
"submarine_cables",
@@ -70,6 +96,8 @@ async def websocket_endpoint(
"earth_news",
EARTH_UPDATES_CHANNEL,
]
if user_role == "super_admin":
supported_channels = [*supported_channels, LOG_TAIL_CHANNEL]
await manager.connect(websocket, user_id)
try:
@@ -100,6 +128,7 @@ async def websocket_endpoint(
payload_data = data.get("data", {})
if not isinstance(payload_data, dict):
payload_data = {}
log_tail_config = None
channels = payload_data.get("channels", [])
if isinstance(channels, str):
channels = [channels]
@@ -108,6 +137,26 @@ async def websocket_endpoint(
channel = payload_data.get("channel")
if channel and channel not in channels:
channels = [*channels, channel]
if LOG_TAIL_CHANNEL in channels:
if user_role != "super_admin":
await websocket.send_json(
{
"type": "subscription_error",
"data": {"channel": LOG_TAIL_CHANNEL, "detail": "Only super_admin can subscribe logs"},
}
)
channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
else:
try:
log_tail_config = await log_tail_manager.subscribe(websocket, payload_data)
except ValueError as exc:
await websocket.send_json(
{
"type": "subscription_error",
"data": {"channel": LOG_TAIL_CHANNEL, "detail": str(exc)},
}
)
channels = [item for item in channels if item != LOG_TAIL_CHANNEL]
if is_anonymous:
channels = [channel for channel in channels if channel in supported_channels]
vessel_subscription = None
@@ -131,14 +180,20 @@ async def websocket_endpoint(
"action": "subscribe",
"channels": [
*channels,
*([LOG_TAIL_CHANNEL] if log_tail_config else []),
*(["vessels"] if vessel_subscription else []),
],
"vessels": vessel_subscription,
"logs_tail": log_tail_config.__dict__ if log_tail_config else None,
},
}
)
elif data.get("type") == "unsubscribe":
channels = data.get("data", {}).get("channels", [])
if isinstance(channels, str):
channels = [channels]
if LOG_TAIL_CHANNEL in channels:
await log_tail_manager.unsubscribe(websocket)
manager.unsubscribe(websocket, channels)
await websocket.send_json(
{
@@ -159,4 +214,5 @@ async def websocket_endpoint(
except WebSocketDisconnect:
pass
finally:
await log_tail_manager.disconnect(websocket)
manager.disconnect(websocket, user_id)

View File

@@ -41,6 +41,7 @@ class Settings(BaseSettings):
AI_PROVIDER_SERVICE_TOKEN: str = ""
AI_PROVIDER_TIMEOUT_SECONDS: int = 60
AI_PROVIDER_RETRY_ATTEMPTS: int = 2
OBSERVABILITY_INGEST_TOKEN: str = ""
@property
def REDIS_URL(self) -> str:

View File

@@ -13,7 +13,7 @@ from app.models.compute_center_location import ComputeCenterLocationRecord
from app.models.system_setting import SystemSetting
from app.models.playground_session import PlaygroundSession
from app.models.playground_message import PlaygroundMessage
from app.models.system_log import SystemLog, AuditLog
from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth, VesselPosition, VesselStatic
from app.models.datasource_mapping import DataSourceMappingTemplate
from app.models.earth_news import EarthNewsItem
@@ -37,6 +37,8 @@ __all__ = [
"ComputeCenterLocationRecord",
"SystemLog",
"AuditLog",
"ObservabilityEvent",
"ObservabilityEventGroup",
"PlaygroundSession",
"PlaygroundMessage",
"VesselPosition",

View File

@@ -38,3 +38,46 @@ class AuditLog(Base):
ip = Column(String(64), nullable=True)
details = Column(JSON, nullable=False, default=dict)
created_at = Column(DateTime(timezone=True), server_default=func.now())
class ObservabilityEvent(Base):
__tablename__ = "observability_events"
id = Column(Integer, primary_key=True, autoincrement=True)
occurred_at = Column(DateTime(timezone=True), server_default=func.now(), index=True)
source = Column(String(50), nullable=False, index=True)
service = Column(String(50), nullable=True, index=True)
module = Column(String(120), nullable=True, index=True)
category = Column(String(80), nullable=True, index=True)
event = Column(String(160), nullable=True, index=True)
level = Column(String(20), nullable=False, index=True)
message = Column(Text, nullable=False)
fingerprint = Column(String(80), nullable=False, index=True)
request_id = Column(String(64), nullable=True, index=True)
trace_id = Column(String(64), nullable=True, index=True)
task_id = Column(String(120), nullable=True, index=True)
source_ref_id = Column(String(120), nullable=True, index=True)
provider = Column(String(120), nullable=True, index=True)
user_id = Column(Integer, nullable=True, index=True)
context = Column(JSON, nullable=False, default=dict)
occurrence_count = Column(Integer, nullable=False, default=1)
created_at = Column(DateTime(timezone=True), server_default=func.now())
class ObservabilityEventGroup(Base):
__tablename__ = "observability_event_groups"
fingerprint = Column(String(80), primary_key=True)
source = Column(String(50), nullable=False, index=True)
service = Column(String(50), nullable=True, index=True)
module = Column(String(120), nullable=True, index=True)
category = Column(String(80), nullable=True, index=True)
event = Column(String(160), nullable=True, index=True)
last_level = Column(String(20), nullable=False, index=True)
sample_message = Column(Text, nullable=False)
sample_detail = Column(Text, nullable=True)
affected_sources = Column(JSON, nullable=False, default=list)
count = Column(Integer, nullable=False, default=0)
first_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
last_seen_at = Column(DateTime(timezone=True), nullable=False, index=True)
updated_at = Column(DateTime(timezone=True), server_default=func.now(), onupdate=func.now())

View File

@@ -322,6 +322,21 @@ class AISStreamCollector(BaseCollector):
last_success_at=now if data else None,
lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
)
if snapshot_id is not None:
from app.models.data_snapshot import DataSnapshot
snapshot = await db.get(DataSnapshot, snapshot_id)
if snapshot:
snapshot.record_count = records_added
snapshot.status = "success"
snapshot.completed_at = now
snapshot.summary = {
"created": records_added,
"updated": 0,
"unchanged": 0,
"deleted": 0,
"storage": "ais_raw_observations",
}
await db.commit()
await self.update_progress(records_added, force=True)
return records_added

View File

@@ -25,11 +25,17 @@ FALLBACK_GROUPS = (
"starlink",
"gps-ops",
"galileo",
"glonass",
"glo-ops",
"beidou",
"leo",
"geo",
"iridium-next",
"stations",
"visual",
"weather",
"science",
"cubesat",
"amateur",
"last-30-days",
)
FETCH_RETRY_ATTEMPTS = 3
FETCH_RETRY_BASE_DELAY_SECONDS = 0.8
@@ -220,27 +226,28 @@ class CelesTrakTLECollector(BaseCollector):
try:
for group in FALLBACK_GROUPS:
group_url = self._group_url(group)
try:
body_path = await self._downloader.download_file(
client,
group_url,
extension=".json",
accept="application/json",
validate_existing=self._validate_json_file,
)
except DownloadHTTPStatusError as exc:
if not self._is_not_updated_response(exc):
raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
cached_path = self._downloader.get_cached_file(
group_url,
".json",
validate_existing=self._validate_json_file,
)
if cached_path is None:
cached_path = self._downloader.get_cached_file(
group_url,
".json",
validate_existing=self._validate_json_file,
)
if cached_path is not None:
body_path = cached_path
else:
try:
body_path = await self._downloader.download_file(
client,
group_url,
extension=".json",
accept="application/json",
validate_existing=self._validate_json_file,
)
except DownloadHTTPStatusError as exc:
if not self._is_not_updated_response(exc):
raise RuntimeError(f"CelesTrak fallback group '{group}' download failed: {exc}") from exc
raise RuntimeError(
f"CelesTrak fallback group '{group}' has not updated and no local cached copy is available"
) from exc
body_path = cached_path
group_records = await self._load_downloaded_payload(
body_path,

View File

@@ -119,6 +119,21 @@ class VesselAISCollector(BaseCollector):
last_success_at=now if data else None,
lag_seconds=max((now - latest_observed_at).total_seconds(), 0),
)
if snapshot_id is not None:
from app.models.data_snapshot import DataSnapshot
snapshot = await db.get(DataSnapshot, snapshot_id)
if snapshot:
snapshot.record_count = records_added
snapshot.status = "success"
snapshot.completed_at = now
snapshot.summary = {
"created": records_added,
"updated": 0,
"unchanged": 0,
"deleted": 0,
"storage": "ais_raw_observations",
}
await db.commit()
await self._broadcast_vessel_snapshot(data)
await self.update_progress(records_added, force=True)

View File

@@ -54,6 +54,9 @@ DATA_WRITE_JOB_TYPES = (JOB_TYPE_COLLECT, JOB_TYPE_CLEAR_DATA, JOB_TYPE_CLEAR_CA
SOURCE_LOCK_JOB_STATUSES = (JOB_STATUS_RUNNING, JOB_STATUS_CANCELLING)
QUEUE_POLL_SECONDS = 0.35
JOB_STALE_LOCK_MINUTES = 90
ORPHAN_CANCELLING_GRACE_SECONDS = 30
JOB_RECOVERY_SWEEP_SECONDS = 15
DATA_DELETE_BATCH_SIZE = 50_000
DEFAULT_WORKER_CONCURRENCY = 2
RUNNING_DATA_JOB_TASKS: dict[int, asyncio.Task[Any]] = {}
@@ -281,6 +284,7 @@ class DataJobWorker:
self._task: asyncio.Task[None] | None = None
self._stop_event: asyncio.Event | None = None
self._running: set[asyncio.Task[Any]] = set()
self._last_recovery_sweep_at: datetime | None = None
def start(self) -> None:
if self._task and not self._task.done():
@@ -303,6 +307,11 @@ class DataJobWorker:
await self._recover_stale_running_jobs()
while not self._stop_event.is_set():
self._running = {task for task in self._running if not task.done()}
if (
self._last_recovery_sweep_at is None
or (_utcnow() - self._last_recovery_sweep_at).total_seconds() >= JOB_RECOVERY_SWEEP_SECONDS
):
await self._recover_stale_running_jobs()
if len(self._running) >= self.concurrency:
await asyncio.sleep(QUEUE_POLL_SECONDS)
continue
@@ -316,7 +325,9 @@ class DataJobWorker:
self._running.add(runner)
async def _recover_stale_running_jobs(self) -> None:
self._last_recovery_sweep_at = _utcnow()
cutoff = _utcnow() - timedelta(minutes=JOB_STALE_LOCK_MINUTES)
orphan_cancelling_cutoff = _utcnow() - timedelta(seconds=ORPHAN_CANCELLING_GRACE_SECONDS)
async with async_session_factory() as db:
result = await db.execute(
select(CollectionTask)
@@ -332,6 +343,21 @@ class DataJobWorker:
job.error_message = "Marked failed after stale data job lock timeout"
if stale_jobs:
await db.commit()
orphan_result = await db.execute(
select(CollectionTask)
.where(CollectionTask.status == JOB_STATUS_CANCELLING)
.where(CollectionTask.locked_at.is_(None))
.where(CollectionTask.requested_cancel_at.is_not(None))
.where(CollectionTask.requested_cancel_at < orphan_cancelling_cutoff)
)
for job in orphan_result.scalars().all():
if job.id in RUNNING_DATA_JOB_TASKS:
continue
await _cancel_task_without_runner(
db,
job,
reason=job.cancel_reason or "cancelled_after_orphaned_runner",
)
async def _claim_next_job(self) -> int | None:
async with async_session_factory() as db:
@@ -477,15 +503,22 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
await db.commit()
await _broadcast_task_update(task)
count_result = await db.execute(
select(CollectedData.id).where(CollectedData.source == source)
deleted_count = await _delete_table_rows_by_source(
db,
task,
table_name="collected_data",
source_column="source",
source=source,
)
derived_deleted_counts = await _clear_derived_datasource_data_in_batches(
db,
task,
source,
progress_offset=deleted_count,
)
collected_ids = [row[0] for row in count_result.all()]
derived_deleted_counts = await clear_derived_datasource_data(db, source)
if collected_ids:
await db.execute(CollectedData.__table__.delete().where(CollectedData.id.in_(collected_ids)))
deleted_count = len(collected_ids)
derived_deleted_count = sum(derived_deleted_counts.values())
if any(key.startswith("ais_") for key in derived_deleted_counts):
await db.execute(text("ANALYZE ais_raw_observations"))
task.records_processed = deleted_count + derived_deleted_count
task.total_records = task.records_processed
@@ -504,10 +537,99 @@ async def _run_clear_data_job(db: AsyncSession, task: CollectionTask) -> None:
task.phase = "completed"
task.phase_message = "数据库数据已清理"
task.completed_at = _utcnow()
datasource = await db.get(DataSource, task.datasource_id)
if datasource is not None:
datasource.last_status = JOB_STATUS_SUCCESS
datasource.last_run_at = task.completed_at
await db.execute(
DataSnapshot.__table__.update()
.where(DataSnapshot.source == source)
.values(is_current=False)
)
await db.commit()
await _broadcast_task_update(task)
async def _delete_table_rows_by_source(
db: AsyncSession,
task: CollectionTask,
*,
table_name: str,
source_column: str,
source: str,
progress_offset: int = 0,
) -> int:
deleted = 0
while True:
result = await db.execute(
text(
f"""
WITH doomed AS (
SELECT ctid
FROM {table_name}
WHERE {source_column} = :source
LIMIT :batch_size
),
deleted_rows AS (
DELETE FROM {table_name}
USING doomed
WHERE {table_name}.ctid = doomed.ctid
RETURNING 1
)
SELECT COUNT(*) FROM deleted_rows
"""
),
{"source": source, "batch_size": DATA_DELETE_BATCH_SIZE},
)
batch_deleted = max(int(result.scalar_one() or 0), 0)
if batch_deleted <= 0:
break
deleted += batch_deleted
task.records_processed = progress_offset + deleted
task.phase_current = task.records_processed
task.phase_unit = "records"
task.phase_message = f"正在删除数据:{task.records_processed}"
await db.commit()
await _broadcast_task_update(task)
return deleted
async def _clear_derived_datasource_data_in_batches(
db: AsyncSession,
task: CollectionTask,
source: str,
progress_offset: int = 0,
) -> dict[str, int]:
deleted_counts: dict[str, int] = {}
if source in {"barentswatch_vessels", "aisstream_vessels"}:
deleted_counts["ais_conflict_records"] = await _delete_table_rows_by_source(
db,
task,
table_name="ais_conflict_records",
source_column="selected_source",
source=source,
progress_offset=progress_offset + sum(deleted_counts.values()),
)
deleted_counts["ais_source_health"] = await _delete_table_rows_by_source(
db,
task,
table_name="ais_source_health",
source_column="source",
source=source,
progress_offset=progress_offset + sum(deleted_counts.values()),
)
deleted_counts["ais_raw_observations"] = await _delete_table_rows_by_source(
db,
task,
table_name="ais_raw_observations",
source_column="source",
source=source,
progress_offset=progress_offset + sum(deleted_counts.values()),
)
return deleted_counts
return await clear_derived_datasource_data(db, source)
async def _run_clear_cache_job(db: AsyncSession, task: CollectionTask) -> None:
source = str(task.source or (task.payload or {}).get("source") or "").strip()
if not source:

View File

@@ -62,11 +62,11 @@ def interactables_to_geojson(items: list[EarthInteractable]) -> dict[str, Any]:
def invalidate_interactable_cache(layer: str | None = None) -> int:
layer_key = str(layer or "*").strip() or "*"
deleted = earth_layer_cache.delete_pattern(
f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:{layer_key}*"
f"{EARTH_LAYER_CACHE_PREFIX}:interactables:interactable_layer:{layer_key}*"
)
if layer_key != "all":
deleted += earth_layer_cache.delete_pattern(
f"{EARTH_LAYER_CACHE_PREFIX}:interactables:layer:all*"
f"{EARTH_LAYER_CACHE_PREFIX}:interactables:interactable_layer:all*"
)
return deleted

View File

@@ -24,6 +24,7 @@ EARTH_LAYER_ADAPTERS: tuple[EarthLayerAdapter, ...] = (
tables=frozenset({"vessel_static", "vessel_position", "ais_raw_observations", "ais_source_health"}),
layers=("vessels",),
cache_patterns=("vessels*", "summary*"),
derived_models=("ais_raw_observations", "ais_conflict_records", "ais_source_health"),
),
EarthLayerAdapter(
sources=frozenset(
@@ -163,17 +164,24 @@ async def clear_derived_datasource_data(db: AsyncSession, source: str) -> dict[s
from app.models.bgp_anomaly import BGPAnomaly
from app.models.bgp_incident import BGPIncident
from app.models.bgp_observation import BGPObservation
from app.models.vessel import AISConflictRecord, AISRawObservation, AISSourceHealth
model_by_key: dict[str, Any] = {
"bgp_observations": BGPObservation,
"bgp_anomalies": BGPAnomaly,
"bgp_incidents": BGPIncident,
"ais_raw_observations": AISRawObservation,
"ais_conflict_records": AISConflictRecord,
"ais_source_health": AISSourceHealth,
}
deleted_counts: dict[str, int] = {}
for key in adapter.derived_models:
model = model_by_key.get(key)
if model is None:
continue
result = await db.execute(model.__table__.delete().where(model.source == source))
if key == "ais_conflict_records":
result = await db.execute(model.__table__.delete().where(model.selected_source == source))
else:
result = await db.execute(model.__table__.delete().where(model.source == source))
deleted_counts[key] = int(result.rowcount or 0)
return deleted_counts

File diff suppressed because it is too large Load Diff

View File

@@ -11,6 +11,7 @@ from app.services.earth_news import (
ParsedNewsItem,
apply_enrichment_patch_to_item,
build_anchor_location_patch,
_news_meta_patch,
)
@@ -34,6 +35,8 @@ def _location_patch_from_record(record: EarthNewsItem) -> dict[str, Any]:
def record_to_parsed_news_item(record: EarthNewsItem) -> ParsedNewsItem:
location_meta = dict(record.location_meta or {})
news_meta = location_meta.get("news_meta") if isinstance(location_meta.get("news_meta"), dict) else {}
item = ParsedNewsItem(
id=record.id,
title=record.title,
@@ -49,11 +52,29 @@ def record_to_parsed_news_item(record: EarthNewsItem) -> ParsedNewsItem:
enrichment_status=record.enrichment_status or "pending",
enrichment_error=record.enrichment_error,
enriched_at=_coerce_datetime(record.enriched_at),
source_tags=list(news_meta.get("source_tags") or []),
feed_id=str(news_meta.get("feed_id") or ""),
feed_type=str(news_meta.get("feed_type") or "rss"),
feed_default_category=str(news_meta.get("feed_default_category") or "other"),
category=str(news_meta.get("category") or "other"),
item_tags=list(news_meta.get("item_tags") or []),
tagging_source=str(news_meta.get("tagging_source") or "rules"),
tagging_confidence=float(news_meta.get("tagging_confidence") or 0),
importance_score=int(news_meta.get("importance_score") or 0),
importance_level=str(news_meta.get("importance_level") or "low"),
importance_reasons=list(news_meta.get("importance_reasons") or []),
market_impact=str(news_meta.get("market_impact") or "none"),
)
return apply_enrichment_patch_to_item(item, _location_patch_from_record(record))
def _query_sort_key(active_region: str):
if active_region == "global":
return (
EarthNewsItem.published_at.is_(None),
EarthNewsItem.published_at.desc().nullslast(),
EarthNewsItem.feed_name.asc(),
)
return (
EarthNewsItem.region != active_region,
EarthNewsItem.published_at.is_(None),
@@ -62,28 +83,93 @@ def _query_sort_key(active_region: str):
)
def _category_filter_clause(categories: set[str] | None):
if not categories:
return None
return EarthNewsItem.location_meta.op("->")("news_meta").op("->>")("category").in_(sorted(categories))
def _source_filter_clause(source_ids: set[str] | None):
if not source_ids:
return None
return EarthNewsItem.location_meta.op("->")("news_meta").op("->>")("source_id").in_(sorted(source_ids))
def _record_source_id(record: EarthNewsItem) -> str:
location_meta = dict(record.location_meta or {})
news_meta = location_meta.get("news_meta") if isinstance(location_meta.get("news_meta"), dict) else {}
source_id = str(news_meta.get("source_id") or "").strip()
if source_id:
return source_id
if isinstance(record.id, str) and ":" in record.id:
return record.id.split(":", 1)[0]
return record.feed_name or record.source or record.id
def _diversify_records_by_source(records: list[EarthNewsItem], *, limit: int) -> list[EarthNewsItem]:
if limit <= 0 or len(records) <= limit:
return records[:limit]
buckets: dict[str, list[EarthNewsItem]] = {}
order: list[str] = []
for record in records:
source_id = _record_source_id(record)
if source_id not in buckets:
buckets[source_id] = []
order.append(source_id)
buckets[source_id].append(record)
diversified: list[EarthNewsItem] = []
while len(diversified) < limit and order:
next_order: list[str] = []
for source_id in order:
bucket = buckets.get(source_id) or []
if bucket and len(diversified) < limit:
diversified.append(bucket.pop(0))
if bucket:
next_order.append(source_id)
order = next_order
return diversified
async def list_earth_news_items(
db: AsyncSession,
*,
active_region: str,
limit: int,
categories: set[str] | None = None,
source_ids: set[str] | None = None,
) -> list[ParsedNewsItem]:
regions = {"global", active_region}
result = await db.execute(
query_limit = limit if source_ids else min(max(limit * 4, limit), 100)
query = (
select(EarthNewsItem)
.where(EarthNewsItem.region.in_(regions))
.order_by(*_query_sort_key(active_region))
.limit(limit)
.limit(query_limit)
)
return [record_to_parsed_news_item(record) for record in result.scalars().all()]
if active_region != "global":
query = query.where(EarthNewsItem.region.in_({"global", active_region}))
category_clause = _category_filter_clause(categories)
if category_clause is not None:
query = query.where(category_clause)
source_clause = _source_filter_clause(source_ids)
if source_clause is not None:
query = query.where(source_clause)
result = await db.execute(query)
records = list(result.scalars().all())
if not source_ids:
records = _diversify_records_by_source(records, limit=limit)
else:
records = records[:limit]
return [record_to_parsed_news_item(record) for record in records]
async def list_earth_news_cruise_items(
db: AsyncSession,
*,
limit: int,
categories: set[str] | None = None,
source_ids: set[str] | None = None,
) -> list[ParsedNewsItem]:
result = await db.execute(
query = (
select(EarthNewsItem)
.order_by(
EarthNewsItem.region.asc(),
@@ -93,6 +179,13 @@ async def list_earth_news_cruise_items(
)
.limit(limit)
)
category_clause = _category_filter_clause(categories)
if category_clause is not None:
query = query.where(category_clause)
source_clause = _source_filter_clause(source_ids)
if source_clause is not None:
query = query.where(source_clause)
result = await db.execute(query)
return [record_to_parsed_news_item(record) for record in result.scalars().all()]
@@ -101,13 +194,13 @@ async def get_earth_news_freshness(
*,
active_region: str,
) -> tuple[int, datetime | None]:
regions = {"global", active_region}
result = await db.execute(
select(
func.count(EarthNewsItem.id),
func.max(func.coalesce(EarthNewsItem.published_at, EarthNewsItem.last_seen_at)),
).where(EarthNewsItem.region.in_(regions))
query = select(
func.count(EarthNewsItem.id),
func.max(func.coalesce(EarthNewsItem.published_at, EarthNewsItem.last_seen_at)),
)
if active_region != "global":
query = query.where(EarthNewsItem.region.in_({"global", active_region}))
result = await db.execute(query)
count, newest = result.one()
item_count = int(count or 0)
if item_count == 0:
@@ -115,6 +208,33 @@ async def get_earth_news_freshness(
return item_count, _coerce_datetime(newest)
async def get_earth_news_feed_coverage(
db: AsyncSession,
*,
active_region: str,
recent_after: datetime | None = None,
) -> set[tuple[str, str]]:
query = select(
EarthNewsItem.id,
EarthNewsItem.location_meta.op("->")("news_meta").op("->>")("source_id"),
EarthNewsItem.location_meta.op("->")("news_meta").op("->>")("feed_id"),
)
if active_region != "global":
query = query.where(EarthNewsItem.region.in_({"global", active_region}))
if recent_after is not None:
query = query.where(func.coalesce(EarthNewsItem.published_at, EarthNewsItem.last_seen_at) >= recent_after)
result = await db.execute(query)
coverage: set[tuple[str, str]] = set()
for item_id, source_id, feed_id in result.all():
normalized_source_id = str(source_id or "").strip()
normalized_feed_id = str(feed_id or "").strip()
if not normalized_source_id and isinstance(item_id, str) and ":" in item_id:
normalized_source_id = item_id.split(":", 1)[0]
if normalized_source_id and normalized_feed_id:
coverage.add((normalized_source_id, normalized_feed_id))
return coverage
async def upsert_earth_news_items(db: AsyncSession, items: list[ParsedNewsItem]) -> int:
if not items:
return 0
@@ -165,12 +285,20 @@ async def upsert_earth_news_items(db: AsyncSession, items: list[ParsedNewsItem])
record.homepage_url = item.homepage_url
record.published_at = item.published_at
record.last_seen_at = now
location_meta = dict(record.location_meta or {})
location_meta["news_meta"] = _news_meta_patch(item)
record.location_meta = location_meta
if item.localizations:
merged_localizations = {
**dict(record.localizations or {}),
**dict(item.localizations or {}),
}
record.content_language = item.content_language
record.localizations = dict(item.localizations or {})
record.enrichment_status = item.enrichment_status
record.enrichment_error = item.enrichment_error
record.enriched_at = item.enriched_at
record.localizations = merged_localizations
if item.enrichment_status != "pending" or item.enrichment_error or item.enriched_at:
record.enrichment_status = item.enrichment_status
record.enrichment_error = item.enrichment_error
record.enriched_at = item.enriched_at
changed += 1
await db.flush()
return changed

View File

@@ -0,0 +1,161 @@
from __future__ import annotations
import asyncio
from dataclasses import dataclass, field
from datetime import UTC, datetime
from typing import Any
from fastapi import WebSocket
from app.db.session import async_session_factory
from app.services.system_logs import (
DEFAULT_LOG_LINE_LIMIT,
LOG_SOURCES,
MAX_LOG_LINE_LIMIT,
read_database_log_events,
read_log_events,
)
DATABASE_LOG_SOURCE_IDS = {"system-db", "audit-db"}
LOG_TAIL_CHANNEL = "logs_tail"
LOG_TAIL_INTERVAL_SECONDS = 1.5
LOG_TAIL_SCAN_MULTIPLIER = 5
@dataclass(frozen=True)
class LogTailConfig:
source_id: str
limit: int = DEFAULT_LOG_LINE_LIMIT
level: str = "all"
levels: str | None = None
start_date: str | None = None
end_date: str | None = None
search: str | None = None
@dataclass
class LogTailSubscription:
config: LogTailConfig
emitted_cursors: set[str] = field(default_factory=set)
task: asyncio.Task | None = None
class LogTailManager:
def __init__(self) -> None:
self._subscriptions: dict[WebSocket, LogTailSubscription] = {}
def normalize_config(self, payload: dict[str, Any]) -> LogTailConfig:
source_id = str(payload.get("source_id") or payload.get("source") or "").strip()
if not source_id:
raise ValueError("source_id is required")
if source_id not in LOG_SOURCES and source_id not in DATABASE_LOG_SOURCE_IDS:
raise ValueError("Log source not found")
try:
limit = int(payload.get("limit") or DEFAULT_LOG_LINE_LIMIT)
except (TypeError, ValueError) as exc:
raise ValueError("limit must be a number") from exc
if limit < 1 or limit > MAX_LOG_LINE_LIMIT:
raise ValueError(f"limit must be between 1 and {MAX_LOG_LINE_LIMIT}")
return LogTailConfig(
source_id=source_id,
limit=limit,
level=str(payload.get("level") or "all"),
levels=str(payload.get("levels")).strip() if payload.get("levels") else None,
start_date=str(payload.get("start_date")).strip() if payload.get("start_date") else None,
end_date=str(payload.get("end_date")).strip() if payload.get("end_date") else None,
search=str(payload.get("search")).strip() if payload.get("search") else None,
)
async def subscribe(self, websocket: WebSocket, payload: dict[str, Any]) -> LogTailConfig:
config = self.normalize_config(payload)
await self.unsubscribe(websocket)
subscription = LogTailSubscription(config=config)
subscription.task = asyncio.create_task(self._run_tail(websocket, subscription))
self._subscriptions[websocket] = subscription
return config
async def unsubscribe(self, websocket: WebSocket) -> None:
subscription = self._subscriptions.pop(websocket, None)
if subscription and subscription.task:
subscription.task.cancel()
try:
await subscription.task
except asyncio.CancelledError:
pass
async def disconnect(self, websocket: WebSocket) -> None:
await self.unsubscribe(websocket)
async def _run_tail(self, websocket: WebSocket, subscription: LogTailSubscription) -> None:
first_frame = True
while True:
events = await self._read_events(subscription.config)
if first_frame:
visible_events = events[-subscription.config.limit :]
subscription.emitted_cursors.update(event.cursor for event in visible_events)
await self._send_frame(websocket, subscription.config, "snapshot", visible_events)
first_frame = False
else:
new_events = [
event
for event in events
if event.cursor not in subscription.emitted_cursors
]
if new_events:
visible_events = new_events[-subscription.config.limit :]
subscription.emitted_cursors.update(event.cursor for event in visible_events)
await self._send_frame(websocket, subscription.config, "append", visible_events)
await asyncio.sleep(LOG_TAIL_INTERVAL_SECONDS)
async def _read_events(self, config: LogTailConfig):
scan_limit = max(config.limit * LOG_TAIL_SCAN_MULTIPLIER, config.limit)
if config.source_id in DATABASE_LOG_SOURCE_IDS:
async with async_session_factory() as db:
events = await read_database_log_events(
config.source_id,
scan_limit=scan_limit,
level=config.level,
levels=config.levels,
start_date=config.start_date,
end_date=config.end_date,
search=config.search,
db=db,
)
return events or []
events = read_log_events(
config.source_id,
scan_limit=scan_limit,
level=config.level,
levels=config.levels,
start_date=config.start_date,
end_date=config.end_date,
search=config.search,
)
return events or []
async def _send_frame(self, websocket: WebSocket, config: LogTailConfig, mode: str, events) -> None:
await websocket.send_json(
{
"type": "data_frame",
"channel": LOG_TAIL_CHANNEL,
"timestamp": datetime.now(UTC).isoformat(),
"payload": {
"mode": mode,
"source_id": config.source_id,
"line_count": len(events),
"lines": [event.line for event in events],
"filters": {
"limit": config.limit,
"level": config.level,
"levels": config.levels,
"start_date": config.start_date,
"end_date": config.end_date,
"search": config.search,
},
"status": "ok",
},
}
)
log_tail_manager = LogTailManager()

View File

@@ -1,14 +1,185 @@
from __future__ import annotations
import hashlib
import re
from datetime import UTC, datetime
from typing import Any
from app.core.logging import get_logger, sanitize_log_value
from app.core.request_context import get_request_id
from app.db.session import async_session_factory
from app.models.system_log import AuditLog, SystemLog
from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
logger = get_logger(__name__)
HLS_TRANSIENT_RE = re.compile(r"(index|chunk|segment)[_-]?\d+(?:_\d+)?\.(?:ts|m4s|vtt)", re.IGNORECASE)
QUERY_RE = re.compile(r"([?&](?:m|t|token|expires|signature|X-Amz-[^=]+)=[^&\\s]+)", re.IGNORECASE)
UUID_RE = re.compile(r"\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b", re.IGNORECASE)
CONNECTION_RE = re.compile(r"\bconn_[A-Za-z0-9:._-]+\b")
NUMBER_RE = re.compile(r"\b\d{5,}\b")
def normalize_observability_text(value: Any) -> str:
text = str(sanitize_log_value(value or "")).strip()
text = QUERY_RE.sub("", text)
text = HLS_TRANSIENT_RE.sub("<hls-fragment>", text)
text = UUID_RE.sub("<uuid>", text)
text = CONNECTION_RE.sub("<connection>", text)
text = NUMBER_RE.sub("<number>", text)
return re.sub(r"\s+", " ", text).strip()
def build_observability_fingerprint(
*,
source: str,
service: str | None = None,
module: str | None = None,
category: str | None = None,
event: str | None = None,
message: str,
context: dict[str, Any] | None = None,
) -> str:
context = context or {}
stable_context = {
key: context.get(key)
for key in (
"task_type",
"source_id",
"source",
"provider",
"status_code",
"error_type",
"details",
)
if context.get(key) not in (None, "")
}
raw = "|".join(
[
normalize_observability_text(source),
normalize_observability_text(service),
normalize_observability_text(module),
normalize_observability_text(category),
normalize_observability_text(event),
normalize_observability_text(message),
normalize_observability_text(stable_context),
]
)
return hashlib.sha1(raw.encode("utf-8", errors="replace")).hexdigest()
def _context_text(context: dict[str, Any] | None, key: str) -> str | None:
value = (context or {}).get(key)
if value in (None, ""):
return None
return str(value)
async def record_observability_event(
*,
source: str,
level: str,
message: str,
service: str | None = None,
module: str | None = None,
event: str | None = None,
request_id: str | None = None,
trace_id: str | None = None,
user_id: int | None = None,
category: str | None = None,
context: dict[str, Any] | None = None,
fingerprint: str | None = None,
occurred_at: datetime | None = None,
occurrence_count: int = 1,
) -> None:
normalized_context = sanitize_log_value(context or {})
if not isinstance(normalized_context, dict):
normalized_context = {"value": normalized_context}
safe_message = str(sanitize_log_value(message))
normalized_level = str(level or "info").lower()
count = max(1, int(occurrence_count or 1))
event_time = occurred_at or datetime.now(UTC)
event_fingerprint = fingerprint or build_observability_fingerprint(
source=source,
service=service,
module=module,
category=category,
event=event,
message=safe_message,
context=normalized_context,
)
detail = _context_text(normalized_context, "detail") or _context_text(normalized_context, "error")
affected_sources = sorted(
{
item
for item in (
source,
service,
module,
_context_text(normalized_context, "source_id"),
_context_text(normalized_context, "source"),
)
if item
}
)
try:
async with async_session_factory() as session:
session.add(
ObservabilityEvent(
source=source,
service=service,
module=module,
category=category,
event=event,
level=normalized_level,
message=safe_message,
fingerprint=event_fingerprint,
occurred_at=event_time,
request_id=request_id or get_request_id(),
trace_id=trace_id,
user_id=user_id,
task_id=_context_text(normalized_context, "task_id"),
source_ref_id=_context_text(normalized_context, "source_id") or _context_text(normalized_context, "source"),
provider=_context_text(normalized_context, "provider"),
context=normalized_context,
occurrence_count=count,
)
)
group = await session.get(ObservabilityEventGroup, event_fingerprint)
if group is None:
session.add(
ObservabilityEventGroup(
fingerprint=event_fingerprint,
source=source,
service=service,
module=module,
category=category,
event=event,
last_level=normalized_level,
sample_message=safe_message,
sample_detail=detail,
affected_sources=affected_sources,
count=count,
first_seen_at=event_time,
last_seen_at=event_time,
)
)
else:
group.count = int(group.count or 0) + count
group.last_seen_at = event_time
group.last_level = normalized_level
group.sample_message = safe_message
group.sample_detail = detail
merged_sources = sorted(set(group.affected_sources or []) | set(affected_sources))
group.affected_sources = merged_sources
await session.commit()
except Exception:
logger.exception_event(
"Failed to persist observability event",
event="observability_event.persist.failed",
context={"event_name": event, "source": source},
)
async def record_system_log(
*,
@@ -23,6 +194,8 @@ async def record_system_log(
user_id: int | None = None,
category: str | None = None,
context: dict[str, Any] | None = None,
fingerprint: str | None = None,
occurrence_count: int = 1,
) -> None:
try:
async with async_session_factory() as session:
@@ -48,6 +221,21 @@ async def record_system_log(
event="system_log.persist.failed",
context={"event_name": event, "source": source},
)
await record_observability_event(
source=source,
service=service,
module=module,
event=event,
level=level,
message=message,
request_id=request_id,
trace_id=trace_id,
user_id=user_id,
category=category,
context=context,
fingerprint=fingerprint,
occurrence_count=occurrence_count,
)
async def record_audit_log(

View File

@@ -5,6 +5,7 @@ import os
import re
import shutil
import subprocess
import hashlib
from collections import Counter, deque
from dataclasses import dataclass
@@ -13,6 +14,9 @@ from pathlib import Path
from typing import Any
from app.core.security import redis_client
from app.models.system_log import AuditLog, ObservabilityEvent, ObservabilityEventGroup, SystemLog
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
DEFAULT_LOG_LINE_LIMIT = 200
MAX_LOG_LINE_LIMIT = 1000
@@ -99,6 +103,16 @@ class StructuredLogEntry:
search_text: str
@dataclass(frozen=True)
class LogEvent:
source_id: str
cursor: str
timestamp: datetime | None
level: str | None
line: str
search_text: str
@dataclass
class DailyLogMarker:
date_token: str
@@ -106,6 +120,10 @@ class DailyLogMarker:
dominant_level: str
def _normalize_search_query(search: str | None) -> str:
return (search or "").strip().lower()
def _planet_state_dir() -> Path:
configured = os.getenv("PLANET_STATE_DIR")
if configured:
@@ -135,7 +153,7 @@ LOG_SOURCES: dict[str, LogSource] = {
name="前端开发服务",
kind="file",
location=_state_log_path("frontend.log"),
description="控制台与 Earth 前端开发服务输出。",
description="控制台与智能星球前端开发服务输出。",
category="service",
fallback_locations=("/tmp/planet_frontend.log",),
),
@@ -150,13 +168,22 @@ LOG_SOURCES: dict[str, LogSource] = {
),
"earth-client": LogSource(
source_id="earth-client",
name="Earth 浏览器端",
name="智能星球浏览器端",
kind="buffer",
location="redis://planet:system_logs:earth-client",
description="Earth 浏览器端上报的运行时错误与关键业务日志。",
description="智能星球浏览器端上报的运行时错误与关键业务日志。",
category="client",
buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:earth-client",
),
"admin-client": LogSource(
source_id="admin-client",
name="控制台浏览器端",
kind="buffer",
location="redis://planet:system_logs:admin-client",
description="控制台浏览器端上报的运行时错误。",
category="client",
buffer_key=f"{LOG_BUFFER_KEY_PREFIX}:admin-client",
),
}
@@ -365,6 +392,59 @@ def build_buffer_entry(payload: dict[str, Any]) -> StructuredLogEntry:
)
def compact_log_context(context: dict | None) -> str:
if not context:
return ""
allowed = {
key: value
for key, value in (context or {}).items()
if key
in {
"status",
"duration_ms",
"provider",
"model",
"result_provider",
"result_model",
"collector_name",
"datasource_id",
"task_id",
"snapshot_id",
"raw_count",
"transformed_count",
"saved_count",
"created",
"updated",
"unchanged",
"deleted",
"result_count",
"status_code",
"error_type",
"error",
"route",
"module",
}
}
if not allowed:
return ""
return json.dumps(allowed, ensure_ascii=False, sort_keys=True)
def context_search_aliases(context: dict | None) -> str:
if not context:
return ""
aliases: list[str] = []
for key, value in sorted((context or {}).items()):
if value is None or isinstance(value, (dict, list, tuple, set)):
continue
normalized_key = str(key).strip()
normalized_value = str(value).strip()
if not normalized_key or not normalized_value:
continue
aliases.append(f"{normalized_key}={normalized_value}")
return " ".join(aliases)
def read_file_entries(source: LogSource, scan_limit: int) -> list[StructuredLogEntry]:
path = resolve_file_log_path(source)
if not path.exists():
@@ -437,6 +517,400 @@ def read_source_entries(source: LogSource, scan_limit: int) -> list[StructuredLo
return []
def _database_event_from_system_record(record: SystemLog) -> LogEvent:
record_level = normalize_log_level(record.level)
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
record_level.upper(),
record.source,
record.category or "",
record.event or "",
f"request_id={record.request_id}" if record.request_id else "",
record.message,
compact_log_context(record.context),
]
if part
)
search_text = " ".join(
[
line,
f"id={record.id}",
f"user_id={record.user_id}" if record.user_id else "",
context_search_aliases(record.context),
json.dumps(record.context or {}, ensure_ascii=False, sort_keys=True),
]
).lower()
return LogEvent(
source_id="system-db",
cursor=f"system-db:{record.id}",
timestamp=record.occurred_at,
level=None if record_level == LOG_LEVEL_ALL else record_level,
line=line,
search_text=search_text,
)
def _database_event_from_audit_record(record: AuditLog) -> LogEvent:
line = " ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
"INFO",
record.action,
record.target_type or "",
record.target_id or "",
record.result or "",
f"request_id={record.request_id}" if record.request_id else "",
]
if part
)
search_text = " ".join(
[
line,
f"id={record.id}",
f"actor_id={record.actor_id}" if record.actor_id else "",
record.actor_name or "",
context_search_aliases(record.details),
json.dumps(record.details or {}, ensure_ascii=False, sort_keys=True),
]
).lower()
return LogEvent(
source_id="audit-db",
cursor=f"audit-db:{record.id}",
timestamp=record.occurred_at,
level=LOG_LEVEL_INFO,
line=line,
search_text=search_text,
)
async def read_database_log_events(
source_id: str,
*,
scan_limit: int,
level: str = LOG_LEVEL_ALL,
levels: str | None = None,
start_date: str | None = None,
end_date: str | None = None,
search: str | None = None,
db: AsyncSession,
) -> list[LogEvent] | None:
selected_levels = normalize_log_levels(level, levels)
search_query = (search or "").strip()
if source_id == "system-db":
query = select(SystemLog).order_by(SystemLog.occurred_at.desc().nullslast(), SystemLog.id.desc()).limit(scan_limit)
result = await db.execute(query)
events = [_database_event_from_system_record(record) for record in result.scalars().all()]
elif source_id == "audit-db":
query = select(AuditLog).order_by(AuditLog.occurred_at.desc().nullslast(), AuditLog.id.desc()).limit(scan_limit)
result = await db.execute(query)
events = [_database_event_from_audit_record(record) for record in result.scalars().all()]
else:
return None
events = list(reversed(events))
return [
event
for event in events
if event_matches_levels(event, selected_levels)
and event_matches_search(event, search_query)
and event_matches_date_range(event, start_date, end_date)
]
async def read_database_log_snapshot(
source_id: str,
*,
limit: int,
level: str,
levels: str | None,
start_date: str | None,
end_date: str | None,
search: str | None,
db: AsyncSession,
) -> dict[str, Any] | None:
events = await read_database_log_events(
source_id,
scan_limit=limit * 5,
level=level,
levels=levels,
start_date=start_date,
end_date=end_date,
search=search,
db=db,
)
if events is None:
return None
visible_events = events[-limit:]
selected_levels = normalize_log_levels(level, levels)
return {
"source_id": source_id,
"name": "系统事件" if source_id == "system-db" else "审计事件",
"kind": "database",
"location": "table://system_logs" if source_id == "system-db" else "table://audit_logs",
"description": "数据库持久化日志",
"category": "database" if source_id == "system-db" else "audit",
"status": "ok" if visible_events else "empty",
"level": level,
"selected_levels": list(selected_levels),
"search_query": search or "",
"available_levels": ["all", "error", "warning", "info", "debug"],
"daily_markers": build_daily_log_markers_from_events(events),
"line_limit": limit,
"line_count": len(visible_events),
"lines": [event.line for event in visible_events],
}
def _observability_group_matches(
group: ObservabilityEventGroup,
*,
selected_levels: tuple[str, ...],
start_date: str | None,
end_date: str | None,
search: str | None,
) -> bool:
if selected_levels and group.last_level not in selected_levels:
return False
if start_date or end_date:
if group.last_seen_at is None:
return False
date_token = group.last_seen_at.astimezone(UTC).date().isoformat()
if start_date and date_token < start_date:
return False
if end_date and date_token > end_date:
return False
query = _normalize_search_query(search)
if not query:
return True
haystack = " ".join(
[
group.fingerprint or "",
group.source or "",
group.service or "",
group.module or "",
group.category or "",
group.event or "",
group.last_level or "",
group.sample_message or "",
group.sample_detail or "",
json.dumps(group.affected_sources or [], ensure_ascii=False, sort_keys=True),
]
).lower()
return query in haystack
def _serialize_observability_group(group: ObservabilityEventGroup) -> dict[str, Any]:
return {
"fingerprint": group.fingerprint,
"source": group.source,
"service": group.service,
"module": group.module,
"category": group.category,
"event": group.event,
"level": group.last_level,
"message": group.sample_message,
"detail": group.sample_detail,
"affected_sources": group.affected_sources or [],
"count": group.count or 0,
"first_seen_at": group.first_seen_at.isoformat() if group.first_seen_at else None,
"last_seen_at": group.last_seen_at.isoformat() if group.last_seen_at else None,
}
def _serialize_observability_event(record: ObservabilityEvent) -> dict[str, Any]:
return {
"id": record.id,
"source": record.source,
"service": record.service,
"module": record.module,
"category": record.category,
"event": record.event,
"level": record.level,
"message": record.message,
"fingerprint": record.fingerprint,
"occurred_at": record.occurred_at.isoformat() if record.occurred_at else None,
"request_id": record.request_id,
"trace_id": record.trace_id,
"task_id": record.task_id,
"source_id": record.source_ref_id,
"provider": record.provider,
"user_id": record.user_id,
"context": record.context or {},
"occurrence_count": record.occurrence_count or 1,
}
async def read_observability_groups(
*,
limit: int,
level: str = LOG_LEVEL_ALL,
levels: str | None = None,
start_date: str | None = None,
end_date: str | None = None,
search: str | None = None,
db: AsyncSession,
) -> dict[str, Any]:
selected_levels = normalize_log_levels(level, levels)
scan_limit = max(limit * 5, limit, DEFAULT_LOG_LINE_LIMIT)
result = await db.execute(
select(ObservabilityEventGroup)
.order_by(ObservabilityEventGroup.last_seen_at.desc().nullslast())
.limit(scan_limit)
)
groups = [
group
for group in result.scalars().all()
if _observability_group_matches(
group,
selected_levels=selected_levels,
start_date=start_date,
end_date=end_date,
search=search,
)
][:limit]
return {
"mode": "grouped",
"line_limit": limit,
"line_count": len(groups),
"groups": [_serialize_observability_group(group) for group in groups],
"filters": {
"level": level,
"levels": list(selected_levels),
"start_date": start_date,
"end_date": end_date,
"search": search or "",
},
}
async def read_observability_group_events(
fingerprint: str,
*,
limit: int,
db: AsyncSession,
) -> dict[str, Any] | None:
group = await db.get(ObservabilityEventGroup, fingerprint)
if group is None:
return None
result = await db.execute(
select(ObservabilityEvent)
.where(ObservabilityEvent.fingerprint == fingerprint)
.order_by(ObservabilityEvent.occurred_at.desc().nullslast(), ObservabilityEvent.id.desc())
.limit(limit)
)
events = list(reversed(result.scalars().all()))
return {
"fingerprint": fingerprint,
"group": _serialize_observability_group(group),
"line_limit": limit,
"line_count": len(events),
"events": [_serialize_observability_event(record) for record in events],
}
async def read_observability_raw_events(
*,
limit: int,
level: str = LOG_LEVEL_ALL,
levels: str | None = None,
start_date: str | None = None,
end_date: str | None = None,
search: str | None = None,
db: AsyncSession,
) -> dict[str, Any]:
selected_levels = normalize_log_levels(level, levels)
query = select(ObservabilityEvent).order_by(ObservabilityEvent.occurred_at.desc().nullslast(), ObservabilityEvent.id.desc())
if selected_levels:
query = query.where(ObservabilityEvent.level.in_(selected_levels))
result = await db.execute(query.limit(max(limit * 5, limit)))
records = result.scalars().all()
search_query = _normalize_search_query(search)
visible: list[ObservabilityEvent] = []
for record in records:
if start_date or end_date:
if record.occurred_at is None:
continue
date_token = record.occurred_at.astimezone(UTC).date().isoformat()
if start_date and date_token < start_date:
continue
if end_date and date_token > end_date:
continue
if search_query:
haystack = " ".join(
[
record.source or "",
record.service or "",
record.module or "",
record.category or "",
record.event or "",
record.message or "",
record.fingerprint or "",
record.request_id or "",
record.trace_id or "",
record.task_id or "",
record.source_ref_id or "",
record.provider or "",
json.dumps(record.context or {}, ensure_ascii=False, sort_keys=True),
]
).lower()
if search_query not in haystack:
continue
visible.append(record)
if len(visible) >= limit:
break
visible = list(reversed(visible))
return {
"mode": "raw",
"line_limit": limit,
"line_count": len(visible),
"events": [_serialize_observability_event(record) for record in visible],
"lines": [
" ".join(
part
for part in [
record.occurred_at.isoformat() if record.occurred_at else "",
record.level.upper(),
record.source,
record.category or "",
record.event or "",
f"fingerprint={record.fingerprint}",
record.message,
]
if part
)
for record in visible
],
}
def _stable_hash(value: str) -> str:
return hashlib.sha1(value.encode("utf-8", errors="replace")).hexdigest()[:16]
def build_log_events(source_id: str, entries: list[StructuredLogEntry]) -> list[LogEvent]:
events: list[LogEvent] = []
seen: dict[str, int] = {}
for entry in entries:
stable_value = entry.raw_line or entry.display_line
digest = _stable_hash(stable_value)
occurrence = seen.get(digest, 0) + 1
seen[digest] = occurrence
events.append(
LogEvent(
source_id=source_id,
cursor=f"{source_id}:{digest}:{occurrence}",
timestamp=entry.timestamp,
level=entry.level,
line=entry.display_line,
search_text=entry.search_text,
)
)
return events
def matches_levels(entry: StructuredLogEntry, selected_levels: tuple[str, ...]) -> bool:
if not selected_levels:
return True
@@ -469,6 +943,34 @@ def matches_search(entry: StructuredLogEntry, search: str | None) -> bool:
return query in entry.search_text
def event_matches_levels(event: LogEvent, selected_levels: tuple[str, ...]) -> bool:
if not selected_levels:
return True
return event.level in selected_levels
def event_matches_date_range(event: LogEvent, start_date: str | None, end_date: str | None) -> bool:
if not start_date and not end_date:
return True
if event.timestamp is None:
return False
date_token = event.timestamp.astimezone(UTC).date().isoformat()
if start_date and date_token < start_date:
return False
if end_date and date_token > end_date:
return False
return True
def event_matches_search(event: LogEvent, search: str | None) -> bool:
if search is None:
return True
query = search.strip().lower()
if not query:
return True
return query in event.search_text
def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str, Any]]:
grouped: dict[str, list[StructuredLogEntry]] = {}
for entry in entries:
@@ -503,6 +1005,69 @@ def build_daily_log_markers(entries: list[StructuredLogEntry]) -> list[dict[str,
return [marker.__dict__ for marker in markers]
def build_daily_log_markers_from_events(events: list[LogEvent]) -> list[dict[str, Any]]:
grouped: dict[str, list[LogEvent]] = {}
for event in events:
if event.timestamp is None:
continue
date_token = event.timestamp.astimezone(UTC).date().isoformat()
grouped.setdefault(date_token, []).append(event)
markers: list[DailyLogMarker] = []
for date_token, group in sorted(grouped.items()):
level_counts = Counter(
event.level
for event in group
if event.level in SUPPORTED_LOG_LEVELS and event.level != LOG_LEVEL_ALL
)
dominant_level = LOG_LEVEL_INFO
if level_counts:
dominant_level = sorted(
level_counts.items(),
key=lambda item: (
-item[1],
("error", "warning", "info", "debug").index(item[0]),
),
)[0][0]
markers.append(
DailyLogMarker(
date_token=date_token,
total=len(group),
dominant_level=dominant_level,
)
)
return [marker.__dict__ for marker in markers]
def read_log_events(
source_id: str,
scan_limit: int,
*,
level: str = LOG_LEVEL_ALL,
levels: str | None = None,
start_date: str | None = None,
end_date: str | None = None,
search: str | None = None,
) -> list[LogEvent] | None:
source = LOG_SOURCES.get(source_id)
if source is None:
return None
selected_levels = normalize_log_levels(level, levels)
search_query = (search or "").strip()
events = build_log_events(source_id, read_source_entries(source, scan_limit))
marker_events = [
event
for event in events
if event_matches_levels(event, selected_levels) and event_matches_search(event, search_query)
]
return [
event
for event in marker_events
if event_matches_date_range(event, start_date, end_date)
]
def read_log_snapshot(
source_id: str,
limit: int,
@@ -520,18 +1085,18 @@ def read_log_snapshot(
selected_levels = normalize_log_levels(level, levels)
search_query = (search or "").strip()
scan_limit = max(min(MAX_LOG_LINE_LIMIT * 5, 5000), limit * 5, BUFFER_LOG_LIMIT if source.kind == "buffer" else 1000)
all_entries = read_source_entries(source, scan_limit)
marker_entries = [
entry
for entry in all_entries
if matches_levels(entry, selected_levels) and matches_search(entry, search_query)
all_events = build_log_events(source_id, read_source_entries(source, scan_limit))
marker_events = [
event
for event in all_events
if event_matches_levels(event, selected_levels) and event_matches_search(event, search_query)
]
filtered_entries = [
entry
for entry in marker_entries
if matches_date_range(entry, start_date, end_date)
filtered_events = [
event
for event in marker_events
if event_matches_date_range(event, start_date, end_date)
]
visible_entries = filtered_entries[-limit:]
visible_events = filtered_events[-limit:]
compatibility_level = selected_levels[0] if len(selected_levels) == 1 else LOG_LEVEL_ALL
return {
@@ -552,8 +1117,8 @@ def read_log_snapshot(
LOG_LEVEL_INFO,
LOG_LEVEL_DEBUG,
],
"daily_markers": build_daily_log_markers(marker_entries),
"daily_markers": build_daily_log_markers_from_events(marker_events),
"line_limit": limit,
"line_count": len(visible_entries),
"lines": [entry.display_line for entry in visible_entries],
"line_count": len(visible_events),
"lines": [event.line for event in visible_events],
}

View File

@@ -655,6 +655,8 @@ async def test_ingest_earth_client_log_accepts_public_events():
"message": "登陆点加载失败: 登陆点接口返回 HTTP 500",
"category": "startup-load",
"module": "layer-startup",
"fingerprint": "client-test",
"occurrence_count": 3,
},
)
assert response.status_code == 200
@@ -668,10 +670,98 @@ async def test_ingest_earth_client_log_accepts_public_events():
assert persisted_kwargs["event"] == "earth.client.runtime_log"
assert persisted_kwargs["category"] == "startup-load"
assert persisted_kwargs["level"] == "error"
assert persisted_kwargs["fingerprint"] == "client-test"
assert persisted_kwargs["occurrence_count"] == 3
finally:
app.dependency_overrides.clear()
@pytest.mark.asyncio
async def test_ingest_admin_client_log_accepts_public_events():
transport = ASGITransport(app=app)
try:
with patch("app.api.v1.system_control.append_buffer_log") as mock_append_buffer_log:
with patch("app.api.v1.system_control.record_system_log", new_callable=AsyncMock) as mock_record_system_log:
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.post(
"/api/v1/system/logs/admin-client",
json={
"level": "error",
"message": "控制台发生未处理 Promise 错误",
"category": "unhandledrejection",
"module": "admin",
"url": "http://test/logs",
"detail": "stack preview",
},
)
assert response.status_code == 200
data = response.json()
assert data["accepted"] is True
assert data["source_id"] == "admin-client"
mock_append_buffer_log.assert_called_once()
mock_record_system_log.assert_awaited_once()
persisted_kwargs = mock_record_system_log.await_args.kwargs
assert persisted_kwargs["source"] == "admin-client"
assert persisted_kwargs["event"] == "admin.client.runtime_log"
assert persisted_kwargs["category"] == "unhandledrejection"
assert persisted_kwargs["context"]["url"] == "http://test/logs"
finally:
app.dependency_overrides.clear()
@pytest.mark.asyncio
async def test_ingest_service_log_requires_configured_token(monkeypatch):
monkeypatch.setattr(settings, "OBSERVABILITY_INGEST_TOKEN", "")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.post(
"/api/v1/system/logs/service",
json={"message": "AI provider failed"},
headers={"X-Planet-Observability-Token": "secret"},
)
assert response.status_code == 503
@pytest.mark.asyncio
async def test_ingest_service_log_accepts_internal_token(monkeypatch):
monkeypatch.setattr(settings, "OBSERVABILITY_INGEST_TOKEN", "service-secret")
transport = ASGITransport(app=app)
with patch("app.api.v1.system_control.record_system_log", new_callable=AsyncMock) as mock_record_system_log:
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.post(
"/api/v1/system/logs/service",
json={
"source": "ai-provider",
"service": "ai-provider",
"module": "provider",
"category": "connectivity",
"event": "ai.provider.test.failed",
"level": "error",
"message": "Provider connectivity failed",
"fingerprint": "ai-provider-test",
"occurrence_count": 4,
"provider": "minimax",
"trace_id": "trace-123",
"context": {"status_code": 502},
},
headers={"Authorization": "Bearer service-secret"},
)
assert response.status_code == 200
data = response.json()
assert data["accepted"] is True
assert data["source_id"] == "ai-provider"
mock_record_system_log.assert_awaited_once()
persisted_kwargs = mock_record_system_log.await_args.kwargs
assert persisted_kwargs["event"] == "ai.provider.test.failed"
assert persisted_kwargs["fingerprint"] == "ai-provider-test"
assert persisted_kwargs["occurrence_count"] == 4
assert persisted_kwargs["context"]["provider"] == "minimax"
assert persisted_kwargs["context"]["trace_id"] == "trace-123"
assert persisted_kwargs["context"]["status_code"] == 502
@pytest.mark.asyncio
async def test_earth_layer_cache_status_requires_super_admin(auth_headers, monkeypatch):
def override_get_current_user():

View File

@@ -74,6 +74,6 @@ def test_interactable_cache_invalidation_clears_layer_and_all(monkeypatch):
assert deleted == 2
assert patterns == [
"earth:layer:v1:interactables:layer:places*",
"earth:layer:v1:interactables:layer:all*",
"earth:layer:v1:interactables:interactable_layer:places*",
"earth:layer:v1:interactables:interactable_layer:all*",
]

View File

@@ -4,14 +4,20 @@ from types import SimpleNamespace
import pytest
from app.services.earth_news import (
NewsFeedEndpoint,
NewsFeedSource,
NewsTargetLocation,
ParsedNewsItem,
apply_news_classification,
default_earth_news_sources_payload,
normalize_earth_news_sources_payload,
_fetch_source,
_enrich_items_with_target_locations,
_extract_target_location_from_text,
_parse_feed_entries,
_serialize_item,
get_earth_news_payload,
test_news_source_config as run_news_source_config_test,
)
from app.services.earth_news_queue import NewsTargetLocationMessage
from app.services.earth_news_worker import process_target_location_message
@@ -164,6 +170,479 @@ def test_parse_aggregated_rss_splits_publisher_from_title():
assert items[0].source == "Reuters"
def test_parse_chinese_rss_marks_source_language_and_keeps_zh_localization():
source = NewsFeedSource(
id="36kr",
name="36氪",
region="asia-pacific",
feed_url="https://36kr.com/feed",
homepage_url="https://www.36kr.com/",
source_tags=("china", "business_news"),
default_category="business",
)
xml = """
<rss>
<channel>
<item>
<title>中国电商平台发布季度增长数据</title>
<description>平台表示,跨境电商订单量同比增长。</description>
<link>https://36kr.com/p/example</link>
</item>
</channel>
</rss>
"""
items = _parse_feed_entries(xml, source)
payload_zh = _serialize_item(items[0], active_region="global", locale="zh-CN")
payload_en = _serialize_item(items[0], active_region="global", locale="en-US")
assert items[0].content_language == "zh-CN"
assert items[0].localizations["zh-CN"]["title"] == "中国电商平台发布季度增长数据"
assert payload_zh["display_title"] == "中国电商平台发布季度增长数据"
assert payload_en["display_title"] == "中国电商平台发布季度增长数据"
def test_default_news_sources_include_business_and_ecommerce_sources():
payload = default_earth_news_sources_payload()
sources_by_id = {source["id"]: source for source in payload["sources"]}
source_ids = {source["id"] for source in payload["sources"]}
category_keys = {category["key"] for category in payload["categories"]}
tag_keys = {tag["key"] for tag in payload["source_tags"]}
assert "cnbc-business" in source_ids
assert "36kr" in source_ids
assert "techcrunch" in source_ids
assert "retaildive" in source_ids
assert "prnewswire-retail" in source_ids
assert "google-news" in source_ids
assert "global-scan" not in source_ids
assert "google-americas" not in source_ids
assert "google-europe" not in source_ids
assert "google-mea" not in source_ids
assert "google-apac" not in source_ids
assert "businesswire-ecommerce" in source_ids
assert "us-census-ecommerce" in source_ids
assert "mofcom-data" in source_ids
assert "stats-china-online-retail" in source_ids
assert "ebrun" in source_ids
assert sources_by_id["36kr"]["source_type"] == "rss"
assert sources_by_id["36kr"]["homepage_url"] == "https://www.36kr.com/"
assert sources_by_id["36kr"]["feed_directory_url"] == "https://www.36kr.com/rss-center"
kr_feeds = {feed["id"]: feed for feed in sources_by_id["36kr"]["feeds"]}
assert set(kr_feeds) == {"feed", "article", "newsflash", "moment"}
assert kr_feeds["feed"]["url"] == "https://36kr.com/feed"
assert kr_feeds["article"]["url"] == "https://36kr.com/feed-article"
assert kr_feeds["newsflash"]["url"] == "https://36kr.com/feed-newsflash"
assert kr_feeds["moment"]["url"] == "https://36kr.com/feed-moment"
assert all(feed["enabled"] is True for feed in kr_feeds.values())
assert all(feed["default_category"] == "business" for feed in kr_feeds.values())
assert "https://36kr.com/feed-article" in sources_by_id["36kr"]["feed_urls"]
assert "https://36kr.com/feed-newsflash" in sources_by_id["36kr"]["feed_urls"]
assert "https://36kr.com/feed-moment" in sources_by_id["36kr"]["feed_urls"]
assert sources_by_id["ebrun"]["source_type"] == "rss"
assert sources_by_id["ebrun"]["homepage_url"] == "https://www.ebrun.com/"
assert sources_by_id["ebrun"]["feed_directory_url"] == "https://www.ebrun.com/rss/"
ebrun_feeds = {feed["id"]: feed for feed in sources_by_id["ebrun"]["feeds"]}
assert {"b2c", "b2b", "retail", "o2o", "service", "data", "policy"}.issubset(ebrun_feeds)
assert all(feed["enabled"] is True for feed in ebrun_feeds.values())
assert all(feed["default_category"] == "ecommerce" for feed in ebrun_feeds.values())
assert "https://www.ebrun.com/rss/news_b2c.xml" in sources_by_id["ebrun"]["feed_urls"]
assert "https://www.ebrun.com/rss/news_retail.xml" in sources_by_id["ebrun"]["feed_urls"]
assert sources_by_id["businesswire-ecommerce"]["source_type"] == "reference"
assert sources_by_id["businesswire-ecommerce"]["enabled"] is False
assert sources_by_id["google-news"]["source_type"] == "aggregated"
assert sources_by_id["google-news"]["homepage_url"] == "https://news.google.com/"
assert sources_by_id["google-news"]["feed_directory_url"] == "https://news.google.com/rss"
google_feeds = {feed["id"]: feed for feed in sources_by_id["google-news"]["feeds"]}
assert set(google_feeds) == {"world", "americas", "europe", "middle-east-africa", "asia-pacific"}
assert all(feed["type"] == "aggregated" for feed in google_feeds.values())
assert all(feed["enabled"] is True for feed in google_feeds.values())
assert google_feeds["world"]["region"] == "global"
assert google_feeds["europe"]["region"] == "europe"
assert sources_by_id["stats-china-online-retail"]["source_type"] == "rss"
assert sources_by_id["stats-china-online-retail"]["enabled"] is True
assert "https://www.stats.gov.cn/sj/zxfb/rss.xml" in sources_by_id["stats-china-online-retail"]["feed_urls"]
assert {"business", "ecommerce", "finance"}.issubset(category_keys)
assert {"official_data", "business_news", "ecommerce", "press_release", "finance", "logistics"}.issubset(tag_keys)
def test_default_enabled_fetchable_sources_have_explicit_types_and_urls():
payload = default_earth_news_sources_payload()
for source in payload["sources"]:
source_type = source["source_type"]
assert source_type in {"rss", "atom", "aggregated", "reference"}
if source_type == "reference":
assert source["enabled"] is False
assert source["feeds"] == []
continue
if source["enabled"]:
assert source["feed_url"]
assert source["feed_urls"]
assert source["feeds"]
assert any(feed["enabled"] for feed in source["feeds"])
for feed in source["feeds"]:
assert feed["url"] != source["homepage_url"]
assert feed["url"] != source.get("feed_directory_url", "")
def test_legacy_news_source_urls_migrate_to_feed_children():
payload = normalize_earth_news_sources_payload(
{
"sources": [
{
"id": "legacy-source",
"name": "Legacy Source",
"region": "global",
"source_type": "rss",
"feed_urls": ["https://example.com/a.xml", "https://example.com/b.xml"],
"default_category": "business",
}
]
}
)
source = payload["sources"][0]
assert source["feed_urls"] == ["https://example.com/a.xml", "https://example.com/b.xml"]
assert [feed["url"] for feed in source["feeds"]] == ["https://example.com/a.xml", "https://example.com/b.xml"]
assert [feed["id"] for feed in source["feeds"]] == ["feed-1", "feed-2"]
assert all(feed["default_category"] == "business" for feed in source["feeds"])
def test_builtin_news_source_legacy_directory_url_is_repaired():
payload = normalize_earth_news_sources_payload(
{
"sources": [
{
"id": "36kr",
"name": "36氪",
"region": "asia-pacific",
"source_type": "rss",
"homepage_url": "https://www.36kr.com/",
"feed_url": "https://www.36kr.com/rss-center",
"feed_urls": ["https://www.36kr.com/rss-center"],
"feeds": [
{
"id": "feed-1",
"name": "36氪",
"url": "https://www.36kr.com/rss-center",
"type": "rss",
"enabled": True,
"default_category": "business",
}
],
"default_category": "business",
}
]
}
)
source = payload["sources"][0]
feed_urls = {feed["url"] for feed in source["feeds"]}
assert source["homepage_url"] == "https://www.36kr.com/"
assert source["feed_directory_url"] == "https://www.36kr.com/rss-center"
assert "https://www.36kr.com/rss-center" not in feed_urls
assert {
"https://36kr.com/feed",
"https://36kr.com/feed-article",
"https://36kr.com/feed-newsflash",
"https://36kr.com/feed-moment",
}.issubset(feed_urls)
def test_builtin_news_source_without_feed_children_gets_explicit_defaults():
payload = normalize_earth_news_sources_payload(
{
"sources": [
{
"id": "ebrun",
"name": "亿邦动力",
"region": "asia-pacific",
"source_type": "rss",
"homepage_url": "https://www.ebrun.com/",
"feed_url": "https://www.ebrun.com/rss/news_b2c.xml",
"feed_urls": ["https://www.ebrun.com/rss/news_b2c.xml"],
"default_category": "ecommerce",
}
]
}
)
source = payload["sources"][0]
feed_urls = {feed["url"] for feed in source["feeds"]}
assert source["feed_directory_url"] == "https://www.ebrun.com/rss/"
assert "https://www.ebrun.com/rss/" not in feed_urls
assert {
"https://www.ebrun.com/rss/news_b2c.xml",
"https://www.ebrun.com/rss/news_b2b.xml",
"https://www.ebrun.com/rss/news_retail.xml",
"https://www.ebrun.com/rss/news_o2o.xml",
"https://www.ebrun.com/rss/news_service.xml",
"https://www.ebrun.com/rss/news_data.xml",
"https://www.ebrun.com/rss/news_policy.xml",
}.issubset(feed_urls)
def test_builtin_fetchable_source_saved_as_reference_is_repaired():
payload = normalize_earth_news_sources_payload(
{
"sources": [
{
"id": "stats-china-online-retail",
"name": "国家统计局数据发布",
"region": "asia-pacific",
"source_type": "reference",
"enabled": False,
"homepage_url": "https://www.stats.gov.cn/sj/zxfb/",
"feed_url": "https://www.stats.gov.cn/sj/zxfb/",
"default_category": "ecommerce",
}
]
}
)
source = payload["sources"][0]
assert source["source_type"] == "rss"
assert source["enabled"] is True
assert source["priority"] == 19
assert source["source_tags"] == ["official_data", "ecommerce", "retail", "china"]
assert source["default_category"] == "ecommerce"
assert source["importance_weight"] == 36
assert source["feed_directory_url"] == ""
assert source["feeds"] == [
{
"id": "release",
"name": "数据发布",
"url": "https://www.stats.gov.cn/sj/zxfb/rss.xml",
"type": "rss",
"region": "asia-pacific",
"enabled": True,
"default_category": "ecommerce",
"tags": [],
"priority": 1,
}
]
def test_legacy_google_sources_merge_into_google_news_source():
payload = normalize_earth_news_sources_payload(
{
"sources": [
{
"id": "global-scan",
"name": "Global Monitor / World",
"region": "global",
"source_type": "aggregated",
"feed_url": "https://news.google.com/rss/search?q=world",
"homepage_url": "https://news.google.com/",
},
{
"id": "google-europe",
"name": "Global Monitor / Europe",
"region": "europe",
"source_type": "aggregated",
"feed_url": "https://news.google.com/rss/search?q=europe",
"homepage_url": "https://news.google.com/",
},
]
}
)
sources_by_id = {source["id"]: source for source in payload["sources"]}
assert "global-scan" not in sources_by_id
assert "google-europe" not in sources_by_id
assert "google-news" in sources_by_id
assert {feed["id"] for feed in sources_by_id["google-news"]["feeds"]} == {
"world",
"americas",
"europe",
"middle-east-africa",
"asia-pacific",
}
def test_feed_child_default_category_overrides_source_default():
source = NewsFeedSource(
id="multi-feed",
name="Multi Feed",
region="global",
feed_url="https://example.com/source.xml",
homepage_url="https://example.com",
default_category="business",
)
feed = NewsFeedEndpoint(
id="ecommerce-feed",
name="Ecommerce Feed",
url="https://example.com/ecommerce.xml",
default_category="ecommerce",
)
xml = """
<rss>
<channel>
<item>
<title>Quarterly results released</title>
<description>Company update.</description>
<link>https://example.com/results</link>
</item>
</channel>
</rss>
"""
items = _parse_feed_entries(xml, source, feed=feed)
assert items[0].feed_id == "ecommerce-feed"
assert items[0].feed_name == "Ecommerce Feed"
assert items[0].feed_default_category == "ecommerce"
assert items[0].category == "ecommerce"
@pytest.mark.asyncio
async def test_fetch_source_only_requests_enabled_feed_children(monkeypatch):
source = NewsFeedSource(
id="multi-feed",
name="Multi Feed",
region="global",
feed_url="https://example.com/source.xml",
homepage_url="https://example.com",
feeds=(
NewsFeedEndpoint(id="enabled", name="Enabled", url="https://example.com/enabled.xml", enabled=True),
NewsFeedEndpoint(id="disabled", name="Disabled", url="https://example.com/disabled.xml", enabled=False),
),
)
calls = []
async def fake_fetch_single(_client, feed_source, feed, *, config_payload=None):
calls.append(feed.id)
item = ParsedNewsItem(
id=f"{feed_source.id}:{feed.id}:1",
title="Fetched story",
summary="Fetched summary",
url=f"https://example.com/{feed.id}",
source="Example",
feed_name=feed.name,
feed_region="global",
homepage_url="https://example.com",
published_at=None,
feed_id=feed.id,
)
return feed_source, [item], None, {"source_id": feed_source.id, "feed_id": feed.id, "ok": True, "status": "ok", "item_count": 1, "count": 1}
monkeypatch.setattr("app.services.earth_news._fetch_single_feed_url", fake_fetch_single)
source_result, items, error, health = await _fetch_source(object(), source)
assert source_result.id == "multi-feed"
assert calls == ["enabled"]
assert error is None
assert [item.feed_id for item in items] == ["enabled"]
assert health["ok"] is True
assert [result["feed_id"] for result in health["feed_results"]] == ["enabled"]
@pytest.mark.asyncio
async def test_fetch_source_filters_google_feed_children_by_active_region(monkeypatch):
source = NewsFeedSource(
id="google-news",
name="Google News",
region="global",
feed_url="https://news.google.com/rss",
homepage_url="https://news.google.com/",
source_type="aggregated",
feeds=(
NewsFeedEndpoint(id="world", name="全球", url="https://example.com/world.xml", type="aggregated", region="global"),
NewsFeedEndpoint(id="europe", name="欧洲", url="https://example.com/europe.xml", type="aggregated", region="europe"),
NewsFeedEndpoint(id="americas", name="美洲", url="https://example.com/americas.xml", type="aggregated", region="americas"),
),
)
calls = []
async def fake_fetch_single(_client, feed_source, feed, *, config_payload=None):
calls.append(feed.id)
item = ParsedNewsItem(
id=f"{feed_source.id}:{feed.id}:1",
title=f"{feed.name} headline",
summary="Fetched summary",
url=f"https://example.com/{feed.id}",
source="Example",
feed_name=feed.name,
feed_region=feed.region,
homepage_url="https://example.com",
published_at=None,
feed_id=feed.id,
)
return feed_source, [item], None, {"source_id": feed_source.id, "feed_id": feed.id, "ok": True, "status": "ok", "item_count": 1, "count": 1}
monkeypatch.setattr("app.services.earth_news._fetch_single_feed_url", fake_fetch_single)
_source_result, items, error, health = await _fetch_source(object(), source, active_region="europe")
assert error is None
assert calls == ["world", "europe"]
assert [item.feed_region for item in items] == ["global", "europe"]
assert [result["feed_id"] for result in health["feed_results"]] == ["world", "europe"]
def test_parse_rdf_rss_items_with_namespaces():
source = NewsFeedSource(
id="dw-top",
name="DW Top Stories",
region="europe",
feed_url="https://rss.dw.com/rdf/rss-en-top",
homepage_url="https://www.dw.com/en/top-stories/s-9097",
)
xml = """
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns="http://purl.org/rss/1.0/">
<item rdf:about="https://example.com/dw">
<title>German retail sales rise</title>
<link>https://example.com/dw</link>
<description>Retail summary</description>
</item>
</rdf:RDF>
"""
items = _parse_feed_entries(xml, source)
assert len(items) == 1
assert items[0].title == "German retail sales rise"
def test_news_classification_marks_ecommerce_and_importance():
source = NewsFeedSource(
id="ebrun",
name="亿邦动力",
region="asia-pacific",
feed_url="https://www.ebrun.com/rss/",
homepage_url="https://www.ebrun.com/",
source_tags=("business_news", "ecommerce", "china"),
default_category="ecommerce",
importance_weight=14,
)
item = ParsedNewsItem(
id="ebrun:test",
title="跨境电商平台 GMV 同比增长,物流履约效率提升",
summary="订单量和网上零售额继续增长。",
url="https://example.com/ecommerce",
source="亿邦动力",
feed_name="亿邦动力",
feed_region="asia-pacific",
homepage_url="https://www.ebrun.com/",
published_at=None,
)
apply_news_classification(item, source)
assert item.category == "ecommerce"
assert "cross_border_ecommerce" in item.item_tags
assert "logistics_fulfillment" in item.item_tags
assert item.importance_level in {"high", "critical"}
assert "命中电商数据指标" in item.importance_reasons
@pytest.mark.asyncio
async def test_enrich_items_with_target_locations_uses_ai_and_geocode(monkeypatch):
item = ParsedNewsItem(
@@ -338,8 +817,8 @@ async def test_earth_news_payload_returns_anchor_items_and_enqueues_location_job
published_at=datetime(2026, 5, 15, 3, 0, tzinfo=UTC),
)
async def fake_fetch_source(_client, feed_source):
return feed_source, [item], None
async def fake_fetch_source(_client, feed_source, **_kwargs):
return feed_source, [item], None, {"source_id": feed_source.id, "ok": True, "status": "ok", "count": 1}
async def fake_get_cached_target_location_patch(_item_id):
return None
@@ -402,7 +881,7 @@ async def test_earth_news_payload_uses_fresh_database_items_without_rss(monkeypa
async def fake_get_earth_news_freshness(_db, *, active_region):
return 12, datetime.now(UTC)
async def fake_list_earth_news_items(_db, *, active_region, limit):
async def fake_list_earth_news_items(_db, *, active_region, limit, categories=None):
assert limit == 12
return [item]
@@ -452,10 +931,10 @@ async def test_earth_news_payload_keeps_current_items_and_all_cruise_items(monke
async def fake_get_earth_news_freshness(_db, *, active_region):
return 12, datetime.now(UTC)
async def fake_list_earth_news_items(_db, *, active_region, limit):
async def fake_list_earth_news_items(_db, *, active_region, limit, categories=None):
return [current_item]
async def fake_list_earth_news_cruise_items(_db, *, limit):
async def fake_list_earth_news_cruise_items(_db, *, limit, categories=None):
return [current_item, cruise_item]
async def fake_enqueue_target_location_job(_payload, **_kwargs):
@@ -477,6 +956,89 @@ async def test_earth_news_payload_keeps_current_items_and_all_cruise_items(monke
assert payload["cruise_items"][1]["region"] == "asia-pacific"
@pytest.mark.asyncio
async def test_earth_news_payload_passes_region_and_category_filters_to_store(monkeypatch):
class FakeDb:
execute = object()
captured = {}
item = ParsedNewsItem(
id="db:business",
title="Business story",
summary="Business summary",
url="https://example.com/business",
source="Stored Source",
feed_name="Stored Feed",
feed_region="europe",
homepage_url="https://example.com",
published_at=datetime(2026, 5, 15, 3, 0, tzinfo=UTC),
category="business",
)
async def fake_get_earth_news_freshness(_db, *, active_region):
captured["freshness_region"] = active_region
return 12, datetime.now(UTC)
async def fake_list_earth_news_items(_db, *, active_region, limit, categories=None, source_ids=None):
captured["items_region"] = active_region
captured["items_categories"] = categories
captured["items_source_ids"] = source_ids
return [item]
async def fake_list_earth_news_cruise_items(_db, *, limit, categories=None, source_ids=None):
captured["cruise_categories"] = categories
captured["cruise_source_ids"] = source_ids
return [item]
async def fake_enqueue_target_location_job(_payload, **_kwargs):
return True
monkeypatch.setattr("app.services.earth_news_store.get_earth_news_freshness", fake_get_earth_news_freshness)
monkeypatch.setattr("app.services.earth_news_store.list_earth_news_items", fake_list_earth_news_items)
monkeypatch.setattr("app.services.earth_news_store.list_earth_news_cruise_items", fake_list_earth_news_cruise_items)
monkeypatch.setattr("app.services.earth_news_queue.enqueue_target_location_job", fake_enqueue_target_location_job)
monkeypatch.setattr("app.services.earth_news._fetch_rss_items_for_sources", lambda _sources: (_ for _ in ()).throw(AssertionError("fresh database items should not fetch RSS")))
payload = await get_earth_news_payload(
lat=35.0,
lon=-100.0,
region="europe",
categories={"business", "ecommerce"},
db=FakeDb(),
)
assert captured["freshness_region"] == "europe"
assert captured["items_region"] == "europe"
assert captured["items_categories"] == {"business", "ecommerce"}
assert captured["items_source_ids"] is None
assert captured["cruise_categories"] == {"business", "ecommerce"}
assert captured["cruise_source_ids"] is None
assert payload["filters"] == {
"region": "europe",
"categories": ["business", "ecommerce"],
"sources": [],
"limit": 12,
"locale": "zh-CN",
}
assert payload["items"][0]["category"] == "business"
@pytest.mark.asyncio
async def test_news_source_test_treats_type_reference_as_non_fetching():
result = await run_news_source_config_test(
{
"id": "reference-only",
"name": "Reference Only",
"type": "reference",
"feed_url": "https://example.com",
}
)
assert result["ok"] is False
assert result["health"]["status"] == "reference"
assert "不参与 RSS/Atom 抓取" in result["error"]
@pytest.mark.asyncio
async def test_earth_news_payload_initializes_empty_database_from_rss(monkeypatch):
db = object()
@@ -512,14 +1074,14 @@ async def test_earth_news_payload_initializes_empty_database_from_rss(monkeypatc
async def fake_get_earth_news_freshness(_db, *, active_region):
return 0, None
async def fake_fetch_rss_items_for_sources(_sources):
return [item], []
async def fake_fetch_rss_items_for_sources(_sources, **_kwargs):
return [item], [], {"test-feed": {"source_id": "test-feed", "ok": True, "status": "ok", "count": 1}}
async def fake_upsert_earth_news_items(_db, items):
upserted.extend(items)
return len(items)
async def fake_list_earth_news_items(_db, *, active_region, limit):
async def fake_list_earth_news_items(_db, *, active_region, limit, categories=None):
return [item]
async def fake_enqueue_target_location_job(payload, **_kwargs):
@@ -568,14 +1130,14 @@ async def test_earth_news_payload_supplements_stale_database_items(monkeypatch):
async def fake_get_earth_news_freshness(_db, *, active_region):
return 12, datetime(2026, 5, 14, 3, 0, tzinfo=UTC)
async def fake_fetch_rss_items_for_sources(_sources):
async def fake_fetch_rss_items_for_sources(_sources, **_kwargs):
fetched.append(True)
return [old_item], []
return [old_item], [], {"stored": {"source_id": "stored", "ok": True, "status": "ok", "count": 1}}
async def fake_upsert_earth_news_items(_db, items):
return len(items)
async def fake_list_earth_news_items(_db, *, active_region, limit):
async def fake_list_earth_news_items(_db, *, active_region, limit, categories=None):
return [old_item]
async def fake_enqueue_target_location_job(_payload, **_kwargs):
@@ -630,8 +1192,8 @@ async def test_earth_news_payload_merges_cached_location_patch(monkeypatch):
},
}
async def fake_fetch_source(_client, feed_source):
return feed_source, [item], None
async def fake_fetch_source(_client, feed_source, **_kwargs):
return feed_source, [item], None, {"source_id": feed_source.id, "ok": True, "status": "ok", "count": 1}
async def fake_get_cached_target_location_patch(_item_id):
return cached_patch
@@ -697,8 +1259,8 @@ async def test_earth_news_payload_requeues_cached_failed_localization(monkeypatc
}
enqueued = []
async def fake_fetch_source(_client, feed_source):
return feed_source, [item], None
async def fake_fetch_source(_client, feed_source, **_kwargs):
return feed_source, [item], None, {"source_id": feed_source.id, "ok": True, "status": "ok", "count": 1}
async def fake_get_cached_target_location_patch(_item_id):
return cached_patch

View File

@@ -9,6 +9,8 @@ import pytest
from app.core.logging import PlanetContextFilter, PlanetFormatter, get_logger
from app.core.request_context import set_request_id
from app.services import business_logs
from app.services import persistent_logs
from app.models.system_log import ObservabilityEvent, ObservabilityEventGroup
def _capture_output(callback):
@@ -98,6 +100,86 @@ def test_business_context_redacts_nested_sensitive_values():
assert context["nested"]["safe"] == "visible"
def test_observability_fingerprint_normalizes_hls_fragments():
first = persistent_logs.build_observability_fingerprint(
source="earth-client",
service="earth",
module="tv",
category="hls-proxy",
event="hls.fragment.failed",
message="HLS 分片加载失败: index_5_9086220.ts?m=1725933270",
context={"status_code": 502},
)
second = persistent_logs.build_observability_fingerprint(
source="earth-client",
service="earth",
module="tv",
category="hls-proxy",
event="hls.fragment.failed",
message="HLS 分片加载失败: index_5_9086361.ts?m=1725934270",
context={"status_code": 502},
)
assert first == second
@pytest.mark.asyncio
async def test_record_observability_event_updates_group_count(monkeypatch):
events: list[ObservabilityEvent] = []
groups: dict[str, ObservabilityEventGroup] = {}
class FakeSession:
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
def add(self, item):
if isinstance(item, ObservabilityEvent):
events.append(item)
elif isinstance(item, ObservabilityEventGroup):
groups[item.fingerprint] = item
async def get(self, model, key):
if model is ObservabilityEventGroup:
return groups.get(key)
return None
async def commit(self):
return None
monkeypatch.setattr(persistent_logs, "async_session_factory", lambda: FakeSession())
await persistent_logs.record_observability_event(
source="earth-client",
level="error",
service="earth",
module="tv",
category="hls-proxy",
event="hls.fragment.failed",
message="HLS 分片加载失败: index_5_9086220.ts?m=1725933270",
context={"status_code": 502},
occurrence_count=2,
)
await persistent_logs.record_observability_event(
source="earth-client",
level="error",
service="earth",
module="tv",
category="hls-proxy",
event="hls.fragment.failed",
message="HLS 分片加载失败: index_5_9086361.ts?m=1725934270",
context={"status_code": 502},
occurrence_count=1,
)
assert len(events) == 2
assert len(groups) == 1
group = next(iter(groups.values()))
assert group.count == 3
@pytest.mark.asyncio
async def test_emit_business_log_persists_sanitized_system_event(monkeypatch):
events = []

View File

@@ -2,8 +2,10 @@ from __future__ import annotations
import json
from datetime import UTC, datetime
from pathlib import Path
from app.models.system_log import SystemLog
from app.services import system_logs
@@ -40,10 +42,10 @@ def test_read_log_snapshot_uses_structured_buffer_timestamp_level_and_search(mon
{
"earth-client": system_logs.LogSource(
source_id="earth-client",
name="Earth 浏览器端",
name="智能星球浏览器端",
kind="buffer",
location="redis://planet:system_logs:earth-client",
description="Earth 浏览器端上报日志",
description="智能星球浏览器端上报日志",
category="client",
buffer_key=system_logs.get_buffer_log_key("earth-client"),
)
@@ -156,6 +158,52 @@ def test_append_buffer_log_persists_normalized_level(monkeypatch):
assert payload["message"] == "feed delayed"
def test_list_log_sources_includes_admin_client(monkeypatch):
fake_redis = FakeRedis()
monkeypatch.setattr(system_logs, "redis_client", fake_redis)
sources = system_logs.list_log_sources()
admin_source = next(item for item in sources if item["source_id"] == "admin-client")
assert admin_source["kind"] == "buffer"
assert admin_source["category"] == "client"
def test_read_log_events_returns_stable_cursors(tmp_path: Path, monkeypatch):
log_path = tmp_path / "backend.log"
log_path.write_text(
"\n".join(
[
"2026-04-22 08:00:00 INFO service booted",
"2026-04-22 08:01:00 ERROR service failed",
]
),
encoding="utf-8",
)
monkeypatch.setattr(
system_logs,
"LOG_SOURCES",
{
"backend": system_logs.LogSource(
source_id="backend",
name="后端服务",
kind="file",
location=str(log_path),
description="测试文件日志",
category="service",
)
},
)
events = system_logs.read_log_events("backend", 50, level="error")
assert events is not None
assert len(events) == 1
assert events[0].source_id == "backend"
assert events[0].cursor.startswith("backend:")
assert events[0].line.endswith("ERROR service failed")
def test_infer_log_level_prefers_leading_prefix_over_query_string():
line = 'INFO: 127.0.0.1 - "GET /api/v1/system/logs/backend?limit=200&level=error&levels=error HTTP/1.1" 200 OK'
@@ -215,3 +263,22 @@ def test_read_log_snapshot_strips_nul_bytes_from_file_lines(tmp_path: Path, monk
"ERROR: bind failed",
"2026-04-23 23:41:32 INFO service=backend message=request served",
]
def test_database_system_log_search_matches_context_key_value_aliases():
record = SystemLog(
id=2218,
occurred_at=datetime(2026, 5, 28, 9, 14, 50, tzinfo=UTC),
source="backend",
service="collector",
module="app.services.collectors.base",
event="collector.run.failed",
level="error",
message="Collector run failed",
context={"collector_name": "celestrak_tle", "datasource_id": 20, "task_id": 26906},
)
event = system_logs._database_event_from_system_record(record)
assert system_logs.event_matches_search(event, "task_id=26906")
assert system_logs.event_matches_search(event, "datasource_id=20")

View File

@@ -0,0 +1,35 @@
from app.api.v1.tv import _rewrite_hls_uri_attributes, _should_strip_hls_metadata_line
def test_rewrite_hls_uri_attributes_rewrites_subtitle_manifest_url():
line = '#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",URI="index_3_0.m3u8"'
rewritten = _rewrite_hls_uri_attributes(
line,
base_url="https://example.com/live/master.m3u8",
)
assert 'URI="/api/v1/tv/proxy?url=https%3A%2F%2Fexample.com%2Flive%2Findex_3_0.m3u8"' in rewritten
def test_rewrite_hls_uri_attributes_rewrites_absolute_uri():
line = '#EXT-X-I-FRAME-STREAM-INF:BANDWIDTH=1234,URI="https://cdn.example.com/live/iframe.m3u8"'
rewritten = _rewrite_hls_uri_attributes(
line,
base_url="https://example.com/live/master.m3u8",
)
assert 'URI="/api/v1/tv/proxy?url=https%3A%2F%2Fcdn.example.com%2Flive%2Fiframe.m3u8"' in rewritten
def test_strip_hls_subtitle_media_metadata():
line = '#EXT-X-MEDIA:TYPE=SUBTITLES,GROUP-ID="subs",NAME="English",URI="index_3_0.m3u8"'
assert _should_strip_hls_metadata_line(line) is True
def test_keep_hls_audio_media_metadata():
line = '#EXT-X-MEDIA:TYPE=AUDIO,GROUP-ID="audio",NAME="English",URI="audio.m3u8"'
assert _should_strip_hls_metadata_line(line) is False

View File

@@ -8,6 +8,72 @@ This project follows the repository versioning rule:
- `improvement` -> `+0.0.1`bugfix + 小功能混合)
- `bugfix` -> `+0.0.1`
## [0.69.0] — 2026-06-03
Released: 2026-06-03
### Highlights
- 新增 Earth 新闻源治理能力,支持多 Feed 子项、源属性标签、新闻类型过滤、重要度规则和健康测试。
- 新增观测日志聚合视图,按 fingerprint 汇总 Earth、Admin 和服务端重复运行时事件,并保留原始发生明细。
- 改进 TV/HLS 播放恢复和代理重写,降低字幕、分片和源站波动导致的直播不可用噪声。
### Added / Fixed / Improved
- Earth 新闻面板和 UE 端统一通过 `/api/v1/news/earth-feed` 使用 `categories``locale` 服务端过滤Web 端新闻类型偏好仅保存在当前浏览器。
- 控制台日志页新增重复统计、原始日志和审计日志模式,前端上报器会合并短窗口内的重复错误并提交 `occurrence_count`
- AI Provider / 服务端运行时可通过受保护的 observability ingest 入口写入结构化事件。
- 新闻源文档新增中英文配置说明,并补齐 Earth 前端、控制台日志和公开 Docs 索引。
---
## [0.68.1] — 2026-05-28
Released: 2026-05-28
### Highlights
- 修复 CelesTrak active 未更新窗口下清库后无法恢复的问题fallback 会优先复用本地有效 group 缓存。
- 修复数据源任务队列“查看日志”无法按 `task_id=...` 命中数据库结构化日志的问题。
### Added / Fixed / Improved
- CelesTrak fallback group 列表改为公开可用分组,移除失效 group并在没有 active 缓存时仍可从本地 group 缓存恢复采集。
- 数据库日志搜索补充 JSON context 的 `key=value` 别名,支持 `task_id=26906``datasource_id=20` 这类控制台跳转查询。
- 补充 CelesTrak 缓存边界、数据源任务日志跳转和运维恢复说明的中英文文档。
---
## [0.68.0] — 2026-05-28
Released: 2026-05-28
### Highlights
- 新增数据源任务队列的实时指标校准和批量删除进度,让大表清理、取消和完成状态在控制台中可感知。
- 新增智能星球 interactable 可插拔聚类策略,支持稳定 3D 球面聚类、动态屏幕聚类和 250% 以上自动散开。
- 改进新设备启动流程,`planet.sh` 会在启动前同步前端依赖,避免缺失依赖导致控制台动态导入失败。
### Added / Fixed / Improved
- 数据源列表改为中文记录数指标,并对 AIS 大表使用统计估算 + 单条详情精确校准,降低首次加载成本。
- 数据删除任务改为分批删除并广播进度,清理 AIS 衍生表后自动 `ANALYZE`,同时修复取消中任务恢复和状态文案。
- Earth BGP、算力中心和 interactable 图层默认使用 `stable-spherical` 聚类,船舶实时层保留 `dynamic-screen`
- 新增中英文 Earth interactable clustering 文档,并补充采集队列、后端删除语义和前端依赖同步说明。
---
## [0.67.0] — 2026-05-27
Released: 2026-05-27
### Highlights
- 新增控制台日志实时跟随与前端运行时错误上报,帮助在控制台内直接排查 Admin / Earth 客户端异常。
- 重构智能星球 Interactable 聚合和 wheel 缩放输入,保持真实地理锚点稳定,同时让鼠标滚轮和触控板拥有各自合适的缩放手感。
### Added / Fixed / Improved
- 新增 `/ws` 日志 tail 通道、数据库/文件日志统一事件读取,以及 Admin `error` / `unhandledrejection` / React ErrorBoundary 上报链路。
- 优化控制台日志页状态颜色、跟随体验和运行时错误展示,并将 Admin 本地工具模块从 `lib` 命名迁移为局部 `utils`
- 修复智能星球国界/高清材质壳半径对齐、Interactable cluster 圆点显示、缩放目标累积和触控板连续缩放问题。
- 更新 `planet.sh``.gitignore`,避免新环境构建污染锁文件并移除前端 `lib` 目录特殊放行。
- 补充智能星球前端、渲染层级、控制台状态与运行时日志文档。
---
## [0.66.3] — 2026-05-26
Released: 2026-05-26

View File

@@ -2,7 +2,7 @@
## 背景
状态Phase 1 已经开始落地Phase 2 的 BGP 事件 / 观测站迁移和 Phase 3 的算力中心迁移也已完成。`frontend/public/earth/js/interactable.js` 已新增AIS 船只、BGP 事件、BGP 观测站和算力中心图层已经改为通过 `createInteractableLayer()` 使用通用批量 `Points`、hover / locked overlay、默认 glow、状态更新、asset icon 预加载、屏幕空间 picking、固定 / 距离缩放和跨 Interactable 同坐标避让。登陆点因 `THREE.Points` 边缘深度裁切和贴地层级要求,已退回专用 `THREE.Sprite` 黄色球路径,并与海缆同高度同 renderOrder。后续阶段聚焦把可复用的扩圈 / 雷达扇形动画正式沉淀成 `animations` 扩展。
状态Phase 1 已经开始落地Phase 2 的 BGP 事件 / 观测站迁移和 Phase 3 的算力中心迁移也已完成。`frontend/public/earth/js/interactable.js` 已新增AIS 船只、BGP 事件、BGP 观测站和算力中心图层已经改为通过 `createInteractableLayer()` 使用通用批量 `Points`、hover / locked overlay、默认 glow、状态更新、asset icon 预加载、屏幕空间 picking、固定 / 距离缩放和跨 Interactable 同坐标关系元数据。登陆点因 `THREE.Points` 边缘深度裁切和贴地层级要求,已退回专用 `THREE.Sprite` 黄色球路径,并与海缆同高度同 renderOrder。后续阶段聚焦把可复用的扩圈 / 雷达扇形动画正式沉淀成 `animations` 扩展。
当前实现说明和接入示例见:
@@ -100,9 +100,10 @@ createInteractableLayer({
| `picking.throttleMs` | `100` | `80` | hover picking 节流。 |
| `picking.skipWhileDragging` | `true` | `true` | 拖动和惯性期间跳过 hover picking。 |
| `zIndexPolicy` | `"surface-icon"` | `"surface-icon"` | 预设层级策略,避免每个业务图层手写高度和 renderOrder。 |
| `avoidance.enabled` | `true / false` | `true` | 是否参与跨 Interactable 的同坐标避让。默认开启,同一经纬度下的图标会沿地表切平面小幅排开,方便辨认和选择。 |
| `avoidance.radius` | `number` | `1.1` | 同坐标避让的第一圈半径,单位为地球本地坐标单位。 |
| `avoidance.enabled` | `true / false` | `true` | 是否记录跨 Interactable 的同坐标关系。该配置只生成 overlap 元数据,不允许移动真实 marker 坐标。 |
| `avoidance.precision` | `number` | `4` | 经纬度归并精度,默认约等于只处理几乎完全重叠的图标。 |
| `cluster.enabled` | `true / false` | 跟随 avoidance | 是否参与跨 Interactable 的当前帧屏幕重叠合并。只影响显示,不改变业务坐标。 |
| `cluster.maxMarkersPerDot` | `number` | `14` | 单个聚合圆点的对象上限;超过后按屏幕局部邻近关系拆成多个较小圆点,避免一个点过大或跨区域串联。 |
| `legend` | `{ label, color, shape }[]` | `[]` | 可选图例声明,业务层也可以继续自己导出。 |
| `metadata` | object | `{}` | 业务扩展数据,不参与渲染但参与 tooltip / info-card / search。 |

View File

@@ -21,7 +21,9 @@ This is the current Intelligent Planet documentation entry point. Docs are organ
- [Earth Satellite Footprint Policy](/home/ray/dev/linkong/planet/docs/technical/en/earth-satellite-footprint-policy.md): satellite footprint display boundaries and strategy
- [BGP Context](/home/ray/dev/linkong/planet/docs/technical/en/earth-bgp-context.md): BGP rendering, aggregation, and collector implementation in Earth
- [Earth Interactable Usage](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-usage.md): `Interactable` API, lifecycle, and integration examples
- [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md): pluggable cluster strategies, stable spherical clustering, and dynamic screen clustering boundaries
- [Earth Toolbar and Overlay Coordination](/home/ray/dev/linkong/planet/docs/technical/en/earth-toolbar-overlay-coordination.md): close matrix for toolbar buttons, search, settings, news, and layer overlays
- [Intelligent Planet News Source Configuration](/home/ray/dev/linkong/planet/docs/technical/en/earth-news-sources.md): default sources, feed children, source property tags, content categories, importance rules, and configuration APIs
## Frontend Implementation

View File

@@ -75,6 +75,10 @@ async def run(self, db):
Manual trigger, data clearing, and cache clearing now enter the PostgreSQL data job queue. `collection_tasks` remains the task ledger. Collectors only own `fetch -> transform -> save`; the `data_jobs.py` worker claims `collect` / `clear_data` / `clear_cache` / `earth_refresh` jobs and writes progress back. Earth layer refresh relationships live in `earth_layer_adapters.py`; do not hand-code cache invalidation or WebSocket broadcasts inside individual collectors or buttons.
Data deletion runs in batches so AIS-scale tables are not locked by one huge statement. A `clear_data` job clears `collected_data`, then source-specific AIS derived tables, and broadcasts `records_processed` as it goes; the console queue renders only user-facing text such as `Deleting data` and `Delete complete`, while internal table names remain in logs and raw task details. After AIS cleanup, the backend runs `ANALYZE ais_raw_observations` so datasource-list estimates converge quickly. The datasource directory uses PostgreSQL statistics for AIS record counts by default to avoid a cold-start `count(*)`; opening a single datasource detail row requests the exact count for that source.
The CelesTrak TLE collector prefers the complete `active` catalog. If CelesTrak returns the "GP data has not updated" HTTP 403, the backend first reuses the active raw download cache under `$PLANET_CACHE_DIR/downloads/celestrak`; if that cache is missing, it enters fallback group mode. Fallback group mode uses only currently valid public CelesTrak groups, including `starlink`, `gps-ops`, `galileo`, `glo-ops`, `beidou`, `geo`, `iridium-next`, `stations`, `visual`, `weather`, `science`, `cubesat`, `amateur`, and `last-30-days`. Disaster-recovery fallback prefers valid local group caches before touching the network, so small-group update windows or unreliable HEAD metadata do not incorrectly fail recovery. Console `Clear Data` and `Clear Cache` jobs only touch database rows, Earth layer cache, and dashboard cache; they do not remove this raw download cache.
## III. Collector List
| Collector | Data type | Content | Frequency |

View File

@@ -75,6 +75,8 @@ This is currently the most critical UI control entry point for the Earth fronten
Earth settings are now grouped by `data-settings-tab` and `data-settings-tab-panel`. Desktop and mobile share the same category semantics: Runtime, Display, Panels, Motion, Shortcuts, and System. When adding a setting, first choose its category, then add the DOM, persistence field, and restore logic; do not keep growing one long undifferentiated panel.
The news category selector in Display reuses the same chip-selector pattern as Cruise Modules. It only filters news categories for the current browser on the Earth frontend. It does not toggle layers, basemap, boundaries, TV, data points, BGP, vessels, satellites, or compute centers; those remain owned by the layer panel, media panel, and admin configuration. `controls.js` persists only `shared.newsCategoryFilters` and broadcasts `earth:news-category-filters-change`; `news.js` sends the selected categories to `/api/v1/news/earth-feed?categories=...&locale=zh-CN`, so Web and UE clients share the same backend category filtering path.
Shortcut configuration is a device-local preference owned by `controls.js`: read, capture, enable/disable, and reset all stay in the Earth frontend. It should not be written to backend user settings and should not affect other browsers. New shortcuts must provide a default key, display label, disabled/enabled state, and reset path instead of being hard-coded only in a keydown handler.
### 4. UI and Status Messages
@@ -130,7 +132,7 @@ Responsibilities:
- Terrain tile fetch, decode, displacement, and shading
- Whole-globe land/ocean and border base overlays
The Earth surface is a stack of near-concentric shells, not a single mesh. The base sphere and HD texture overlay in `earth.js`, plus the land/ocean base in `country-boundaries.js`, need explicit radius separation. At far zoom, GPU depth precision drops; neighboring shells that are too close can z-fight and show black flicker blocks or snow. The current stable spacing is `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = 0.48`. When adding or adjusting whole-globe surface overlays, update [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md) and verify at 50% zoom.
The Earth surface is a stack of near-concentric shells, not a single mesh. The base sphere and HD texture overlay in `earth.js`, plus the land/ocean base in `country-boundaries.js`, need explicit radius separation. At far zoom, GPU depth precision drops; neighboring shells that are too close can z-fight and show black flicker blocks or snow. The current stable spacing is `landAltitudeOffset = 0.32` and `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` for the HD texture shell. Country borders, coastlines, claim lines, and country hover lines must use that same HD texture shell radius so they do not drift relative to the texture while the globe rotates. When adding or adjusting whole-globe surface overlays, update [Earth Render Layer Order](/home/ray/dev/linkong/planet/docs/technical/en/earth-render-layer-order.md) and verify at 50% zoom.
### 7. Layer Modules
@@ -313,6 +315,18 @@ If future cable, satellite, or news cruise is added, do not copy a new set of `m
- [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js)
- The business adapter pattern from [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
## View Control Feedback
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) owns the Earth zoom state, and every zoom entry point must ultimately call `setZoomLevel()` to write the camera distance. Do not write `camera.position.z` from other modules, or the zoom percentage, drag sensitivity, and Interactable clustering thresholds will diverge again.
Interactable clustering is selected per layer through `cluster.strategy`. `stable-spherical` uses discrete zoom bands and local 3D bucket clustering, so BGP, compute centers, and Earth interactables do not regroup while the globe rotates inside the same band. `dynamic-screen` keeps the projection-based behavior for high-frequency realtime layers such as vessels, and `none` disables clustering. Stable cluster dots stay rigidly aligned to their 3D centroid projection and do not participate in 2D avoidance. See [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md) for strategy configuration and tuning.
Wheel input has two paths. Traditional mouse wheels keep the 10% step and short animation, using `wheelZoomTarget` as the logical base for continuous wheel input. Trackpads and high-precision wheels use the pixel delta for continuous zoom and call `setZoomLevel()` directly instead of passing through the 10% stepped animation. The trackpad path also filters a short-window, old-direction residual delta after a real direction change so inertia tails do not pull a just-reversed zoom back in the previous direction.
The gesture capsule updates at most once every 90ms and fades after 760ms. It is view feedback, not data loading progress, and should not be written into layer loading state.
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) only calls `setZoomLevel()` and `showZoomStatusCapsule()` for pinch zoom and motion zoom. Mouse wheel and zoom-button capsule feedback should stay in `controls.js` so the same zoom feedback does not spread across modules.
## Recommended Change Approach
For future Earth changes:

View File

@@ -0,0 +1,84 @@
# Intelligent Planet Interactable Clustering
Earth interactable icons are managed by `createInteractableLayer`. Rendering still uses Three.js `Points`, and picking, hover, locked state, tooltips, and cruise focus still depend on marker `userData`; the clustering strategy only decides which markers are represented by a cluster dot.
## Strategies
`cluster.strategy` supports three modes:
- `stable-spherical`: stable spherical clustering. It clusters by local 3D positions on the globe and caches topology by zoom band. Rotation and small zoom changes within the same band only update projection and size. Use it for semi-static layers such as BGP, compute centers, and Earth interactables.
- `dynamic-screen`: dynamic screen-space clustering. This keeps the previous projection-based behavior and evaluates visible relationships per frame. Use it for high-frequency realtime layers such as vessels, or as a fallback.
- `none`: no clustering. Every marker is shown independently. Use it for low-count layers or precision-first views.
`cluster: false` is equivalent to `strategy: "none"`. If a layer enables clustering without declaring a strategy, it keeps the compatible `dynamic-screen` behavior.
## Stable Spherical
`stable-spherical` moves cluster identity from screen distance to globe distance:
- Each marker uses `icon_base_position` as its geographic anchor.
- By default, zoom levels above `2.5` force clustering off and show every marker as its original icon.
- The current zoom maps to a discrete band; rotation and small zoom changes inside the same band do not recompute topology.
- Clusters are recomputed only when the band, data revision, visibility, or filter state changes.
- A cluster centroid is computed from member 3D positions and normalized back to the globe shell, so the cluster dot stays rigidly aligned to its geographic center.
- Cluster dots do not participate in 2D avoidance, so screen-space repulsion cannot push them away from their real geographic projection.
- Band changes use a small hysteresis margin so zooming at a boundary does not repeatedly bounce between two bands.
- Newly created marker and cluster dots run a short scale + opacity ease. This is only a rendering transition; it does not change marker coordinates, picking objects, or locked state.
The stable strategy uses spherical bucket/hash neighbor lookup and must not use an all-pairs loop. Most frames only pay projection and material-size cost; topology cost is paid only on band or data changes.
## Configuration
```js
const computeCenterIconLayer = createInteractableLayer({
id: "computeCenters",
// ...
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
minCount: 2,
maxMarkersPerDot: 14,
transitionMs: 220,
bandHysteresis: 0.08,
disableAboveZoom: 2.5,
bands: [
{ key: "far", maxZoom: 1.7, distance: 15 },
{ key: "mid", maxZoom: 2.6, distance: 8 },
{ key: "near", maxZoom: 3.5, distance: 4 },
{ key: "detail", maxZoom: Infinity, distance: 0 },
],
},
});
```
Realtime layers can stay dynamic:
```js
const vesselIconLayer = createInteractableLayer({
id: "vessels",
// ...
cluster: {
strategy: "dynamic-screen",
enabled: true,
maxMarkersPerDot: 10,
},
});
```
## Tuning
- `distance` is the 3D globe-distance threshold and uses the same unit as `CONFIG.earthRadius`. Larger values cluster more aggressively.
- The farthest band usually uses a larger `distance`; the nearest detail band usually uses `0` to split clusters into original icons.
- `bandHysteresis` controls band-boundary stickiness. Too little can flicker near thresholds; too much can make band changes feel late.
- `transitionMs` controls cluster split/merge easing. Keep it around 160-260ms; longer durations can make realtime layers feel sluggish.
- `disableAboveZoom` controls the precision-view threshold. It defaults to `2.5`; above that zoom, no cluster dots are generated. Set it to `false` to disable the hard threshold.
- High-frequency realtime layers should prefer `dynamic-screen` so frequent data changes do not trigger stable topology recomputation.
- If a layer behaves poorly, switch it back to `dynamic-screen` or use `cluster: false`.
## Acceptance Checks
- Rotating the globe inside one zoom band should not cause clusters to flicker or regroup.
- Cluster dots should stay aligned with the globe-surface centroid and should not be pushed by avoidance.
- Zooming into the detail band should restore the layer's original icon textures and click behavior.
- Zoom levels above 250% should not show cluster dots.
- Realtime layers such as vessels should reflect incoming updates immediately.

View File

@@ -0,0 +1,161 @@
# Earth News Source Configuration
Earth situational news is served through `/api/v1/news/earth-feed`. News source configuration lives in `SystemSetting.category = "earth_news_sources"`; when no database configuration exists, the backend uses the built-in default sources as the fallback seed.
## Default Sources
The default set contains four groups:
- **News feeds**: BBC World, DW Top Stories, CNBC Business, BBC Business, Guardian Business, NPR Business, MarketWatch, TechCrunch, Retail Dive, PR Newswire Retail, 36Kr, Ebrun, and China NBS data releases.
- **Industry insight sources**: McKinsey Retail and Deloitte Retail.
- **Official data sources**: China NBS data releases, US Census Retail / E-Commerce, MOFCOM Data, MOFCOM e-commerce updates, and China e-commerce logistics index.
- **Lead sources**: BusinessWire Electronic Commerce; Google News is one aggregated source with feed children for global, Americas, Europe, Middle East / Africa, and Asia Pacific.
Config data sources are visible in Admin by default. If a source is not a stable RSS/Atom feed, it is kept disabled for automatic fetching until an administrator replaces it with a fetchable URL and enables it.
| Source | Type | Default state | Default category | Main tags | Purpose |
| --- | --- | --- | --- | --- | --- |
| BBC World | RSS | Enabled | Politics | `official_media`, `global` | Global public-news baseline |
| DW Top Stories | RSS | Enabled | Politics | `official_media`, `europe` | Europe and international baseline |
| CNBC Business | RSS | Enabled | Business | `business_news`, `us`, `global` | International business news |
| BBC Business / Guardian Business / NPR Business / MarketWatch | RSS | Enabled | Business / Finance | `business_news`, `finance` | UK / US business and finance baseline |
| TechCrunch / Retail Dive / PR Newswire Retail | RSS | Enabled | Technology / Business | `business_news`, `ecommerce`, `retail`, `press_release` | Technology, e-commerce, retail, and company announcements |
| 36Kr | RSS | Enabled | Business | `business_news`, `ecommerce`, `china` | China business, venture, and newsflash feeds; homepage is `https://www.36kr.com/`, the Feed Directory is `https://www.36kr.com/rss-center`, and feed children are the general, article, newsflash, and moment feeds |
| Ebrun | RSS | Enabled | E-commerce | `ecommerce`, `business_news`, `china`, `retail` | China e-commerce industry news; homepage is `https://www.ebrun.com/`, the Feed Directory is `https://www.ebrun.com/rss/`, and feed children are B2C, B2B, retail, O2O, service, data, and policy XML feeds |
| China NBS data releases | RSS | Enabled | E-commerce | `official_data`, `ecommerce`, `retail`, `china` | Official data release RSS; retail and online retail items are identified by category and importance rules |
| Google News | Aggregated | Enabled | Politics | `aggregated`, `low_stability` | One aggregated source with global, Americas, Europe, Middle East / Africa, and Asia Pacific feed children; lower priority than real RSS |
| BusinessWire Electronic Commerce | Reference | Disabled | E-commerce | `press_release`, `ecommerce`, `low_stability` | Corporate announcement leads |
| McKinsey Retail Insights | Reference | Disabled | Business | `industry_insight`, `retail` | Retail industry insight |
| Deloitte Retail | Reference | Disabled | Business | `industry_insight`, `retail` | Retail industry insight |
| US Census Retail / E-Commerce | Reference | Disabled | E-commerce | `official_data`, `ecommerce`, `retail`, `us` | US retail and e-commerce official data |
| MOFCOM Data | Reference | Disabled | Business | `official_data`, `china` | China commerce data |
| MOFCOM e-commerce updates | Reference | Disabled | E-commerce | `official_data`, `ecommerce`, `china` | China e-commerce policy and updates |
| China e-commerce logistics index | Reference | Disabled | E-commerce | `official_data`, `ecommerce`, `logistics`, `china` | Logistics fulfillment and e-commerce activity |
`Reference` means a reference link or future collector lead. It records a homepage, report page, or data page and does not participate in RSS/Atom fetching. This lets commercial and official sources enter Admin governance without letting non-feed pages break the live news feed.
The news source model has two levels:
- `source` is the brand or aggregator, such as 36Kr, Ebrun, Google News, or BBC.
- `homepage_url` is the source homepage, section page, or report page.
- `feed_directory_url` is the Feed Directory page, such as an RSS subscription center or feed index. It is for human inspection and is not fetched.
- `feeds` are the actual RSS, Atom, or Aggregated child entries under that source. Each feed child has `id / name / url / type / enabled / default_category / tags / priority`.
The backend iterates over every enabled feed child under the same source, fetches them independently, merges and deduplicates items, and writes per-feed diagnostics into `health.feed_results`. This is not a backup URL model: all four 36Kr subscription feeds, multiple Ebrun category XML feeds, and the five Google News regional RSS feeds can be enabled at the same time, and each feed can have its own default category and enabled state. HTML subscription-center pages belong in `feed_directory_url`, not in feed URLs. Every default enabled fetchable feed is tested item by item: RSS/Atom/Aggregated feeds must parse at least one item, while Reference sources only retain a reference URL and future collector lead.
Items that still remain Reference are not treated as broken feeds; no stable directly consumable RSS/Atom feed was verified:
- BusinessWire documents customizable RSS/Atom support, but the public pages do not expose a stable industry feed URL; the e-commerce industry page is kept as an announcement lead.
- McKinsey and Deloitte retail insight pages are report/article collections, not public RSS feeds.
- The US Census press-release RSS is reachable, but its items currently have empty links; the Quarterly E-Commerce page remains an official data reference.
- MOFCOM data and China e-commerce logistics index pages do not expose stable RSS feeds yet; they should become dedicated collectors or be replaced with administrator-provided fetchable feeds.
## Source Property Tags and News Categories
News sources have `source_tags`, shown in Admin as source property tags. They describe the source, not the media name and not the content category of an individual story:
- `official_data`
- `business_news`
- `ecommerce`
- `finance`
- `retail`
- `logistics`
- `industry_insight`
- `press_release`
- `china`, `global`, `us`
- `aggregated`, `low_stability`
Each news item has exactly one primary `category`. Defaults are politics, business, e-commerce, finance, sports, technology, military, disaster, energy, society, culture, and other. `item_tags` are item-level secondary tags, such as cross-border e-commerce, live commerce, retail data, logistics fulfillment, platform governance, AI, semiconductor, election, oil price, football, and supply chain.
The primary category is generated by a rule-based scorer over title, summary, and source text. If the rules do not match, the feed child default category is used first, then the source default category. AI enrichment does not block news display.
## Importance
Each item includes:
- `importance_score`
- `importance_level`
- `importance_reasons`
- `market_impact`
Official data, e-commerce metrics, major platforms, and numeric business signals increase importance. Press releases start with a lower baseline and rise only when they match stronger platform, amount, M&A, or regulatory signals.
## Configuration and Cache
`GET /api/v1/earth/news-sources` returns the default or saved configuration. `PUT /api/v1/earth/news-sources` saves it, increments `cache_version`, and clears the process region cache. `POST /api/v1/earth/news-sources/reset` restores defaults. `POST /api/v1/earth/news-sources/test` tests one RSS/Atom/Aggregated source without writing news items.
## Feed Query and Category Filtering
The Web Earth client and UE client both consume `GET /api/v1/news/earth-feed`. The endpoint supports server-side filtering, so clients do not need to fetch the full list and apply the primary category filter locally.
- `lat` / `lon`: infer the active region from the current view, used by the Web Earth client.
- `region`: explicitly select a region for UE or service integrations. Supported values include `global`, `americas`, `europe`, `asia-pacific`, and `middle-east-africa`. `global` is an aggregate view and can include every region; non-global regions include only their own region plus `global` sources.
- `categories`: comma-separated news category keys, for example `business,ecommerce`. Omit it when all categories are selected.
- `locale`: display locale, currently `zh-CN` or `en-US`, defaulting to `zh-CN`. Chinese RSS items are stored as Chinese source content and enriched with `en-US`; English RSS items are enriched with `zh-CN`.
Examples:
```http
GET /api/v1/news/earth-feed?region=europe&categories=business,ecommerce
GET /api/v1/news/earth-feed?lat=48&lon=10&categories=technology
GET /api/v1/news/earth-feed?region=global&categories=business,ecommerce&locale=zh-CN
```
Unknown category or locale values return `422` with the allowed values. The response includes `filters`, which confirms the region, category, and locale filters applied by the backend. `items` and `cruise_items` use the same category filter set.
The Web Earth category chips only store the current browser preference; changing them triggers a new API request. UE should pass its selected categories through the `categories` query parameter and does not need to perform the primary filtering itself.
Source testing only proves that a specific RSS/Atom/XML feed can be parsed. It does not mean those items have already been written to the news table or are visible in the current region/category view. Saving or resetting news sources increments the configuration version and clears cache; if an enabled feed has no recent stored items, the next `earth-feed` request supplements from RSS so newly enabled sources such as 36Kr and Ebrun are not masked by fresh Google News rows.
## Connectivity Monitoring
`POST /api/v1/earth/news-sources/test` tests one source and writes the result to `earth_news_sources.health[source_id]`. Normal RSS/Atom fetches update the same health map.
Health results include:
- `status`: `ok`, `empty`, `format_error`, `http_error`, `timeout`, `network_error`, or `reference`.
- `status_code`, `content_type`, `item_count`, `latency_ms`, `error`, and `fetched_at`.
- `feed_results`: per-feed diagnostics for multi-feed sources, including `feed_id`, `feed_name`, `feed_type`, `feed_url`, status, item count, and error.
Common diagnostics:
- HTML response: the configured URL is not an RSS/Atom feed, for example a web page listing RSS options.
- HTTP 403: the source or CDN rejected the crawler request.
- Reference: the source is a reference link only and must be converted to RSS, Atom, or Aggregated before fetch testing.
The Admin entry is `Earth Content -> News Sources`. It is not a raw whole-payload JSON editor. The UI has two layers:
- **News sources**: a left-side source list with filters for enabled, disabled, reference links, RSS/Atom/Aggregated, region, and source property tags; the right side edits one selected source and its feed child list.
- **Policy rules**: global source property tags, news categories, item tag rules, and default health policy. Advanced JSON is reserved for diagnostics, not the default edit path.
The single-source form is split into source information and feed children:
- Source information covers name, ID, region, homepage URL, Feed Directory URL, source type, enabled state, source property tags, importance weight, fetch interval, timeout, failure threshold, and circuit breaker.
- Feed children cover feed ID, name, real feed URL, type, enabled switch, default news category, priority, and feed tags. The `+` button under the feed child list creates a frontend-only draft; saving the source persists it, while canceling destroys the draft.
The per-source “test source” action tests all enabled feed children under the current source. The feed-row test action tests only that feed child. Both send to `/api/v1/earth/news-sources/test`, but the feed-row action submits the current source with only the selected feed child.
Reference links show that they only record a homepage, report page, or future collector lead and do not participate in RSS/Atom fetching. They can remain as commercial or official-data leads, but must be converted to RSS, Atom, or Aggregated with fetchable feed URLs before they can be enabled for fetching.
```mermaid
flowchart LR
Admin["Admin: Earth Content / News Sources"] --> Source["Source config"]
Source --> Feed["Feed children"]
Feed --> ConfigAPI["/api/v1/earth/news-sources"]
ConfigAPI --> Config["SystemSetting: earth_news_sources"]
Earth["Earth News Panel"] --> NewsAPI["/api/v1/news/earth-feed"]
NewsAPI --> Resolver["Source Resolver"]
Resolver --> Config
Resolver --> Cache["Region Feed Cache"]
Resolver --> Fetcher["RSS / Atom Fetcher"]
Fetcher --> Parser["Feed Parser"]
Parser --> Classifier["Classifier: category + item_tags + importance"]
Classifier --> Store["earth_news_items"]
Fetcher --> Health["source health"]
Health --> Config
Store --> EnrichQueue["Location / Localization Queue"]
EnrichQueue --> AI["AI Provider"]
Store --> NewsAPI
UE["UE Client"] --> NewsAPI
```

View File

@@ -18,14 +18,14 @@ Note: the layer control panel order and the registration / startup load order ar
| 0 | Earth base sphere | `earth.js` | `CONFIG.earthRadius` | Surface picking fallback target | Dark base; still visible when all optional map layers are off. |
| 0.2 | Country dark tint | `country-boundaries.js` | `tintAltitudeOffset` | Raycast disabled | Used when HD texture is off. |
| 0.86 | Land/ocean base fill | `country-boundaries.js` | `landAltitudeOffset = 0.32`; ocean `#010609`, land `#080f1b` | Raycast disabled | Base map remains usable even when country borders are off; radius is separated from the base sphere to avoid far-zoom z-fighting. |
| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset = 0.48` | Surface picking target when visible | HD texture always overlays the land/ocean base fill; radius must stay above the land/ocean base and far enough from the base sphere. |
| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | Surface picking target when visible | HD texture always overlays the land/ocean base fill; radius must stay above the land/ocean base and far enough from the base sphere. |
| 1 | Atmospheric glow and clouds | `earth.js` | Atmosphere / cloud spheres | Not in normal object selection path | Cloud layer controlled by the "Cloud Layer" toggle. |
| 1 | Submarine cables | `cables.js` | `CABLE_CONFIG.line.renderOrder` | Cable picking path | Preserves existing cable layer level. |
| 1.2 | Real terrain | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` plus terrain displacement | Raycast disabled | Terrain overlays HD texture; temporarily hidden when HD texture is off, restores to prior state when re-enabled. |
| 2.05 | Grid lines | `earth.js` | `CONFIG.earthRadius + 0.14` | Raycast disabled | Low-opacity lines over HD texture. |
| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset` | Raycast disabled | Only needs to stay above HD texture. |
| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`; claim lines have no extra lift | Raycast disabled | Line geometry still has its own `renderOrder`, but it shares the exact same radius as the HD texture shell to avoid parallax while the globe rotates. |
| 2.29 | Country border hover glow | `country-boundaries.js` | Hover radius + glow offset | `depthTest: false`, raycast disabled | Additive glow to reinforce border edge and terrain hover visibility. |
| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset` | `depthTest: false`, raycast disabled | Neon red-orange hover line; China and Taiwan share the same highlight group. |
| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: false`, raycast disabled | Neon red-orange hover line; aligned with the normal borders and HD texture shell to avoid ghosting or floating; China and Taiwan share the same highlight group. |
| 3 | Satellite footprint fill / Iridium coverage ring | `satellites.js`, `iridium-footprint-adapter.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-tested; Iridium adapter fill / ring use the same renderOrder | Footprint above land / texture / terrain and country borders, below compute centers and satellites. |
| 3-5 | BGP markers and overlays | `bgp.js` | Each marker's own renderOrder | BGP picking path | Preserves existing BGP visual level. |
| 4.5 | Compute centers | `compute-centers.js` | `COMPUTE_CENTER_RENDER_ORDER` | Compute center picking path | Surface facilities, below satellites. |
@@ -52,7 +52,8 @@ The Earth surface is not a single mesh. It is a stack of near-concentric shells:
Maintenance rules:
- Do not reach first for hiding layers at far zoom. Check neighboring shell `altitudeOffset`, `renderOrder`, `depthTest`, and `depthWrite` first.
- Whole-globe overlays such as the land/ocean base and HD texture must keep explicit separation from `CONFIG.earthRadius`; the current stable values are `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = 0.48`.
- Whole-globe overlays such as the land/ocean base and HD texture must keep explicit separation from `CONFIG.earthRadius`; the current stable values are `landAltitudeOffset = 0.32` and `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`.
- Country borders, coastlines, claim lines, and country hover lines must use `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET`, exactly matching the HD texture shell. Do not give border lines an independent lower or higher offset, because globe rotation will make the lines appear to drift relative to the surface texture.
- Any new whole-globe or near-whole-globe surface overlay must be screenshot-verified at 50% zoom and at common close zooms, with no black blocks, snow, flicker, or obvious floating.
- If these radii change, update this document and the intent around the constants in `frontend/public/earth/js/constants.js`.

View File

@@ -92,6 +92,35 @@ The Admin sidebar theme switcher still reuses shared [SegmentedControl.tsx](/hom
- Dark mode uses the same dark base, `#202938` slider, and dark external shadow semantics as Docs.
- Product code only hides the text label and keeps icon + tooltip behavior; it should not recreate the private slider DOM.
## Admin Status Colors
Admin status labels should use [StatusText](/home/ray/dev/linkong/planet/frontend/src/admin/patterns/patterns.tsx) or [Badge](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/badge.tsx). Color variables come from [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin/styles.css). New states should map to an existing tone instead of adding page-local hex colors.
`StatusText` is an indicator-light pill: the pill background and border stay on the component base color, while only the dot and text use the status color. `Badge` does not carry the indicator-light meaning, so it may use a light same-tone background and border for stronger hierarchy.
| Tone | Color variable | Meaning | Examples |
| --- | --- | --- | --- |
| `success` | `--an-success` | available, successful, connected, enabled | log source `Available`, collection `Success` |
| `warning` | `--an-warning` | needs attention but is not necessarily failed | missing log file, degraded or skipped state |
| `danger` | `--an-danger` | failed, unavailable, permission/connection error | `Docker unavailable`, endpoint failure |
| `info` / `running` | `--an-info` | in progress, syncing, informational | `Following`, `Syncing` |
| `neutral` | `--an-muted` | empty, disabled, not reported yet, unknown | `No reports yet`, disabled |
| `ai` | fixed purple | AI-specific emphasis | AI generation or model action |
The log source list follows the same rule: `ok` is success; `empty` means the source exists but has not reported yet and is neutral; `missing` means an expected log file is not present and is warning; `docker_unavailable` / `source_unavailable` are danger.
## Admin Runtime Logs
Admin runtime errors are reported through [runtimeLogs.ts](/home/ray/dev/linkong/planet/frontend/src/admin/runtimeLogs.ts) to `/api/v1/system/logs/admin-client`. The reporter only runs on Admin routes and skips `/earth`, `/docs`, login, and registration pages so public-page noise does not enter the Admin client log source.
[AdminErrorBoundary.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/AdminErrorBoundary.tsx) catches React render failures and reuses the same reporter; global `error` and `unhandledrejection` events use that channel as well. Reporting failures must stay silent, because the logging path must not create another frontend error.
The Logs page follows log increments through the `/ws` `logs_tail` channel. File logs and database logs are both normalized into line events by the backend. When adding a new log source, wire it through the backend source registry and tail manager instead of adding a page-local poller.
The Logs page now opens in the grouped view by default. It reads `/api/v1/system/logs/observability/groups`, groups Earth, Admin, and service runtime reports by `fingerprint`, and then reads `/api/v1/system/logs/observability/groups/{fingerprint}/events` when an operator opens one group. Raw logs and audit logs remain separate views; only the raw-log view can follow WebSocket updates. Frontend reporters coalesce repeated errors in a short window and submit `occurrence_count`, while the backend writes both `system_logs` and `observability_events` / `observability_event_groups`, so the page should not add another browser-side aggregation pass over identical messages.
The datasource task queue `View Logs` action opens `/logs?source=system-db&search=task_id=<id>`. Backend database-log search indexes must expand simple JSON context fields into `key=value` aliases such as `task_id=26906` and `datasource_id=20`, so historical task logs remain discoverable without rerunning the task.
## Current Shared Components
### 1. `Scrollbar`

View File

@@ -172,6 +172,10 @@ After selecting a task, the page shows the effective prompt, whether it is custo
The legacy link `/settings?tab=ai` redirects to `/ai?tab=providers`.
## Datasources and Task Logs
`/datasources` is the datasource directory. Built-in sources can be filtered by product domain, level, enabled state, latest run state, collected-data state, and keyword. With no rows selected the main action triggers all matching sources; selecting rows changes it to `Trigger Selected N`. The queue button opens a grouped task panel for running, completed, failed, and skipped work. Failed rows can be retried, completed rows can jump back to their datasource detail, and each task can open `/logs` filtered by its task id.
## System Settings
`/settings` manages system-level configuration. Sub-tabs:
@@ -296,6 +300,8 @@ Adopt All is for batch processing the compute-center unresolved queue. It starts
The settings panel is grouped into Runtime, Display, Panels, Motion, Shortcuts, and System. It covers rotate / cruise / motion mode, cruise modules (BGP/news/compute centers/vessels/cables/satellites), view (satellite display style, hover tooltip, satellite idle breathing, real satellite altitude, track display, compact dots, day-night mode, panel toggles), motion debug mode / input source / skeleton-only, shortcut enablement and remapping, default globe size, terrain opacity, reset.
News categories use the same chip selector as Cruise Modules. They only filter the news panel and news cruise items in the current browser; they do not affect layers, TV, data points, basemap, boundaries, collector jobs, or admin news-source configuration.
"Real Satellite Altitude" is enabled by default: satellite positions use a compressed display height based on TLE/SGP4 orbital altitude. LEO satellites remain close to the globe, while high-orbit satellites render farther out without leaving the normal view. The high-orbit display height is capped at about one quarter of the globe radius, so GEO / MEO objects remain visually separated from LEO without spreading trails and selection targets too far apart. Turning it off restores the legacy same-sphere satellite display. "Track Display" controls satellite trail visibility; trails are unavailable while the satellite layer is hidden.
"Hover Tooltip" controls the tooltip shown when the pointer hovers over the globe surface: `Country` shows country details only when land matches a country, and stays silent over oceans such as the Pacific; `Position` shows latitude, longitude, and elevation over land and ocean; `Full` is the default and shows country + position over land and position over ocean.

View File

@@ -75,7 +75,7 @@ Cleanup order and boundaries:
- Docker cleanup targets resources whose Compose project is `planet`, plus the explicit volumes `planet_postgres_data`, `planet_redis_data`, `postgres_data`, and `redis_data`; do not delete unlabeled volumes by a broad `planet_*` pattern, because another local project could own them.
- Local build state removes `.venv`, frontend `node_modules` / `dist`, Planet state, and scattered Python / Vite cache directories. `$PLANET_CACHE_DIR/downloads` is preserved so upstream raw downloads such as CelesTrak can survive database resets and local rebuild cleanup.
After the reset, run `./planet.sh init` again to recreate tables and default seed data. Old collected records are not restored, and Earth OOBE is evaluated from the backend's real collection state on the next visit. When CelesTrak later returns its "GP data has not updated" HTTP 403, the backend first reuses the preserved download cache to repopulate the database; if no cache exists, wait for the next CelesTrak update window or use Space-Track as a fallback.
After the reset, run `./planet.sh init` again to recreate tables and default seed data. Old collected records are not restored, and Earth OOBE is evaluated from the backend's real collection state on the next visit. When CelesTrak later returns its "GP data has not updated" HTTP 403, the backend first reuses the preserved download cache to repopulate the database; if no active cache exists, it tries valid CelesTrak fallback group caches; if no download cache exists at all, wait for the next CelesTrak update window or use Space-Track. Datasource `Clear Data` and `Clear Cache` actions in the console do not delete `$PLANET_CACHE_DIR/downloads/celestrak`.
## Health Check
@@ -286,6 +286,8 @@ bun run build
Do not use `npm run ...`. In the WSL / Windows mixed environment Bun avoids Node/npm path inconsistencies.
`./planet.sh start` / `init` now runs `bun install` before startup instead of only checking whether the Vite entry file exists. This keeps new devices, cleaned `node_modules`, and lockfile changes synchronized before the console loads, avoiding dynamic-import 500s caused by missing frontend dependencies.
Validate the frontend build:
```bash

View File

@@ -21,7 +21,9 @@
- [智能星球卫星覆盖策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-satellite-footprint-policy.md):卫星 footprint 的显示边界和策略
- [BGP 态势上下文](/home/ray/dev/linkong/planet/docs/technical/zh/earth-bgp-context.md)BGP 在智能星球中的渲染、聚合和观测站实现
- [智能星球可交互图标接入](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)`Interactable` 的接口、生命周期和接入示例
- [智能星球可交互图标聚类策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-clustering.md):可插拔 cluster strategy、稳定球面聚类和动态屏幕聚类的适用边界
- [智能星球工具栏与浮层协同](/home/ray/dev/linkong/planet/docs/technical/zh/earth-toolbar-overlay-coordination.md):工具栏按钮与搜索、设置、新闻、图层浮层的关闭矩阵
- [智能星球新闻源配置](/home/ray/dev/linkong/planet/docs/technical/zh/earth-news-sources.md)默认新闻源、Feed 子项、源属性标签、内容类型、重要度规则和配置接口
## 前端技术实现

View File

@@ -75,6 +75,10 @@ async def run(self, db):
手动触发、删除数据、清理缓存现在统一进入 PostgreSQL 数据作业队列,任务账本仍是 `collection_tasks`。采集器只负责 `fetch -> transform -> save`,由 `data_jobs.py` worker 领取 `collect` / `clear_data` / `clear_cache` / `earth_refresh` 任务并回写进度。Earth 图层刷新关系集中在 `earth_layer_adapters.py`,不要再在单个采集器或按钮里手写缓存失效和 WebSocket 广播。
删除数据任务按批次执行,避免 AIS 这类千万级表一次性锁表。`clear_data` 会先清 `collected_data`,再按来源清理 AIS 衍生表,并持续广播 `records_processed`;前端任务队列只展示“正在删除数据 / 删除完成”,内部表名只保留在日志和原始任务详情。删除结束后后端会 `ANALYZE ais_raw_observations`,让数据源列表的估算指标尽快收敛。数据源目录页默认使用 PostgreSQL 统计信息估算 AIS 大表记录数,避免冷启动做 `count(*)`;打开单条详情时再用精确计数校准当前数据源。
CelesTrak TLE 采集优先拉取完整 `active` 目录。如果 CelesTrak 返回“本轮 GP 数据未更新”的 403后端先复用 `$PLANET_CACHE_DIR/downloads/celestrak` 下的 active 原始下载缓存;没有 active 缓存时进入 fallback group 模式。fallback group 只使用 CelesTrak 当前公开有效的分组,例如 `starlink``gps-ops``galileo``glo-ops``beidou``geo``iridium-next``stations``visual``weather``science``cubesat``amateur``last-30-days`。救灾 fallback 会优先使用本地有效 group 缓存,避免 CelesTrak 小分组在未更新窗口或 HEAD 元数据异常时被误判失败。控制台的“删除数据库”和“清理缓存”只处理数据库记录、Earth layer cache 和 dashboard cache不删除该原始下载缓存。
## 三、采集器列表
| 采集器 | 数据类型 | 数据内容 | 采集频率 |

View File

@@ -75,6 +75,8 @@ Earth 收到 `/ws` 的 `earth_updates` 时只把它当作刷新提示,真实
Earth 设置面板现在按 `data-settings-tab``data-settings-tab-panel` 分类组织。桌面端和移动端使用同一组分类语义:运行、显示、面板、动捕、快捷键、系统。新增设置项时应先判断它属于哪个分类,再补 DOM、持久化字段和恢复逻辑不要把所有控件继续堆到一个长面板里。
`显示` 分类里的新闻类型选择复用巡航模块的 chip 选择器形态只控制星球端当前浏览器的新闻分类显示。它不会打开或关闭图层、底图、边界、TV、数据点、BGP、船舶、卫星或算力中心这些仍由图层面板、媒体面板和控制台配置各自负责。`controls.js` 只持久化 `shared.newsCategoryFilters` 并广播 `earth:news-category-filters-change``news.js` 会把选中的类型拼到 `/api/v1/news/earth-feed?categories=...&locale=zh-CN`,让 Web 和 UE 走同一套后端类型过滤。
快捷键配置属于设备本地偏好,由 `controls.js` 负责读取、捕获、启用/禁用和重置。它不应写入后端用户设置,也不应影响其它浏览器。后续新增快捷键时,必须同时提供默认键、显示标签、可禁用状态和重置路径,避免只在 keydown handler 中硬编码。
### 4. UI 与状态消息
@@ -138,7 +140,7 @@ Browser Camera provider 的手势识别管线在 [motion-browser-provider.js](/h
- terrain tile 拉取、解码、位移、着色
- 海陆基座与国界底图的整球 overlay
智能星球地表是多层近似同心球,不是单一 mesh。`earth.js` 的基座球、高清材质 overlay、云层/大气,以及 `country-boundaries.js` 的海陆基座都需要明确半径间距。远距视图下 GPU 深度精度会下降,相邻 shell 过近会 z-fighting表现为黑色闪烁块或雪花。当前稳定策略是让海陆基座使用 `landAltitudeOffset = 0.32`,高清材质使用 `textureOverlayAltitudeOffset = 0.48`后续新增或调整整球地表 overlay 时,必须同步检查 [智能星球渲染图层顺序](/home/ray/dev/linkong/planet/docs/technical/zh/earth-render-layer-order.md),并在 50% 缩放视图验证。
智能星球地表是多层近似同心球,不是单一 mesh。`earth.js` 的基座球、高清材质 overlay、云层/大气,以及 `country-boundaries.js` 的海陆基座都需要明确半径间距。远距视图下 GPU 深度精度会下降,相邻 shell 过近会 z-fighting表现为黑色闪烁块或雪花。当前稳定策略是让海陆基座使用 `landAltitudeOffset = 0.32`,高清材质使用 `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`国界线、coastline、claim 线和国界 hover 线也必须使用同一个高清材质壳半径,避免转动地球时相对高清贴图产生视差漂浮感。后续新增或调整整球地表 overlay 时,必须同步检查 [智能星球渲染图层顺序](/home/ray/dev/linkong/planet/docs/technical/zh/earth-render-layer-order.md),并在 50% 缩放视图验证。
### 7. 图层模块
@@ -381,23 +383,24 @@ asset 图标大小由 `Interactable` 的 `icon.fitSize` 控制。SVG / 图片文
`Interactable` 不再把图标本体额外抬离业务高度。`altitudeOffset` 就是 marker、hover glow、locked glow 和 picking 共同使用的地表高度;这样船只图标会继续贴着船只轨迹线,不会因为单独抬高显示位置而显得漂浮。后续如果要解决边缘 glow 裁切,应优先考虑 glow 纹理、overlay 尺寸或图层专属特效,而不是把通用图标层整体抬高。
跨 Interactable 的同坐标避让也在公共层处理。每个 marker 会保留 `icon_base_position` 作为业务原始位置;当多个 Interactable marker 归入同一个经纬度 key 时,公共层会把它们沿地表切平面排成小圈,并刷新已创建的 `THREE.Points` geometry。这样视觉位置和屏幕空间 picking 位置一致,不需要业务层再单独判断“算力中心和 BGP 事件重叠”这类场景
跨 Interactable 的同坐标关系也在公共层记录,但真实位置必须始终以 `icon_base_position` 为准。缩放、避让、聚合和后续 spiderfy 展开都只能改变屏幕表现,不能写回 `marker.position``THREE.Points` 里的业务锚点;巡航定位、详情卡、搜索定位和 picking 返回对象都必须落回真实经纬度。多个图标归入同一个经纬度 key 时,公共层只写 `icon_avoidance_*` 元数据,供业务层弱化 halo 或显示聚合提示;真正的低缩放聚合应通过独立 cluster glyph / screen layout 层实现,而不是把对象沿地表切平面挪开
`Interactable` 的单点显示只由全局地图缩放决定170% 及以下强制显示小圆点,超过 170% 显示原图标。cluster 现在由 `cluster.strategy` 决定:`stable-spherical` 使用离散 zoom band 和 3D 球面分桶BGP、算力中心和 Earth interactable 在同一 band 内旋转或细微缩放时不会重新计算聚合拓扑;`dynamic-screen` 保留屏幕空间聚类,适合船只这类实时高频图层;`none` 关闭聚类。稳定球面聚类的 cluster 圆点刚性落在成员 3D 质心投影上,不参与 2D 避让避免缩放时被推离真实地理位置。cluster 圆点大小随包含对象数量增长,数量过多时按稳定地理顺序拆成多个较小圆点;数量默认只在 hover tooltip 中显示。这个过程只设置 `icon_cluster_*` 展示元数据和重建渲染 Points不改变每个 marker 的真实经纬度。
接口细节、生命周期和接入示例见:
- [智能星球可交互图标接入](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)
- [智能星球可交互图标聚类策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-clustering.md)
### 视角控制反馈
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) 统一维护 Earth 缩放状态。滚轮缩放、缩放按钮和触屏双指捏合最终都会更新 `zoomLevel`,并通过 `showZoomStatusCapsule()` 显示当前缩放比例:
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) 统一维护智能星球缩放状态,所有入口最终都必须调用 `setZoomLevel()` 写相机距离。不要在其他模块直接写 `camera.position.z`,否则缩放百分比、拖拽灵敏度和 Interactable 聚合阈值会再次分叉。
```javascript
showGestureStatusMessage(`缩放 ${Math.round(zoomLevel * 100)}%`, "info");
```
滚轮输入分两类处理:传统鼠标滚轮保留 10% 档位和短动画,并用 `wheelZoomTarget` 作为连续滚动的逻辑基准;触控板 / 高精度滚轮按 pixel delta 连续缩放,直接调用 `setZoomLevel()`,不走 10% 档位动画。触控板路径还会过滤反向后的短窗口旧方向小 residual delta避免惯性尾巴把用户刚反向的缩放又拉回旧方向。
该提示每 90ms 最多更新一次,显示 760ms 后淡出。它是视角反馈,不是数据加载进度,也不应该写进图层 loading 状态。
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 只负责在双指捏合缩放时调用 `setZoomLevel()``showZoomStatusCapsule()`。鼠标滚轮与缩放按钮的胶囊提示应继续放在 `controls.js`,避免同一种缩放反馈散落在多个模块。
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 只负责在双指捏合缩放和动捕缩放时调用 `setZoomLevel()``showZoomStatusCapsule()`。鼠标滚轮与缩放按钮的胶囊提示应继续放在 `controls.js`,避免同一种缩放反馈散落在多个模块。
拖拽地球的旋转灵敏度会根据当前缩放连续衰减,而不是按某个缩放阈值分段:

View File

@@ -0,0 +1,84 @@
# 智能星球可交互图标聚类策略
智能星球的可交互图标由 `createInteractableLayer` 统一管理。图标仍然使用 Three.js `Points` 渲染拾取、hover、locked、tooltip 和巡航焦点继续依赖 marker 的 `userData`;聚类策略只决定“哪些 marker 被合成一个 cluster dot”。
## 策略
`cluster.strategy` 支持三种模式:
- `stable-spherical`:稳定球面聚类。按地球局部 3D 坐标聚合,并用 zoom band 缓存拓扑;旋转和同档缩放只更新投影和尺寸,不重算谁和谁聚在一起。适合 BGP、超算/GPU 中心和 Earth interactable 这类半静态图层。
- `dynamic-screen`:动态屏幕聚类。保留原来的屏幕空间聚类逻辑,每帧按可见投影关系判断。适合船舶等高频实时图层,也可作为稳定策略的回退。
- `none`:不聚类。所有 marker 独立显示,适合低数量或需要精确展示的图层。
`cluster: false` 等价于 `strategy: "none"`。未显式声明 `strategy` 时,保持兼容行为:开启聚类的旧图层继续走 `dynamic-screen`
## Stable Spherical
`stable-spherical` 的核心是把聚类身份从屏幕距离迁到球面距离:
- 每个 marker 使用 `icon_base_position` 作为真实地理锚点。
- 默认 zoom 超过 `2.5` 时强制关闭聚类,所有 marker 展开为原图标。
- 当前 zoom 只映射到离散 band同一 band 内旋转地球或细微缩放不会重算拓扑。
- 跨 band、数据变更、图层显隐变化时才重新计算 cluster。
- cluster 质心由成员 3D 坐标平均后 normalize 回球壳半径,因此 cluster dot 刚性贴在地理质心投影上。
- cluster dot 不参与 2D 避让,避免被屏幕排斥推离真实地理位置。
- band 切换带有少量 hysteresis避免缩放停在临界点时在两个 band 之间来回跳。
- 新生成的 marker / cluster dot 会执行短 scale + opacity ease这只是渲染过渡不改变 marker 的真实经纬度、拾取对象或 locked 状态。
稳定策略使用球面 bucket/hash 邻域查询,禁止用全量双循环。这样大多数帧只承担投影与材质尺寸更新,聚类成本只在 band 或数据版本变化时支付。
## 配置示例
```js
const computeCenterIconLayer = createInteractableLayer({
id: "computeCenters",
// ...
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
minCount: 2,
maxMarkersPerDot: 14,
transitionMs: 220,
bandHysteresis: 0.08,
disableAboveZoom: 2.5,
bands: [
{ key: "far", maxZoom: 1.7, distance: 15 },
{ key: "mid", maxZoom: 2.6, distance: 8 },
{ key: "near", maxZoom: 3.5, distance: 4 },
{ key: "detail", maxZoom: Infinity, distance: 0 },
],
},
});
```
实时层可以保留动态策略:
```js
const vesselIconLayer = createInteractableLayer({
id: "vessels",
// ...
cluster: {
strategy: "dynamic-screen",
enabled: true,
maxMarkersPerDot: 10,
},
});
```
## 调参原则
- `distance` 是球面 3D 距离阈值,单位与 `CONFIG.earthRadius` 一致。值越大,越容易聚合。
- 最远 band 用较大的 `distance` 降低视觉密度;最近 band 通常设为 `0`,让图标完全解散。
- `bandHysteresis` 控制 band 边界滞回。值太小容易临界闪烁,值太大会让切换略显迟钝。
- `transitionMs` 控制聚散过渡时间。建议保持在 160-260ms过长会让实时层显得拖泥带水。
- `disableAboveZoom` 控制精细查看阈值。默认 `2.5`,超过后不再生成 cluster设为 `false` 可关闭这个硬阈值。
- 高实时性图层优先用 `dynamic-screen`,避免数据频繁变更时触发稳定策略的拓扑重算。
- 若某图层出现异常,可临时切回 `dynamic-screen``cluster: false`
## 验收重点
- 同一 zoom band 内旋转地球cluster 不应闪烁或重新聚散。
- cluster dot 应跟随地球表面质心,不被避让逻辑推开。
- 放大到 detail band 后,应恢复该图层原本的图标纹理和点击行为。
- zoom 超过 250% 后不应再显示 cluster dot。
- vessel 等实时层更新后,图标和 cluster 应立即反映最新数据。

View File

@@ -0,0 +1,161 @@
# Earth 新闻源配置
Earth 态势新闻使用 `/api/v1/news/earth-feed` 输出给前端。新闻源配置存放在 `SystemSetting.category = "earth_news_sources"`;没有数据库配置时,后端使用内置默认源作为 fallback seed。
## 默认源
默认源包含四类:
- **新闻源**BBC World、DW Top Stories、CNBC Business、BBC Business、Guardian Business、NPR Business、MarketWatch、TechCrunch、Retail Dive、PR Newswire Retail、36氪、亿邦动力、国家统计局数据发布。
- **行业洞察源**McKinsey Retail、Deloitte Retail。
- **官方数据源**国家统计局数据发布、US Census Retail / E-Commerce、商务数据中心、商务部电商动态、电商物流指数。
- **线索源**BusinessWire Electronic CommerceGoogle News 作为一个聚合 source下面挂 global / americas / europe / middle-east-africa / asia-pacific 五个区域 Feed 子项。
配置型数据源默认保留在 Admin 配置中,但若不是稳定 RSS/Atom则默认不参与自动抓取。管理员可以在 `Earth 内容 -> 新闻源` 中改成可抓取 RSS、启用或禁用。
| 来源 | 类型 | 默认状态 | 默认主类型 | 主要标签 | 用途 |
| --- | --- | --- | --- | --- | --- |
| BBC World | RSS | 启用 | 政治 | `official_media`, `global` | 全球公共新闻基线 |
| DW Top Stories | RSS | 启用 | 政治 | `official_media`, `europe` | 欧洲与国际新闻基线 |
| CNBC Business | RSS | 启用 | 商业 | `business_news`, `us`, `global` | 国际商业新闻 |
| BBC Business / Guardian Business / NPR Business / MarketWatch | RSS | 启用 | 商业 / 金融 | `business_news`, `finance` | 英美商业与金融基线 |
| TechCrunch / Retail Dive / PR Newswire Retail | RSS | 启用 | 科技 / 商业 | `business_news`, `ecommerce`, `retail`, `press_release` | 科技、电商、零售和企业公告 |
| 36氪 | RSS | 启用 | 商业 | `business_news`, `ecommerce`, `china` | 国内商业、创投和快讯;主页是 `https://www.36kr.com/`Feed 信息页是 `https://www.36kr.com/rss-center`Feed 子项是综合资讯、文章资讯、最新快讯、动态内容 |
| 亿邦动力 | RSS | 启用 | 电商 | `ecommerce`, `business_news`, `china`, `retail` | 国内电商行业新闻;主页是 `https://www.ebrun.com/`Feed 信息页是 `https://www.ebrun.com/rss/`Feed 子项是 B2C、B2B、零售、O2O、服务、数据、政策 XML |
| 国家统计局数据发布 | RSS | 启用 | 电商 | `official_data`, `ecommerce`, `retail`, `china` | 官方数据发布 RSS社零和网上零售条目由分类/重要度规则识别 |
| Google News | Aggregated | 启用 | 政治 | `aggregated`, `low_stability` | 一个聚合 sourceFeed 子项为全球、美洲、欧洲、中东与非洲、亚太区域兜底 RSS优先级低于真实 RSS |
| BusinessWire Electronic Commerce | Reference | 禁用 | 电商 | `press_release`, `ecommerce`, `low_stability` | 企业公告线索 |
| McKinsey Retail Insights | Reference | 禁用 | 商业 | `industry_insight`, `retail` | 零售行业洞察 |
| Deloitte Retail | Reference | 禁用 | 商业 | `industry_insight`, `retail` | 零售行业洞察 |
| US Census Retail / E-Commerce | Reference | 禁用 | 电商 | `official_data`, `ecommerce`, `retail`, `us` | 美国零售和电商官方数据 |
| 商务数据中心 | Reference | 禁用 | 商业 | `official_data`, `china` | 国内商务数据 |
| 商务部电商动态 | Reference | 禁用 | 电商 | `official_data`, `ecommerce`, `china` | 国内电商政策与动态 |
| 电商物流指数 | Reference | 禁用 | 电商 | `official_data`, `ecommerce`, `logistics`, `china` | 物流履约与电商景气度 |
`Reference` 源表示参考链接/未来采集器线索,只记录官网、报告页或数据页,不参与 RSS/Atom 抓取。这样可以把商业与官方数据源先纳入后台治理,同时避免不可抓取页面拖垮新闻 feed。
新闻源模型是两层结构:
- `source` 表示来源品牌或聚合器,例如 36氪、亿邦动力、Google News、BBC。
- `homepage_url` 表示来源官网、栏目页或报告页。
- `feed_directory_url` 表示 Feed 信息页,也就是 RSS 订阅中心或 Feed 聚合页,只用于人工查看,不参与抓取。
- `feeds` 表示该来源下真正抓取的 RSS、Atom 或 Aggregated 子项。每个 Feed 子项都有 `id / name / url / type / enabled / default_category / tags / priority`
后端会遍历同一 source 下所有启用的 Feed 子项,逐个抓取、合并去重,并把单个子项的检测结果写入 `health.feed_results`。这不是“备用地址”逻辑36氪的四个订阅地址、亿邦的多个分类 XML、Google News 的五个区域 RSS 都可以同时启用,并且每个 Feed 可以单独配置默认新闻类型和启用状态。HTML 订阅中心或聚合页只能放在 `feed_directory_url`,不能放进 Feed 地址。默认启用的可抓 Feed 已逐项连通性检测RSS/Atom/Aggregated Feed 必须解析到条目Reference 源只保留参考地址和后续采集器线索。
当前仍保留为 Reference 的项不是“坏源”,而是没有找到稳定、可直接消费的 RSS/Atom
- BusinessWire 官方说明支持可定制 RSS/Atom但公开页面未暴露稳定行业 feed URL当前保留电子商务行业页作为公告线索。
- McKinsey / Deloitte 的零售洞察页是报告和文章集合,不是公开 RSS。
- US Census 的 press release RSS 可访问但条目链接为空Quarterly E-Commerce 页面保留为官方数据参考链接。
- 商务部数据、电商物流指数目前未找到稳定 RSS后续应做专用 collector 或人工配置可抓 feed。
## 源属性标签与新闻类型
新闻源有 `source_tags`,在 Admin 中显示为“源属性标签”。它用于描述 source 的属性,不是媒体来源名,也不是新闻条目的内容类型。例如:
- `official_data`:官方数据
- `business_news`:商业新闻
- `ecommerce`:电商
- `finance`:金融
- `retail`:零售
- `logistics`:物流
- `industry_insight`:行业洞察
- `press_release`:企业公告
- `china``global``us`
- `aggregated``low_stability`
单条新闻有一个主类型 `category`,默认类型包括:政治、商业、电商、金融、体育、科技、军事、灾害、能源、社会、文化、其他。`item_tags` 是条目级补充标签例如跨境电商、直播电商、零售数据、物流履约、平台治理、AI、半导体、选举、油价、足球、供应链。
主类型优先由规则引擎根据标题、摘要、来源名打分生成;规则未命中时优先使用 Feed 子项的默认类型,再回退 source 默认类型。AI enrichment 不阻塞新闻展示。
## 重要度
每条新闻输出:
- `importance_score`
- `importance_level`
- `importance_reasons`
- `market_impact`
官方数据源、电商指标、平台型公司、量化指标会提高重要度;企业公告基础权重较低,只有命中大平台、金额、并购、监管等信号时提升。
## 配置与缓存
`GET /api/v1/earth/news-sources` 返回默认或已保存配置。`PUT /api/v1/earth/news-sources` 保存配置并递增 `cache_version`,同时清理进程内 region cache。`POST /api/v1/earth/news-sources/reset` 恢复默认源。`POST /api/v1/earth/news-sources/test` 只测试单个 RSS/Atom/Aggregated 源,不写入新闻表。
## Feed 查询与类型过滤
星球端和 UE 端统一使用 `GET /api/v1/news/earth-feed` 获取新闻。接口支持服务端过滤,不要求客户端拿全量列表后自行筛选。
- `lat` / `lon`:按当前视角推断区域,适合 Web 星球端。
- `region`:显式指定区域,适合 UE 端或服务端集成;可选值包括 `global``americas``europe``asia-pacific``middle-east-africa``global` 是全局聚合视图,会展示所有区域来源;其它区域只展示该区域和 `global` 来源。
- `categories`:逗号分隔的新闻类型 key例如 `business,ecommerce`。全选时可以不传。
- `locale`:展示语言,支持 `zh-CN``en-US`,默认 `zh-CN`。中文 RSS 会以中文原文入库,并由后台补 `en-US`;英文 RSS 则由后台补 `zh-CN`
示例:
```http
GET /api/v1/news/earth-feed?region=europe&categories=business,ecommerce
GET /api/v1/news/earth-feed?lat=48&lon=10&categories=technology
GET /api/v1/news/earth-feed?region=global&categories=business,ecommerce&locale=zh-CN
```
非法新闻类型或语言会返回 `422`,响应中包含允许值。响应体会带 `filters`,用于确认后端实际应用的区域、类型和语言过滤。`items``cruise_items` 使用同一套类型过滤规则。
Web 星球端的新闻类型按钮只保存当前浏览器的显示偏好偏好变化后会重新请求接口。UE 端应直接把类型选择拼到 `categories` 参数里,不需要再做主过滤。
源测试只证明当前 RSS/Atom/XML 能解析到条目,不等于这些条目已经入库展示。展示链路还会检查区域、类型过滤和数据库新鲜度。保存或重置新闻源会递增配置版本并清理缓存;如果当前启用的 Feed 子项在库里没有近期条目,下一次 `earth-feed` 请求会补抓,避免新启用的 36氪、亿邦被旧 Google News 缓存挡住。
## 连通性监测
`POST /api/v1/earth/news-sources/test` 会测试单个源并把结果写入 `earth_news_sources.health[source_id]`。实际 RSS/Atom 抓取也会更新同一份健康状态。
健康结果包含:
- `status``ok``empty``format_error``http_error``timeout``network_error``reference`
- `status_code``content_type``item_count``latency_ms``error``fetched_at`
- `feed_results`:多 Feed source 的逐 Feed 子项检测结果,包含 `feed_id``feed_name``feed_type``feed_url`、状态、条数和错误。
常见诊断:
- 返回 HTML 页面:说明配置 URL 不是 RSS/Atom feed例如把网页中心页当成 feed。
- HTTP 403通常是 CDN、反爬或源站拒绝抓取。
- Reference参考链接不参与抓取需要改为 RSS、Atom 或 Aggregated 后才可测试抓取。
Admin 入口是 `Earth 内容 -> 新闻源`。界面不是整包 JSON 编辑,而是两层:
- **新闻源**:左侧逐个 source 列表支持按启用、停用、参考链接、RSS/Atom/Aggregated、区域和源属性标签筛选右侧编辑当前 source 字段和 Feed 子项列表。
- **策略规则**:保留源属性标签、新闻类型、条目标签规则、默认健康策略等全局规则。高级 JSON 只用于排障,不作为默认编辑路径。
单源表单分为“来源信息”和“Feed 子项”:
- 来源信息包括名称、ID、区域、主页 URL、Feed 信息页、源类型、启用开关、源属性标签、重要度权重、抓取间隔、超时、失败阈值和熔断开关。
- Feed 子项包括 Feed ID、名称、真实 Feed URL、类型、启用开关、默认新闻类型、优先级和 Feed 标签。Feed 子项底部的 `+` 只新增一个前端草稿;保存 source 后才写入配置,取消会销毁草稿。
单源“测试源”会测试当前 source 下全部启用 FeedFeed 子项上的测试按钮只测试当前 Feed。测试请求仍发送到 `/api/v1/earth/news-sources/test`,但 payload 里只带当前 source 和选中的 Feed 子项。
参考链接会显示“只记录官网、报告页或未来采集器线索,不参与 RSS/Atom 抓取”。它可作为商业或官方数据线索保留在配置中,但启用抓取前必须改成 RSS、Atom 或 Aggregated并提供可抓取的 Feed 地址。
```mermaid
flowchart LR
Admin["Admin: Earth 内容 / 新闻源"] --> Source["Source 配置"]
Source --> Feed["Feed 子项"]
Feed --> ConfigAPI["/api/v1/earth/news-sources"]
ConfigAPI --> Config["SystemSetting: earth_news_sources"]
Earth["Earth 新闻面板"] --> NewsAPI["/api/v1/news/earth-feed"]
NewsAPI --> Resolver["Source Resolver"]
Resolver --> Config
Resolver --> Cache["Region Feed Cache"]
Resolver --> Fetcher["RSS / Atom Fetcher"]
Fetcher --> Parser["Feed Parser"]
Parser --> Classifier["Classifier: category + item_tags + importance"]
Classifier --> Store["earth_news_items"]
Fetcher --> Health["source health"]
Health --> Config
Store --> EnrichQueue["Location / Localization Queue"]
EnrichQueue --> AI["AI Provider"]
Store --> NewsAPI
UE["UE Client"] --> NewsAPI
```

View File

@@ -19,14 +19,14 @@
| 0 | 智能星球基座球 | `earth.js` | `CONFIG.earthRadius` | 地表拾取兜底目标 | 深色基座,所有可选地图层关闭时仍可见。 |
| 0.2 | 国界暗色 tint | `country-boundaries.js` | `tintAltitudeOffset` | 禁用 raycast | 高清材质关闭时使用。 |
| 0.86 | 海陆基座填充 | `country-boundaries.js` | `landAltitudeOffset = 0.32`; 海洋 `#010609`,陆地 `#080f1b` | 禁用 raycast | 即使国界线关闭,基座地图仍保持可用;半径与基座球拉开以避免远距 z-fighting。 |
| 0.96 | 高清 Earth 材质 | `earth.js` | `textureOverlayAltitudeOffset = 0.48` | 可见时作为地表拾取目标 | 高清材质始终压过海陆基座填充;半径必须高于海陆基座并与基座球保持足够间距。 |
| 0.96 | 高清 Earth 材质 | `earth.js` | `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | 可见时作为地表拾取目标 | 高清材质始终压过海陆基座填充;半径必须高于海陆基座并与基座球保持足够间距。 |
| 1 | 大气辉光和云图 | `earth.js` | 大气 / 云层球 | 不走普通对象选择路径 | 云图由“大气云图”图层开关控制。 |
| 1 | 海缆 / 登陆点 | `cables.js` | 海缆线和登陆点都使用 `renderOrder = 1`;半径偏移都为 `0.2`;登陆点是专用 `THREE.Sprite` 黄色扁平球 | 海缆走海缆拾取路径;登陆点 `depthTest: false` 保持球体完整,并用相机到球心的球体遮挡判断避免背面穿透 | 登陆点和海缆同层贴地,避免地表设施层的凌空感。 |
| 1.2 | 真实地形 | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` 加地形位移 | 禁用 raycast | 地形压过高清材质;高清材质关闭时临时隐藏,重新开启后恢复原状态。 |
| 2.05 | 经纬线 | `earth.js` | `CONFIG.earthRadius + 0.14` | 禁用 raycast | 低透明度显示在高清材质上。 |
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset = 0.115` | `depthTest: true`,禁用 raycast | 线层使用独立 line geometry 与 `renderOrder` 控制;地形 `depthWrite: false`,所以地形开启时仍可见。 |
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`claim 线不再额外抬高 | `depthTest: true`,禁用 raycast | 线层使用独立 line geometry 与 `renderOrder` 控制,但半径与高清材质壳完全一致,避免转动地球时与高清贴图出现视差。 |
| 2.29 | 国界 hover 光晕 | `country-boundaries.js` | hover 半径加 glow 偏移 | `depthTest: false`,禁用 raycast | 用 additive 光晕增强交界边和地形开启时的 hover 可见性。 |
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset = 0.115` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;与普通国界线同源半径对齐,避免重影;中国和中国(台湾)共享高亮组。 |
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;与普通国界线和高清材质同源半径对齐,避免重影和漂浮感;中国和中国(台湾)共享高亮组。 |
| 3 | 卫星 footprint 填充 / Iridium coverage ring | `satellites.js`, `iridium-footprint-adapter.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-testedIridium adapter 的 fill / ring 也使用同一 renderOrder | Footprint 在 land / texture / terrain 和国界线之上,但在算力中心和卫星之下。 |
| 3-4.5 | BGP 观测站、事件扩散圈和事件 marker | `bgp.js`, `interactable.js` | BGP 观测站和事件 marker 均使用 `Interactable` 批量 `THREE.Points`;事件 marker 使用 `BGP_EVENT_RENDER_ORDER = 4.5`;观测站主图标使用 `BGP_COLLECTOR_RENDER_ORDER = 4.4``BGP_CONFIG.collectorAltitudeOffset = 0.2`;事件 overlay 进入 `bgp-event-overlay-layer`;观测站 halo 和覆盖扇形进入 `bgp-collector-radar-layer` | BGP 事件和观测站都通过 `Interactable` 屏幕空间 picking并参与同坐标避让 | BGP 观测站主图标与船只同层BGP 事件与算力中心同层;向外扩散圈、观测站雷达/覆盖动画继续由 BGP 业务逻辑驱动。 |
| 4.3 | AIS 船只轨迹线 | `vessels.js` | `VESSEL_RENDER_ORDER - 0.1``CONFIG.earthRadius + VESSEL_CONFIG.track.altitudeOffset` | 跟随船只显隐,不单独参与拾取 | 选中船只后显示最近轨迹,低于船只 marker。 |
@@ -55,7 +55,8 @@ Earth 的地表不是单一 mesh而是多层近似同心球基座球、海
维护规则:
- 不要用“远距隐藏图层”作为第一反应;先检查相邻 shell 的 `altitudeOffset``renderOrder``depthTest``depthWrite`
- 海陆基座和高清材质这类整球 overlay 必须与 `CONFIG.earthRadius` 保持明确间距;当前稳定值为 `landAltitudeOffset = 0.32``textureOverlayAltitudeOffset = 0.48`
- 海陆基座和高清材质这类整球 overlay 必须与 `CONFIG.earthRadius` 保持明确间距;当前稳定值为 `landAltitudeOffset = 0.32``textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`
- 国界线、coastline、claim 线和国界 hover 线必须使用 `EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET`,与高清材质壳完全同半径;不要再用低于或高于高清材质的独立线层 offset否则转动地球时会产生相对地表的视差漂浮感。
- 新增整球或近整球地表 overlay 时,必须在 50% 缩放和常用近距视图各截一次图,确认没有黑块、雪花、闪烁,也没有明显漂浮感。
- 如果必须调整这些半径,需同步更新本文和 `frontend/public/earth/js/constants.js` 的注释/常量意图。

View File

@@ -114,7 +114,7 @@ curl http://localhost:8010/health
如果 `ss -ltnp` 显示前端已经监听 `0.0.0.0:3000`,但 Windows PowerShell 中 `Test-NetConnection <Windows局域网IP> -Port 3000` 仍失败,问题通常不在 Vite 或 `.zshrc`,而是在 Windows 侧端口占用、旧 `portproxy` 或防火墙。
`./planet.sh start --allow-lan` 会直接开放 `3000` / `8000` / `8010`,并在启动前检测端口、旧 `portproxy` 和 Windows 防火墙规则。端口被 Windows 侧 listener 占用时,脚本会请求管理员 PowerShell 清理;缺少入站放行时,也会触发一次 UAC 管理员 PowerShell 请求来自动创建。若自动请求被取消,可以手动清理:
`./planet.sh start --allow-lan` 会直接开放 `3000` / `8000` / `8010`,并在启动前检测端口、旧 `portproxy` 和 Windows 防火墙规则。端口被 Windows 侧 listener 占用时,脚本会请求管理员 PowerShell 清理;缺少入站允许规则时,也会触发一次 UAC 管理员 PowerShell 请求来自动创建。若自动请求被取消,可以手动清理:
```powershell
netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
@@ -128,7 +128,7 @@ New-NetFirewallRule -DisplayName "WSL Planet 8010" -Direction Inbound -Action Al
局域网设备访问 Windows 对外端口,例如 `http://<Windows局域网IP>:3000/earth`
如果 `wslinfo --networking-mode` 输出 `mirrored`,还需要检查 Hyper-V firewall。普通 Windows 防火墙规则存在时Hyper-V firewall 仍可能拦截外部设备进入 WSL。管理员 PowerShell 中按端口放行
如果 `wslinfo --networking-mode` 输出 `mirrored`,还需要检查 Hyper-V firewall。普通 Windows 防火墙规则存在时Hyper-V firewall 仍可能拦截外部设备进入 WSL。管理员 PowerShell 中按端口开放访问
```powershell
New-NetFirewallHyperVRule -Name "Planet-Frontend-3000" -DisplayName "Planet Frontend 3000" -Direction Inbound -VMCreatorId "{40E0AC32-46A5-438A-A0B2-2B479E8F2E90}" -Protocol TCP -LocalPorts 3000 -Action Allow

View File

@@ -92,6 +92,35 @@ Admin 侧栏底部主题切换继续复用共享 [SegmentedControl.tsx](/home/ra
- dark 下使用与 Docs 一致的深色底座、`#202938` slider 和深色外投影。
- 业务侧只隐藏文字 label 并保留 icon + tooltip不重写 slider DOM。
## 控制台状态颜色
Admin 的状态标签统一走 [StatusText](/home/ray/dev/linkong/planet/frontend/src/admin/patterns/patterns.tsx) 或 [Badge](/home/ray/dev/linkong/planet/frontend/src/admin/components/ui/badge.tsx),颜色变量来自 [styles.css](/home/ray/dev/linkong/planet/frontend/src/admin/styles.css)。新增状态时不要在页面里临时写 hex先判断语义再映射到现有 tone。
`StatusText` 是带圆点的指示灯:胶囊背景和边框保持组件原色,只让圆点和文字变成状态色。`Badge` 不带指示灯语义,可以使用同 tone 的浅色背景和边框强化信息层级。
| Tone | 颜色变量 | 语义 | 示例 |
| --- | --- | --- | --- |
| `success` | `--an-success` | 可用、成功、已连接、已启用 | 日志源 `可用`、采集 `成功` |
| `warning` | `--an-warning` | 需要关注但不一定失败 | 日志文件 `暂无日志`、降级或跳过 |
| `danger` | `--an-danger` | 失败、不可用、权限/连接错误 | `Docker 不可用`、接口失败 |
| `info` / `running` | `--an-info` | 进行中、同步中、普通信息态 | `跟随中``同步中` |
| `neutral` | `--an-muted` | 空态、停用、未上报、未知 | `暂无上报``停用` |
| `ai` | 固定紫色 | AI 相关突出态 | AI 生成、模型动作 |
日志源列表遵循同一规则:`ok` 显示 success`empty` 表示来源存在但还没有上报,显示 neutral`missing` 表示期望的日志文件暂时不存在,显示 warning`docker_unavailable` / `source_unavailable` 显示 danger。
## 控制台运行时日志
控制台运行时错误由 [runtimeLogs.ts](/home/ray/dev/linkong/planet/frontend/src/admin/runtimeLogs.ts) 统一上报到 `/api/v1/system/logs/admin-client`。它只在控制台路由内启用,跳过 `/earth``/docs`、登录和注册页面,避免公开页面噪声进入控制台日志源。
[AdminErrorBoundary.tsx](/home/ray/dev/linkong/planet/frontend/src/admin/components/AdminErrorBoundary.tsx) 捕获 React 渲染错误并复用同一上报函数;全局 `error``unhandledrejection` 也进入该通道。上报失败必须静默处理,不能因为日志系统异常再制造新的前端错误。
日志页通过 `/ws``logs_tail` channel 跟随日志增量;文件日志和数据库日志都由后端统一转换成行事件。新增日志源时优先接入后端 source registry 和 tail manager不要在日志页写独立轮询器。
日志页默认进入“重复统计”视图,读取 `/api/v1/system/logs/observability/groups`,按 `fingerprint` 聚合 Earth、Admin 和服务端运行时上报;点击聚合项再读取 `/api/v1/system/logs/observability/groups/{fingerprint}/events` 展示发生明细。原始日志和审计日志仍保留为独立视图;只有原始日志视图允许通过 WebSocket 跟随。前端上报器会在短时间窗口内合并同一错误并提交 `occurrence_count`,后端同时写 `system_logs``observability_events` / `observability_event_groups`,所以日志页不要再按相同消息在浏览器端二次聚合。
数据源任务队列的“查看日志”入口跳转到 `/logs?source=system-db&search=task_id=<id>`。后端数据库日志搜索索引必须把 JSON context 中的简单字段同时展开为 `key=value` 别名,例如 `task_id=26906``datasource_id=20`,这样历史任务日志不依赖重新执行任务也能被精确查到。
## 当前共享组件
### 1. `Scrollbar`

View File

@@ -236,7 +236,7 @@ Base URL 输入框尾端的插头图标会触发连接测试。测试通过会
## 数据探索
- `/datasources`:数据源目录。`内置源` 支持按产品域、层级、启用状态、最近执行状态、是否已有采集数据和关键词筛选;未勾选时主按钮显示“触发全部”,勾选多行后会变成“触发已选 N”并只提交所选数据源。右上角队列按钮空态显示队列图标有任务时显示纯圆环总进度点击后打开队列浮层按运行中、完成、失败和跳过分组失败项可重试完成项可跳到详情。`实时源` 面向 AISStream / WebSocket 长连接,展示连接健康、累计入库、时间窗统计和启动 / 停止 / 重连操作。接口、凭证、请求头的编辑统一在 `/collection-management` 的"采集器"。
- `/datasources`:数据源目录。`内置源` 支持按产品域、层级、启用状态、最近执行状态、是否已有采集数据和关键词筛选;未勾选时主按钮显示“触发全部”,勾选多行后会变成“触发已选 N”并只提交所选数据源。右上角队列按钮空态显示队列图标有任务时显示纯圆环总进度点击后打开队列浮层按运行中、完成、失败和跳过分组失败项可重试完成项可跳到详情或按任务编号打开系统日志`实时源` 面向 AISStream / WebSocket 长连接,展示连接健康、累计入库、时间窗统计和启动 / 停止 / 重连操作。接口、凭证、请求头的编辑统一在 `/collection-management` 的"采集器"。
- `/data`:采集后数据表,适合排查"数据是否已经进入系统"、"更新时间是否符合预期"、"某个数据源是否产出有效记录"
- `/bgp`BGP 专题页面,列表 + 详情 + 研判,与智能星球的 BGP 图层互补
- `/alerts/system``/alerts/bgp``/alerts/situational`系统、BGP、态势告警
@@ -299,6 +299,8 @@ AIS 船只图例按船型显示颜色:货轮、油轮、客船、渔船、军
设置面板按分类组织:运行、显示、面板、动捕、快捷键、系统。里面包含旋转模式 / 巡航模式 / 动捕模式、巡航模块BGP/新闻/算力中心/船只/海缆/卫星)、视图设置(卫星显示风格、悬停提示、卫星呼吸闪烁、真实卫星高度、轨迹显示、低缩放圆点、日夜模式、面板显示开关)、动捕调试模式 / 输入源 / 只显示骨骼、快捷键启用与改键、地球默认大小、地形透明度、重置设置。
新闻类型使用与巡航模块一致的标签选择器只筛选当前浏览器里的新闻面板和新闻巡航条目不影响图层、TV、数据点、底图、边界、采集任务或后台新闻源配置。
“真实卫星高度”默认开启:卫星会按 TLE/SGP4 算出的真实轨道高度做压缩分层显示,低轨仍靠近地球,高轨会更远但不会脱离当前视图。高轨显示高度会被压到地球半径外约四分之一以内,这样 GEO / MEO 仍能和 LEO 分层,但不会把视线、轨迹和选择操作拉得过散;关闭后恢复旧版所有卫星位于同一显示球面的效果。“轨迹显示”控制卫星轨迹线显隐,卫星图层关闭时轨迹也不可见。
“悬停提示”控制鼠标悬停地表时的 tooltip 内容:`国家` 只在陆地命中国家时显示国家信息,太平洋等海洋区域不弹出地表提示;`位置` 在陆地和海洋都显示纬度、经度和海拔;`完整` 是默认模式,陆地显示国家 + 位置,海洋显示位置。

View File

@@ -75,7 +75,7 @@
- Docker 清理只针对 Compose project 为 `planet` 的资源,以及显式列出的 `planet_postgres_data``planet_redis_data``postgres_data``redis_data`;不要按 `planet_*` 模式删除没有 label 的 volume避免误删同机其他项目。
- 本地编译状态会删除 `.venv`、前端 `node_modules` / `dist`、Planet state以及散落的 Python / Vite 缓存目录;`$PLANET_CACHE_DIR/downloads` 会保留,用于保存 CelesTrak 这类受上游下载窗口限制的原始文件缓存。
重置后重新执行 `./planet.sh init` 会重建表和默认数据,但不会恢复旧采集结果;首次进入 Earth 时 OOBE 会重新按后端真实采集状态判断。触发 CelesTrak 采集时,如果上游返回“本轮 GP 数据未更新”的 403后端会优先用保留的下载缓存重新写入数据库如果下载缓存也不存在,只能等待 CelesTrak 下一次更新窗口或使用 Space-Track 作为 fallback
重置后重新执行 `./planet.sh init` 会重建表和默认数据,但不会恢复旧采集结果;首次进入 Earth 时 OOBE 会重新按后端真实采集状态判断。触发 CelesTrak 采集时,如果上游返回“本轮 GP 数据未更新”的 403后端会优先用保留的下载缓存重新写入数据库如果 active 缓存不存在,会尝试有效 CelesTrak 分组缓存作为 fallback如果下载缓存也不存在只能等待 CelesTrak 下一次更新窗口或使用 Space-Track。控制台里的数据源“删除数据库”和“清理缓存”不会删除 `$PLANET_CACHE_DIR/downloads/celestrak`
## 健康检查
@@ -150,7 +150,7 @@ Planet 开发环境建议使用 WSL2。WSL1 下网络、文件系统和进程模
wsl --set-version Ubuntu 2
```
`--allow-lan` 会让前端、后端和 AI Provider 直接对开发机开放:前端 `3000`、后端 `8000`、AI Provider `8010`。脚本启动前会检查这三个端口;如果 WSL/Linux 侧无法释放端口,并检测到 Windows 侧 listener 或旧 `portproxy`,会请求管理员 PowerShell 清理。WSL 中运行时Windows 本机一般可以通过 `localhost` 访问,局域网其他机器访问 Windows 局域网 IP 时还需要 Windows 防火墙放行
`--allow-lan` 会让前端、后端和 AI Provider 直接对开发机开放:前端 `3000`、后端 `8000`、AI Provider `8010`。脚本启动前会检查这三个端口;如果 WSL/Linux 侧无法释放端口,并检测到 Windows 侧 listener 或旧 `portproxy`,会请求管理员 PowerShell 清理。WSL 中运行时Windows 本机一般可以通过 `localhost` 访问,局域网其他机器访问 Windows 局域网 IP 时还需要 Windows 防火墙允许访问
建议按顺序排查:
@@ -162,7 +162,7 @@ curl http://localhost:8010/health
ss -ltnp | grep -E ':3000|:8000|:8010'
```
如果服务已经启动但局域网 IP 仍访问失败,优先清理旧 `portproxy` 并确认 Windows 防火墙放行。脚本会自动检测并请求管理员 PowerShell 处理;自动请求被取消时,手动兜底命令如下:
如果服务已经启动但局域网 IP 仍访问失败,优先清理旧 `portproxy` 并确认 Windows 防火墙允许访问。脚本会自动检测并请求管理员 PowerShell 处理;自动请求被取消时,手动兜底命令如下:
```powershell
netsh interface portproxy delete v4tov4 listenaddress=0.0.0.0 listenport=3000
@@ -286,6 +286,8 @@ bun run build
不要使用 `npm run ...`。项目在 WSL / Windows 混合环境优先依赖 Bun避免 Node/npm 路径差异。
`./planet.sh start` / `init` 会在启动前执行一次 `bun install`,而不是只检查 Vite 入口文件是否存在。这样新设备、清过 `node_modules` 的环境或 lockfile 已变更的环境,都能在进入控制台前同步前端依赖,避免动态 import 因缺失依赖返回 500。
验证前端构建:
```bash

View File

@@ -16,12 +16,16 @@
## Current Version
- `main` 当前主线历史推导到:`0.16.5`
- `dev` 当前开发分支历史推导到:`0.66.3`
- `dev` 当前开发分支历史推导到:`0.69.0`
## Timeline
| Version | Type | Branch | Commit | Summary |
| --- | --- | --- | --- | --- |
| `0.69.0` | feature | `dev` | `pending` | 新增 Earth 新闻源治理、新闻类型服务端过滤、观测日志 fingerprint 聚合和 TV/HLS 播放恢复改进 |
| `0.68.1` | bugfix | `dev` | `pending` | 修复 CelesTrak fallback group/cache 恢复链路,并让数据源任务日志可按 task_id / datasource_id 搜索 |
| `0.68.0` | feature | `dev` | `pending` | 新增数据源任务队列实时指标、AIS 大表分批删除和智能星球可插拔聚类策略,并让新设备启动前同步前端依赖 |
| `0.67.0` | feature | `dev` | `pending` | 新增控制台日志实时跟随和运行时错误上报,重构智能星球 Interactable 聚合、wheel 缩放输入、国界壳半径和开发脚本锁文件保护 |
| `0.66.3` | bugfix | `dev` | `pending` | 补上 Admin utility module 并放开前端源码 lib 例外,修复控制台动态导入 500 与 Mermaid 包解析失败 |
| `0.66.2` | bugfix | `dev` | `pending` | 统一智能星球、控制台和文档的中文显示命名,清理 Docs catalog、使用手册、控制台入口和智能星球设置中的中英混排文案 |
| `0.66.1` | bugfix | `dev` | `pending` | CelesTrak active 限频时复用持久下载缓存并完整 fallback groupdestroy 保留原始下载缓存,同时修复采集失败 toast 重复弹出 |

View File

@@ -1,6 +1,6 @@
{
"name": "planet-frontend",
"version": "0.66.3",
"version": "0.69.0",
"private": true,
"packageManager": "bun@1",
"dependencies": {

View File

@@ -1086,6 +1086,12 @@
font-weight: 600;
}
.earth-mobile-news-filter-popover {
position: relative;
inset: auto;
max-height: 34vh;
}
.earth-mobile-news-board-list .news-story-card {
margin: 0;
}
@@ -2822,6 +2828,7 @@ label.is-disabled.earth-mobile-settings-card {
display: flex;
align-items: center;
gap: 12px;
min-width: 0;
}
.earth-settings-segmented {
@@ -2908,10 +2915,17 @@ label.is-disabled.earth-mobile-settings-card {
display: inline-flex;
align-items: center;
justify-content: center;
flex: 0 0 calc(30px * var(--hud-scale));
width: calc(30px * var(--hud-scale));
height: calc(30px * var(--hud-scale));
min-width: calc(30px * var(--hud-scale));
min-height: calc(30px * var(--hud-scale));
max-width: calc(30px * var(--hud-scale));
max-height: calc(30px * var(--hud-scale));
aspect-ratio: 1 / 1;
padding: 0;
border: 1px solid rgba(122, 180, 255, 0.24);
border-radius: 999px;
border-radius: 50%;
background:
radial-gradient(circle at 35% 22%, rgba(255, 255, 255, 0.2), transparent 58%),
linear-gradient(180deg, rgba(121, 159, 207, 0.18), rgba(72, 101, 139, 0.24));
@@ -2926,12 +2940,16 @@ label.is-disabled.earth-mobile-settings-card {
}
.earth-settings-reload-action--text {
flex-basis: auto;
width: auto;
max-width: none;
min-width: calc(46px * var(--hud-scale));
padding: 0 calc(12px * var(--hud-scale));
aspect-ratio: auto;
font: inherit;
font-size: calc(0.7rem * var(--hud-scale));
font-weight: 600;
border-radius: 999px;
}
.earth-settings-reload-action[hidden] {
@@ -3076,6 +3094,7 @@ label.is-disabled.earth-mobile-settings-card {
.earth-settings-slider {
flex: 1 1 auto;
min-width: calc(120px * var(--hud-scale));
width: 100%;
height: calc(4px * var(--hud-scale));
appearance: none;

View File

@@ -427,6 +427,43 @@
text-wrap: balance;
}
.info-card-news-meta-grid {
display: grid;
grid-template-columns: repeat(2, minmax(0, 1fr));
gap: calc(6px * var(--hud-scale));
}
.info-card-news-meta-grid--mobile {
grid-template-columns: 1fr;
}
.info-card-news-meta-item {
min-width: 0;
padding: calc(6px * var(--hud-scale)) calc(8px * var(--hud-scale));
border: 1px solid rgba(201, 225, 247, 0.08);
background: rgba(255, 255, 255, 0.035);
}
.info-card-news-meta-item span,
.info-card-news-meta-item strong {
display: block;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.info-card-news-meta-item span {
color: rgba(188, 212, 238, 0.62);
font-size: calc(0.58rem * var(--hud-scale));
}
.info-card-news-meta-item strong {
color: rgba(236, 246, 255, 0.9);
font-size: calc(0.68rem * var(--hud-scale));
font-weight: 650;
margin-top: calc(2px * var(--hud-scale));
}
.info-card-news-summary-shell {
position: relative;
padding: calc(10px * var(--hud-scale)) calc(12px * var(--hud-scale));

View File

@@ -303,6 +303,121 @@
font-size: calc(0.72rem * var(--hud-scale));
}
.news-filter-bar {
position: relative;
display: flex;
align-items: center;
gap: calc(8px * var(--hud-scale));
flex-wrap: wrap;
}
.news-filter-pill {
display: inline-flex;
align-items: center;
gap: calc(6px * var(--hud-scale));
min-height: calc(30px * var(--hud-scale));
border: 1px solid rgba(201, 225, 247, 0.1);
border-radius: calc(12px * var(--hud-scale));
padding: calc(5px * var(--hud-scale)) calc(9px * var(--hud-scale));
color: var(--hud-text-soft);
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.07), rgba(116, 166, 224, 0.04));
box-shadow:
inset 0 1px 0 rgba(255, 255, 255, 0.06),
0 8px 18px rgba(2, 10, 22, 0.12);
font: inherit;
font-size: calc(0.72rem * var(--hud-scale));
cursor: pointer;
}
.news-filter-pill:hover,
.news-filter-pill[aria-expanded="true"] {
color: var(--hud-text);
border-color: rgba(147, 202, 255, 0.22);
background:
linear-gradient(180deg, rgba(255, 255, 255, 0.1), rgba(116, 166, 224, 0.07));
}
.news-filter-pill strong {
color: var(--hud-accent-strong);
font-size: calc(0.7rem * var(--hud-scale));
font-weight: 700;
}
.news-filter-pill--view {
margin-left: auto;
}
.news-filter-popover {
position: absolute;
top: calc(154px * var(--hud-scale));
left: calc(14px * var(--hud-scale));
right: calc(14px * var(--hud-scale));
z-index: 30;
display: grid;
gap: calc(10px * var(--hud-scale));
max-height: min(calc(280px * var(--hud-scale)), 44vh);
overflow-y: auto;
border: 1px solid rgba(205, 231, 255, 0.12);
border-radius: calc(16px * var(--hud-scale));
padding: calc(12px * var(--hud-scale));
color: var(--hud-text);
background:
radial-gradient(circle at 18% 10%, rgba(122, 187, 255, 0.16), transparent 42%),
linear-gradient(180deg, rgba(20, 35, 58, 0.96), rgba(10, 20, 35, 0.96));
box-shadow:
0 20px 50px rgba(2, 8, 20, 0.38),
inset 0 1px 0 rgba(255, 255, 255, 0.07);
backdrop-filter: blur(18px) saturate(125%);
-webkit-backdrop-filter: blur(18px) saturate(125%);
}
.news-filter-popover[hidden] {
display: none;
}
.news-filter-popover__header {
display: flex;
align-items: baseline;
justify-content: space-between;
gap: calc(10px * var(--hud-scale));
}
.news-filter-popover__title {
font-size: calc(0.8rem * var(--hud-scale));
font-weight: 700;
}
.news-filter-popover__hint {
color: var(--hud-text-muted);
font-size: calc(0.66rem * var(--hud-scale));
}
.news-filter-chip-group {
display: flex;
flex-wrap: wrap;
gap: calc(8px * var(--hud-scale));
}
.news-filter-chip {
border: 1px solid rgba(201, 225, 247, 0.12);
border-radius: calc(14px * var(--hud-scale));
padding: calc(7px * var(--hud-scale)) calc(10px * var(--hud-scale));
color: var(--hud-text-soft);
background: rgba(255, 255, 255, 0.04);
font: inherit;
font-size: calc(0.74rem * var(--hud-scale));
cursor: pointer;
}
.news-filter-chip.is-active {
color: var(--hud-text);
border-color: rgba(120, 190, 255, 0.36);
background:
linear-gradient(180deg, rgba(79, 143, 232, 0.22), rgba(64, 111, 191, 0.12));
box-shadow: inset 0 0 0 1px rgba(206, 232, 255, 0.08);
}
.news-board {
display: flex;
flex: 1 1 auto;
@@ -379,13 +494,21 @@
.news-story-tags {
display: flex;
align-items: center;
justify-content: space-between;
gap: calc(8px * var(--hud-scale));
flex-wrap: wrap;
}
.news-story-meta {
justify-content: space-between;
}
.news-story-tags {
justify-content: flex-start;
}
.news-story-source,
.news-story-time,
.news-story-origin,
.news-story-tag {
color: var(--hud-text-soft);
font-size: calc(0.66rem * var(--hud-scale));
@@ -393,6 +516,17 @@
.news-story-source {
color: var(--hud-accent-strong);
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.news-story-origin {
color: rgba(188, 212, 238, 0.56);
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
}
.news-story-title {

View File

@@ -606,6 +606,22 @@
<div id="news-source-count" class="news-source-count">0 路聚合源</div>
</div>
<div class="news-filter-bar" data-news-filter-bar>
<button class="news-filter-pill" type="button" data-news-filter-toggle="category" aria-expanded="false">
<span>类型</span>
<strong data-news-filter-summary="category">全部</strong>
</button>
<button class="news-filter-pill" type="button" data-news-filter-toggle="source" aria-expanded="false">
<span>来源</span>
<strong data-news-filter-summary="source">全部</strong>
</button>
<button id="news-view-all-toggle" class="news-filter-pill news-filter-pill--view" type="button">
<span data-news-view-mode-label>查看全部</span>
<strong data-news-filter-summary="limit">12 条</strong>
</button>
</div>
<div class="news-filter-popover" data-news-filter-popover hidden></div>
<div class="news-board">
<div id="news-board-status" class="news-board-status">正在准备全球态势新闻...</div>
<div id="news-board-list" class="news-board-list"></div>
@@ -768,6 +784,21 @@
</div>
<div id="mobile-news-source-count" class="earth-mobile-news-source-count">0 路聚合源</div>
</div>
<div class="news-filter-bar earth-mobile-news-filter-bar" data-news-filter-bar>
<button class="news-filter-pill" type="button" data-news-filter-toggle="category" aria-expanded="false">
<span>类型</span>
<strong data-news-filter-summary="category">全部</strong>
</button>
<button class="news-filter-pill" type="button" data-news-filter-toggle="source" aria-expanded="false">
<span>来源</span>
<strong data-news-filter-summary="source">全部</strong>
</button>
<button id="mobile-news-view-all-toggle" class="news-filter-pill news-filter-pill--view" type="button">
<span data-news-view-mode-label>查看全部</span>
<strong data-news-filter-summary="limit">12 条</strong>
</button>
</div>
<div class="news-filter-popover earth-mobile-news-filter-popover" data-news-filter-popover hidden></div>
<div id="mobile-news-board-status" class="earth-mobile-news-board-status">正在准备全球态势新闻...</div>
<div id="mobile-news-board-list" class="earth-mobile-news-board-list"></div>
<div id="mobile-news-board-empty" class="earth-mobile-news-board-empty" hidden>正在准备全球态势新闻聚合源...</div>
@@ -1049,6 +1080,27 @@
<span class="earth-mobile-settings-switch-track"></span>
</span>
</label>
<div class="earth-mobile-settings-subsection-title">新闻类型</div>
<div class="earth-mobile-settings-card earth-mobile-settings-card--stacked">
<div class="earth-mobile-settings-copy">
<span class="earth-mobile-settings-label">新闻类型</span>
<span class="earth-mobile-settings-subtitle">只筛选当前浏览器的新闻面板与新闻巡航,不改变后台新闻源。</span>
</div>
<div class="earth-mobile-settings-chip-group" role="group" aria-label="选择新闻类型">
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="politics" aria-pressed="true">政治</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="business" aria-pressed="true">商业</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="ecommerce" aria-pressed="true">电商</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="finance" aria-pressed="true">金融</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="sports" aria-pressed="true">体育</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="technology" aria-pressed="true">科技</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="military" aria-pressed="true">军事</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="disaster" aria-pressed="true">灾害</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="energy" aria-pressed="true">能源</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="society" aria-pressed="true">社会</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="culture" aria-pressed="true">文化</button>
<button type="button" class="earth-mobile-settings-chip is-active" data-news-category-toggle="other" aria-pressed="true">其他</button>
</div>
</div>
</div>
<div class="earth-mobile-settings-group" data-settings-tab-panel="panels" hidden>
<div class="earth-mobile-settings-title">面板</div>
@@ -1555,6 +1607,27 @@
<span class="earth-settings-switch-track"></span>
</span>
</label>
<div class="earth-settings-subsection-title">新闻类型</div>
<div class="earth-settings-item earth-settings-item--stacked">
<div class="earth-settings-copy">
<span class="earth-settings-item-title">新闻类型</span>
<span class="earth-settings-item-subtitle">只筛选当前浏览器的新闻面板与新闻巡航,不改变后台新闻源。</span>
</div>
<div class="earth-settings-chip-group" role="group" aria-label="选择新闻类型">
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="politics" aria-pressed="true">政治</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="business" aria-pressed="true">商业</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="ecommerce" aria-pressed="true">电商</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="finance" aria-pressed="true">金融</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="sports" aria-pressed="true">体育</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="technology" aria-pressed="true">科技</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="military" aria-pressed="true">军事</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="disaster" aria-pressed="true">灾害</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="energy" aria-pressed="true">能源</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="society" aria-pressed="true">社会</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="culture" aria-pressed="true">文化</button>
<button type="button" class="earth-settings-chip is-active" data-news-category-toggle="other" aria-pressed="true">其他</button>
</div>
</div>
</div>
</section>
<section class="earth-settings-section" data-settings-tab-panel="panels" hidden>

View File

@@ -333,6 +333,9 @@ const bgpEventIconLayer = createInteractableLayer({
pulseOffset: Math.random() * Math.PI * 2,
}),
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
},
});
const bgpCollectorIconLayer = createInteractableLayer({
@@ -402,6 +405,9 @@ const bgpCollectorIconLayer = createInteractableLayer({
};
},
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
},
});
function clamp(value, min, max) {

View File

@@ -2,6 +2,8 @@ import { PATHS } from "./constants.js";
const RECENT_EVENT_TTL_MS = 15_000;
const recentEventMap = new Map();
const pendingEventMap = new Map();
let pendingFlushTimer = null;
function normalizeErrorDetail(detail) {
if (!detail) return "";
@@ -22,6 +24,37 @@ function dedupeKey(level, message, detail, category) {
return `${level}::${category || ""}::${message}::${detail}`;
}
function normalizeFingerprintText(value) {
return String(value || "")
.replace(/[?&](m|t|token|expires|signature|X-Amz-[^=]+)=[^&\s]+/gi, "")
.replace(/(index|chunk|segment)[_-]?\d+(_\d+)?\.(ts|m4s|vtt)/gi, "<hls-fragment>")
.replace(/\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b/gi, "<uuid>")
.replace(/\bconn_[A-Za-z0-9:._-]+\b/g, "<connection>")
.replace(/\b\d{5,}\b/g, "<number>")
.replace(/\s+/g, " ")
.trim();
}
function hashFingerprint(value) {
let hash = 5381;
for (let index = 0; index < value.length; index += 1) {
hash = ((hash << 5) + hash) ^ value.charCodeAt(index);
}
return `client-${(hash >>> 0).toString(16).padStart(8, "0")}`;
}
function buildFingerprint(level, message, detail, category, module) {
return hashFingerprint(
[
normalizeFingerprintText(level),
normalizeFingerprintText(category),
normalizeFingerprintText(module),
normalizeFingerprintText(message),
normalizeFingerprintText(detail),
].join("|"),
);
}
function shouldSkip(level, message, detail, category) {
const key = dedupeKey(level, message, detail, category);
const now = Date.now();
@@ -37,33 +70,14 @@ function shouldSkip(level, message, detail, category) {
return lastSeenAt && now - lastSeenAt < RECENT_EVENT_TTL_MS;
}
export async function reportEarthClientLog({
level = "error",
message,
category = "runtime",
module = "earth",
detail = "",
}) {
if (!message) return;
const normalizedDetail = normalizeErrorDetail(detail);
if (shouldSkip(level, message, normalizedDetail, category)) {
return;
}
async function sendEarthClientLog(payload) {
try {
await fetch(PATHS.earthClientLogsApi, {
method: "POST",
headers: {
"Content-Type": "application/json",
},
body: JSON.stringify({
level,
message,
category,
module,
url: window.location.href,
detail: normalizedDetail.slice(0, 4000),
}),
body: JSON.stringify(payload),
keepalive: true,
});
} catch {
@@ -71,6 +85,61 @@ export async function reportEarthClientLog({
}
}
function scheduleFlush() {
if (pendingFlushTimer) return;
pendingFlushTimer = window.setTimeout(() => {
pendingFlushTimer = null;
const pending = Array.from(pendingEventMap.values());
pendingEventMap.clear();
pending.forEach((entry) => {
void sendEarthClientLog({
level: entry.level,
message: entry.message,
category: entry.category,
module: entry.module,
url: window.location.href,
detail: entry.detail.slice(0, 4000),
fingerprint: entry.fingerprint,
occurrence_count: entry.occurrenceCount,
metadata: entry.metadata,
});
});
}, 1000);
}
export function reportEarthClientLog({
level = "error",
message,
category = "runtime",
module = "earth",
detail = "",
metadata = {},
}) {
if (!message) return;
const normalizedDetail = normalizeErrorDetail(detail);
const fingerprint = buildFingerprint(level, message, normalizedDetail, category, module);
const key = dedupeKey(level, message, normalizedDetail, category);
const existing = pendingEventMap.get(key);
if (existing) {
existing.occurrenceCount += 1;
existing.metadata = { ...existing.metadata, ...metadata };
} else {
pendingEventMap.set(key, {
level,
message,
category,
module,
detail: normalizedDetail,
fingerprint,
occurrenceCount: 1,
metadata,
});
}
shouldSkip(level, message, normalizedDetail, category);
scheduleFlush();
}
export function registerEarthClientErrorHandlers() {
window.addEventListener("error", (event) => {
console.error("全局错误:", event.error);

View File

@@ -214,7 +214,6 @@ const computeCenterIconLayer = createInteractableLayer({
},
icon: {
coordinates: "canvas",
colorable: false,
fitSize: COMPUTE_CENTER_ICON_FIT_SIZE,
glowBlur: 16,
getSource({ marker, item }) {
@@ -248,6 +247,9 @@ const computeCenterIconLayer = createInteractableLayer({
pulseOffset: Math.random() * Math.PI * 2,
}),
avoidance: SURFACE_AVOIDANCE_PROFILES.city,
cluster: {
strategy: "stable-spherical",
},
});
export function formatComputeCenterTypeLabel(siteType) {

View File

@@ -64,6 +64,8 @@ export const SURFACE_HOVER_INFO_MODES = {
export const DEFAULT_SURFACE_HOVER_INFO_MODE =
SURFACE_HOVER_INFO_MODES.FULL;
export const EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48;
export const CRUISE_CONFIG = {
dwellMs: 7_000,
focusDurationMs: 1_400,
@@ -233,8 +235,12 @@ export const COUNTRY_BOUNDARY_CONFIG = {
tilePrefetchRing: 1,
tileDebounceMs: 180,
tileCacheLimit: 150,
lineAltitudeOffset: 0.115,
hoverAltitudeOffset: 0.115,
// Keep boundary/coastline strokes on the same shell as the high-res earth
// texture overlay. A lower or higher radius creates visible parallax while
// the globe rotates.
lineAltitudeOffset: EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET,
hoverAltitudeOffset: EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET,
claimLineAltitudeOffset: 0,
hoverMissStickyMs: 160,
lineColor: 0x7fc7ff,
lineOpacity: 0.58,
@@ -540,7 +546,7 @@ export const EARTH_MATERIAL_CONFIG = {
shininess: 12,
emissive: 0x010609,
opacity: 1,
textureOverlayAltitudeOffset: 0.48,
textureOverlayAltitudeOffset: EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET,
textureOverlayOpacity: 0.88,
textureOverlayRenderOrder: 0.96,
textureOverlaySpecular: 0x05080d,

View File

@@ -159,7 +159,7 @@ const SETTINGS_SHEET_MAX_SCALE_X = 0.22;
const SETTINGS_SHEET_MAX_SCALE_Y = 0.18;
const EARTH_SETTINGS_STORAGE_KEY = "planet.earth.settings.v2";
const LEGACY_EARTH_SETTINGS_STORAGE_KEY = "planet.earth.settings.v1";
const EARTH_SETTINGS_VERSION = 15;
const EARTH_SETTINGS_VERSION = 16;
const GRID_LINES_DEFAULT_VERSION = 3;
const SATELLITE_DISPLAY_DEFAULT_VERSION = 4;
const MEDIA_PANEL_DEFAULT_VERSION = 5;
@@ -172,6 +172,21 @@ const SURFACE_HOVER_INFO_DEFAULT_VERSION = 11;
const KEYBOARD_SHORTCUTS_DEFAULT_VERSION = 13;
const CRUISE_QUEUE_DEFAULT_VERSION = 14;
const AUTO_ROTATION_SPEED_DEFAULT_VERSION = 15;
const NEWS_CATEGORY_FILTERS_DEFAULT_VERSION = 16;
const DEFAULT_NEWS_CATEGORY_FILTERS = {
politics: true,
business: true,
ecommerce: true,
finance: true,
sports: true,
technology: true,
military: true,
disaster: true,
energy: true,
society: true,
culture: true,
other: true,
};
const AUTO_ROTATION_SPEED_MIN = 0.0001;
const AUTO_ROTATION_SPEED_MAX = 0.0015;
const AUTO_ROTATION_SPEED_STEP = 0.00005;
@@ -185,6 +200,11 @@ const KEYBOARD_ROTATION_STOP_SPEED = 0.012;
const KEYBOARD_ZOOM_STEP = 0.1;
const WHEEL_ZOOM_STEP = 0.1;
const WHEEL_ZOOM_DURATION_MS = 180;
const WHEEL_TRACKPAD_PIXEL_THRESHOLD = 48;
const WHEEL_TRACKPAD_DEADZONE = 0.35;
const WHEEL_TRACKPAD_RESIDUAL_WINDOW_MS = 140;
const WHEEL_TRACKPAD_RESIDUAL_RATIO = 0.65;
const WHEEL_TRACKPAD_SENSITIVITY = 0.0024;
const TARGET_SWITCH_ZOOM_IN_PHASE = 0.28;
const TARGET_SWITCH_ROTATE_PHASE = 0.5;
let settingsModalTimer = null;
@@ -655,7 +675,7 @@ function stopKeyboardRotationControl({ actionId = null, restoreAutoRotate = fals
}
function applyKeyboardZoom(direction) {
setZoomLevel(zoomLevel + direction * KEYBOARD_ZOOM_STEP, activeCamera);
setZoomLevel(getZoomLevelFromCamera(activeCamera) + direction * KEYBOARD_ZOOM_STEP, activeCamera);
showZoomStatusCapsule({ force: true });
}
@@ -1318,6 +1338,19 @@ function clampEarthZoomLevel(nextZoom) {
return Math.min(CONFIG.maxZoom, Math.max(CONFIG.minZoom, parsedZoom));
}
function getZoomLevelFromCamera(camera = activeCamera) {
const cameraZ = Number(camera?.position?.z);
if (!Number.isFinite(cameraZ) || cameraZ <= 0) {
return clampEarthZoomLevel(zoomLevel);
}
return clampEarthZoomLevel(CONFIG.defaultCameraZ / cameraZ);
}
function syncZoomLevelFromCamera(camera = activeCamera) {
zoomLevel = getZoomLevelFromCamera(camera);
return zoomLevel;
}
function formatZoomPercent(zoom) {
return `${Math.round(zoom * 100)}%`;
}
@@ -1349,6 +1382,23 @@ function canUseLocalStorage() {
}
}
function normalizeNewsCategoryFilters(filters) {
const normalized = { ...DEFAULT_NEWS_CATEGORY_FILTERS };
if (!filters || typeof filters !== "object") return normalized;
Object.keys(DEFAULT_NEWS_CATEGORY_FILTERS).forEach((category) => {
if (typeof filters[category] === "boolean") {
normalized[category] = filters[category];
}
});
return normalized;
}
function isNewsCategoryFilterEnabled(filters, category) {
const key = String(category || "").trim();
if (!(key in DEFAULT_NEWS_CATEGORY_FILTERS)) return true;
return normalizeNewsCategoryFilters(filters)[key] !== false;
}
function getCurrentPanelVisibilitySnapshot() {
return Object.fromEntries(
HUD_PANEL_IDS.map((panelId) => {
@@ -1382,6 +1432,9 @@ function getCurrentSharedSettingsSnapshot() {
satelliteRealAltitudeEnabled: getSatelliteRealAltitudeEnabled(),
interactableCompactDotsEnabled: getInteractableCompactDotsEnabled(),
surfaceHoverInfoMode: getSurfaceHoverInfoMode(),
newsCategoryFilters: normalizeNewsCategoryFilters(
earthSettingsState?.shared?.newsCategoryFilters,
),
keyboardShortcuts: normalizeKeyboardShortcuts(keyboardShortcuts),
};
}
@@ -1446,6 +1499,7 @@ function cloneEarthSettings(settings) {
surfaceHoverInfoMode: normalizeSurfaceHoverInfoMode(
settings.shared.surfaceHoverInfoMode,
),
newsCategoryFilters: normalizeNewsCategoryFilters(settings.shared.newsCategoryFilters),
keyboardShortcuts: normalizeKeyboardShortcuts(settings.shared.keyboardShortcuts),
layerVisibility: { ...(settings.shared.layerVisibility || {}) },
},
@@ -1593,6 +1647,13 @@ function normalizeEarthSettings(rawSettings, defaults) {
(rawSettings?.version || 0) >= KEYBOARD_SHORTCUTS_DEFAULT_VERSION
? normalizeKeyboardShortcuts(sharedSettings?.keyboardShortcuts)
: defaults.shared.keyboardShortcuts;
const legacyNewsCategoryFilters = sharedSettings?.["display" + "Types"]?.news;
const nextNewsCategoryFilters =
(rawSettings?.version || 0) >= NEWS_CATEGORY_FILTERS_DEFAULT_VERSION
? normalizeNewsCategoryFilters(
sharedSettings?.newsCategoryFilters || legacyNewsCategoryFilters,
)
: normalizeNewsCategoryFilters(defaults.shared.newsCategoryFilters);
const nextCruiseQueueMode =
(rawSettings?.version || 0) >= CRUISE_QUEUE_DEFAULT_VERSION
? normalizeCruiseQueueMode(sharedSettings?.cruiseQueueMode)
@@ -1633,6 +1694,7 @@ function normalizeEarthSettings(rawSettings, defaults) {
satelliteRealAltitudeEnabled: nextSatelliteRealAltitudeEnabled,
interactableCompactDotsEnabled: nextInteractableCompactDotsEnabled,
surfaceHoverInfoMode: nextSurfaceHoverInfoMode,
newsCategoryFilters: nextNewsCategoryFilters,
keyboardShortcuts: nextKeyboardShortcuts,
},
views: {
@@ -1919,6 +1981,71 @@ function syncInteractableCompactDotsToggle() {
});
}
function syncNewsCategoryFilterControls() {
const filters = normalizeNewsCategoryFilters(
earthSettingsState?.shared?.newsCategoryFilters,
);
document.querySelectorAll("[data-news-category-toggle]").forEach((button) => {
if (!(button instanceof HTMLButtonElement)) return;
const category = button.dataset.newsCategoryToggle || "";
const active = isNewsCategoryFilterEnabled(filters, category);
button.classList.toggle("is-active", active);
button.setAttribute("aria-pressed", active ? "true" : "false");
});
}
function dispatchNewsCategoryFiltersChange(
filters = earthSettingsState?.shared?.newsCategoryFilters,
) {
window.dispatchEvent(
new CustomEvent("earth:news-category-filters-change", {
detail: {
categories: normalizeNewsCategoryFilters(filters),
},
}),
);
}
function applyNewsCategoryFilters(filters = earthSettingsState?.shared?.newsCategoryFilters) {
const normalized = normalizeNewsCategoryFilters(filters);
syncNewsCategoryFilterControls();
dispatchNewsCategoryFiltersChange(normalized);
}
export function getEarthNewsCategoryFilters() {
return normalizeNewsCategoryFilters(earthSettingsState?.shared?.newsCategoryFilters);
}
export function isEarthNewsCategoryEnabled(category) {
return isNewsCategoryFilterEnabled(
earthSettingsState?.shared?.newsCategoryFilters,
category,
);
}
export function setEarthNewsCategoryEnabled(
category,
enabled,
{ persist = true, suppressStatus = false } = {},
) {
const key = String(category || "").trim();
if (!(key in DEFAULT_NEWS_CATEGORY_FILTERS)) return false;
ensureMutableEarthSettingsState();
const nextFilters = normalizeNewsCategoryFilters(earthSettingsState.shared.newsCategoryFilters);
nextFilters[key] = Boolean(enabled);
earthSettingsState.shared.newsCategoryFilters = nextFilters;
applyNewsCategoryFilters(nextFilters);
if (persist) {
persistEarthSettings();
}
if (!suppressStatus) {
showStatusMessage(Boolean(enabled) ? "新闻类型已显示" : "新闻类型已隐藏", "info");
}
return true;
}
function syncSurfaceHoverInfoModeControls() {
const activeMode = getSurfaceHoverInfoMode();
document.querySelectorAll("[data-surface-hover-info-mode]").forEach((button) => {
@@ -2188,8 +2315,7 @@ function setDefaultEarthZoom(nextZoom, { persist = true, applyToCurrentView = tr
syncDefaultEarthZoomUi(defaultEarthZoom);
if (applyToCurrentView && activeCamera) {
zoomLevel = defaultEarthZoom;
applyZoom(activeCamera);
setZoomLevel(defaultEarthZoom, activeCamera);
}
if (persist) {
@@ -2278,6 +2404,7 @@ async function applyEarthSettings(settings, { applyLayers = true } = {}) {
const layerVisibility = { ...(settings.shared.layerVisibility || {}) };
applyImmediateLayerVisibilityHints(layerVisibility);
deferredLayerVisibilitySettings = layerVisibility;
applyNewsCategoryFilters(settings.shared.newsCategoryFilters);
return;
}
@@ -2286,6 +2413,7 @@ async function applyEarthSettings(settings, { applyLayers = true } = {}) {
persist: false,
silent: true,
});
applyNewsCategoryFilters(settings.shared.newsCategoryFilters);
}
export function getMotionDebugEnabled() {
@@ -2398,6 +2526,7 @@ export async function applyDeferredLayerVisibilitySettings(options = {}) {
silent: true,
...options,
});
applyNewsCategoryFilters(earthSettingsState?.shared?.newsCategoryFilters);
}
function resetEarthSettings() {
@@ -3110,12 +3239,7 @@ export function applyImmediateView(targetEarthObj, camera, options = {}) {
targetEarthObj.rotation.x = nextRotation.x;
targetEarthObj.rotation.y = nextRotation.y;
targetEarthObj.rotation.z = nextRotation.z;
zoomLevel = zoom;
if (camera) {
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
updateZoomDisplay(zoomLevel, camera.position.z.toFixed(0));
}
setZoomLevel(zoom, camera);
}
export function setZoomLevel(nextZoom, camera = activeCamera) {
@@ -3127,13 +3251,16 @@ export function setZoomLevel(nextZoom, camera = activeCamera) {
return zoomLevel;
}
export function showZoomStatusCapsule({ force = false } = {}) {
export function showZoomStatusCapsule({ force = false, zoom = null } = {}) {
const now = Date.now();
if (!force && now - lastZoomStatusUpdateTime < ZOOM_STATUS_UPDATE_INTERVAL_MS) {
return;
}
lastZoomStatusUpdateTime = now;
showGestureStatusMessage(`缩放 ${Math.round(zoomLevel * 100)}%`, "info");
const currentZoom = Number.isFinite(Number(zoom))
? clampEarthZoomLevel(zoom)
: syncZoomLevelFromCamera(activeCamera);
showGestureStatusMessage(`缩放 ${Math.round(currentZoom * 100)}%`, "info");
}
function cancelSettingsSheetAnimation() {
@@ -4266,6 +4393,19 @@ function setupSettingsControls() {
});
});
document.querySelectorAll("[data-news-category-toggle]").forEach((toggle) => {
if (!(toggle instanceof HTMLButtonElement)) return;
bindListener(toggle, "click", () => {
const active = toggle.classList.contains("is-active");
setEarthNewsCategoryEnabled(toggle.dataset.newsCategoryToggle, !active);
});
});
bindListener(window, "earth:set-news-category-enabled", (event) => {
const detail = event.detail || {};
setEarthNewsCategoryEnabled(detail.category, Boolean(detail.enabled));
});
document.querySelectorAll("#toggle-daynight, [data-daynight-toggle]").forEach((dayNightToggle) => {
if (!(dayNightToggle instanceof HTMLInputElement)) return;
bindListener(dayNightToggle, "change", () => {
@@ -4308,6 +4448,7 @@ function setupSettingsControls() {
syncSatelliteIdleBreathingToggle();
syncSatelliteRealAltitudeToggle();
syncInteractableCompactDotsToggle();
syncNewsCategoryFilterControls();
syncSurfaceHoverInfoModeControls();
syncDayNightToggle(dayNightEnabled);
syncMotionDebugToggle(motionDebugEnabled);
@@ -4692,27 +4833,25 @@ function setupZoomControls(camera) {
const MAX_PERCENT = CONFIG.maxZoom * 100;
function doZoomStep(direction) {
let currentPercent = Math.round(zoomLevel * 100);
let currentPercent = Math.round(getZoomLevelFromCamera(camera) * 100);
let newPercent =
direction > 0 ? currentPercent + CLICK_STEP : currentPercent - CLICK_STEP;
if (newPercent > MAX_PERCENT) newPercent = MAX_PERCENT;
if (newPercent < MIN_PERCENT) newPercent = MIN_PERCENT;
zoomLevel = newPercent / 100;
applyZoom(camera);
setZoomLevel(newPercent / 100, camera);
showZoomStatusCapsule({ force: true });
}
function doContinuousZoom(direction) {
let currentPercent = Math.round(zoomLevel * 100);
let currentPercent = Math.round(getZoomLevelFromCamera(camera) * 100);
let newPercent = direction > 0 ? currentPercent + 1 : currentPercent - 1;
if (newPercent > MAX_PERCENT) newPercent = MAX_PERCENT;
if (newPercent < MIN_PERCENT) newPercent = MIN_PERCENT;
zoomLevel = newPercent / 100;
applyZoom(camera);
setZoomLevel(newPercent / 100, camera);
showZoomStatusCapsule();
}
@@ -4775,10 +4914,8 @@ function setupZoomControls(camera) {
bindListener(zoomOut, "touchend", () => handleMouseUp(-1));
bindListener(zoomValue, "click", () => {
const startZoomVal = zoomLevel;
const startZoomVal = getZoomLevelFromCamera(camera);
const targetZoom = getDefaultEarthZoomLevel();
const startDistance = CONFIG.defaultCameraZ / startZoomVal;
const targetDistance = CONFIG.defaultCameraZ / targetZoom;
animateValue(
0,
@@ -4786,14 +4923,11 @@ function setupZoomControls(camera) {
600,
(progress) => {
const ease = 1 - Math.pow(1 - progress, 3);
zoomLevel = startZoomVal + (targetZoom - startZoomVal) * ease;
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
const distance =
startDistance + (targetDistance - startDistance) * ease;
updateZoomDisplay(zoomLevel, distance.toFixed(0));
const nextZoom = startZoomVal + (targetZoom - startZoomVal) * ease;
setZoomLevel(nextZoom, camera);
},
() => {
zoomLevel = targetZoom;
setZoomLevel(targetZoom, camera);
showStatusMessage(getZoomResetStatusMessage(targetZoom), "info");
},
);
@@ -4802,9 +4936,51 @@ function setupZoomControls(camera) {
function setupWheelZoom(camera, renderer) {
let wheelZoomFrameId = null;
let wheelZoomTarget = zoomLevel;
let wheelZoomStart = zoomLevel;
let wheelZoomTarget = getZoomLevelFromCamera(camera);
let wheelZoomStart = wheelZoomTarget;
let wheelZoomStartAt = 0;
let lastTrackpadDirection = 0;
let suppressedTrackpadDirection = 0;
let suppressedTrackpadUntil = 0;
let suppressedTrackpadMagnitude = 0;
function getWheelPixelDelta(event) {
if (event.deltaMode === WheelEvent.DOM_DELTA_LINE) {
return event.deltaY * 16;
}
if (event.deltaMode === WheelEvent.DOM_DELTA_PAGE) {
return event.deltaY * window.innerHeight;
}
return event.deltaY;
}
function isTrackpadWheel(event, pixelDelta) {
return event.deltaMode === WheelEvent.DOM_DELTA_PIXEL &&
Math.abs(pixelDelta) < WHEEL_TRACKPAD_PIXEL_THRESHOLD;
}
function shouldSuppressTrackpadResidual(pixelDelta) {
const direction = Math.sign(pixelDelta);
const magnitude = Math.abs(pixelDelta);
const now = performance.now();
return direction !== 0 &&
direction === suppressedTrackpadDirection &&
now < suppressedTrackpadUntil &&
magnitude < suppressedTrackpadMagnitude * WHEEL_TRACKPAD_RESIDUAL_RATIO;
}
function recordTrackpadWheel(pixelDelta) {
const direction = Math.sign(pixelDelta);
const magnitude = Math.abs(pixelDelta);
if (direction === 0) return;
const now = performance.now();
if (lastTrackpadDirection !== 0 && direction !== lastTrackpadDirection) {
suppressedTrackpadDirection = lastTrackpadDirection;
suppressedTrackpadUntil = now + WHEEL_TRACKPAD_RESIDUAL_WINDOW_MS;
suppressedTrackpadMagnitude = magnitude;
}
lastTrackpadDirection = direction;
}
function stopWheelZoomAnimation() {
if (wheelZoomFrameId !== null) {
@@ -4824,28 +5000,55 @@ function setupWheelZoom(camera, renderer) {
1,
);
const ease = 1 - Math.pow(1 - progress, 3);
zoomLevel = wheelZoomStart + (wheelZoomTarget - wheelZoomStart) * ease;
applyZoom(camera);
setZoomLevel(
wheelZoomStart + (wheelZoomTarget - wheelZoomStart) * ease,
camera,
);
if (progress < 1) {
wheelZoomFrameId = window.requestAnimationFrame(animateWheelZoom);
return;
}
zoomLevel = wheelZoomTarget;
applyZoom(camera);
setZoomLevel(wheelZoomTarget, camera);
wheelZoomFrameId = null;
wheelZoomStartAt = 0;
}
function startWheelZoomAnimation() {
wheelZoomStart = zoomLevel;
wheelZoomStart = getZoomLevelFromCamera(camera);
wheelZoomStartAt = 0;
if (wheelZoomFrameId === null) {
wheelZoomFrameId = window.requestAnimationFrame(animateWheelZoom);
}
}
function applyMouseWheelZoom(direction) {
suppressedTrackpadDirection = 0;
suppressedTrackpadUntil = 0;
const baseZoom = wheelZoomFrameId === null
? getZoomLevelFromCamera(camera)
: wheelZoomTarget;
wheelZoomTarget = clampEarthZoomLevel(
baseZoom + direction * WHEEL_ZOOM_STEP,
);
stopWheelZoomAnimation();
startWheelZoomAnimation();
showZoomStatusCapsule({ force: true, zoom: wheelZoomTarget });
}
function applyTrackpadWheelZoom(pixelDelta) {
if (Math.abs(pixelDelta) < WHEEL_TRACKPAD_DEADZONE) return;
if (shouldSuppressTrackpadResidual(pixelDelta)) return;
stopWheelZoomAnimation();
const currentZoom = getZoomLevelFromCamera(camera);
const nextZoom = currentZoom * Math.exp(-pixelDelta * WHEEL_TRACKPAD_SENSITIVITY);
wheelZoomTarget = setZoomLevel(nextZoom, camera);
wheelZoomStart = wheelZoomTarget;
recordTrackpadWheel(pixelDelta);
showZoomStatusCapsule({ force: true, zoom: wheelZoomTarget });
}
cleanupFns.push(stopWheelZoomAnimation);
bindListener(
@@ -4853,25 +5056,17 @@ function setupWheelZoom(camera, renderer) {
"wheel",
(e) => {
e.preventDefault();
const direction = e.deltaY < 0 ? 1 : -1;
const baseZoom =
wheelZoomFrameId === null ? zoomLevel : wheelZoomTarget;
wheelZoomTarget = clampEarthZoomLevel(
baseZoom + direction * WHEEL_ZOOM_STEP,
);
startWheelZoomAnimation();
showZoomStatusCapsule({ force: true });
const pixelDelta = getWheelPixelDelta(e);
if (isTrackpadWheel(e, pixelDelta)) {
applyTrackpadWheelZoom(pixelDelta);
return;
}
applyMouseWheelZoom(pixelDelta < 0 ? 1 : -1);
},
{ passive: false },
);
}
function applyZoom(camera) {
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
const distance = camera.position.z.toFixed(0);
updateZoomDisplay(zoomLevel, distance);
}
function animateValue(start, end, duration, onUpdate, onComplete) {
const animationToken = ++focusViewAnimationToken;
const startTime = performance.now();
@@ -5800,7 +5995,7 @@ export function focusEarthView(camera, options = {}) {
const startRotX = earthObj.rotation.x;
const startRotY = earthObj.rotation.y;
const startRotZ = earthObj.rotation.z;
const startZoom = zoomLevel;
const startZoom = getZoomLevelFromCamera(camera);
const defaultZoom = getDefaultEarthZoomLevel();
const shouldRestoreZoomViaDefault =
zoomTransitionMode === "restore-current-via-default" &&
@@ -5829,30 +6024,26 @@ export function focusEarthView(camera, options = {}) {
if (progress < rotateStartProgress) {
const zoomProgress = progress / rotateStartProgress;
const zoomEase = 1 - Math.pow(1 - zoomProgress, 3);
zoomLevel = startZoom + (defaultZoom - startZoom) * zoomEase;
setZoomLevel(startZoom + (defaultZoom - startZoom) * zoomEase, camera);
} else if (progress <= rotateEndProgress) {
zoomLevel = defaultZoom;
setZoomLevel(defaultZoom, camera);
} else {
const zoomProgress = (progress - rotateEndProgress) / (1 - rotateEndProgress);
const zoomEase = 1 - Math.pow(1 - zoomProgress, 3);
zoomLevel = defaultZoom + (startZoom - defaultZoom) * zoomEase;
setZoomLevel(defaultZoom + (startZoom - defaultZoom) * zoomEase, camera);
}
} else {
earthObj.rotation.x = startRotX + (nextRotation.x - startRotX) * ease;
earthObj.rotation.y = startRotY + (nextRotation.y - startRotY) * ease;
earthObj.rotation.z = startRotZ + (nextRotation.z - startRotZ) * ease;
zoomLevel = startZoom + (zoom - startZoom) * ease;
setZoomLevel(startZoom + (zoom - startZoom) * ease, camera);
}
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
updateZoomDisplay(zoomLevel, camera.position.z.toFixed(0));
},
() => {
zoomLevel = shouldRestoreZoomViaDefault ? startZoom : zoom;
setZoomLevel(shouldRestoreZoomViaDefault ? startZoom : zoom, camera);
earthObj.rotation.x = nextRotation.x;
earthObj.rotation.y = nextRotation.y;
earthObj.rotation.z = nextRotation.z;
camera.position.z = CONFIG.defaultCameraZ / zoomLevel;
updateZoomDisplay(zoomLevel, camera.position.z.toFixed(0));
if (!suppressStatus) {
showStatusMessage("视角已重置", "info");
}
@@ -5863,7 +6054,7 @@ export function focusEarthView(camera, options = {}) {
}
export function getZoomLevel() {
return zoomLevel;
return syncZoomLevelFromCamera(activeCamera);
}
export function getDefaultEarthZoomLevel() {

View File

@@ -210,7 +210,7 @@ function boundaryLineRadius({ claim = false } = {}) {
return (
CONFIG.earthRadius +
COUNTRY_BOUNDARY_CONFIG.lineAltitudeOffset +
(claim ? 0.018 : 0)
(claim ? COUNTRY_BOUNDARY_CONFIG.claimLineAltitudeOffset : 0)
);
}

View File

@@ -1,7 +1,6 @@
import { PATHS } from "./constants.js";
import { createInteractableLayer, SURFACE_AVOIDANCE_PROFILES } from "./interactable.js";
let showEarthInteractables = true;
const interactableRevisions = new Map();
function featureToInteractableItem(feature) {
@@ -93,6 +92,9 @@ const earthInteractableLayer = createInteractableLayer({
...item,
type: "earth_interactable",
}),
cluster: {
strategy: "stable-spherical",
},
});
export async function loadEarthInteractables(earth, { silent = false } = {}) {
@@ -109,7 +111,7 @@ export async function loadEarthInteractables(earth, { silent = false } = {}) {
items.forEach((item) => interactableRevisions.set(item.id, Number(item.revision || 0)));
earthInteractableLayer.setData(items);
earthInteractableLayer.attach(earth);
earthInteractableLayer.setVisible(showEarthInteractables);
earthInteractableLayer.setVisible(true);
if (!silent) {
console.info("Earth interactables loaded", { count: items.length });
}
@@ -156,7 +158,7 @@ export function applyEarthInteractableEvent(earth, payload = {}) {
const changed = earthInteractableLayer.upsertItem(item);
interactableRevisions.set(id, nextRevision || Number(item.revision || 0));
earthInteractableLayer.attach(earth);
earthInteractableLayer.setVisible(showEarthInteractables);
earthInteractableLayer.setVisible(true);
return changed;
}

View File

@@ -287,10 +287,28 @@ function renderNewsCardContent(content, data) {
if (!(content instanceof HTMLElement)) return;
const summary = getNewsCardSummary(data);
const title = getNewsCardTitle(data);
const metaItems = [
['媒体来源', data?.source],
['RSS 来源', data?.feedName],
['源类型', data?.feedSourceTypeLabel],
['抓取通道', data?.fetchChannelLabel],
['新闻类型', data?.categoryLabel],
['区域', data?.regionLabel],
['发布时间', data?.publishedAtDisplay],
].filter(([, value]) => String(value ?? '').trim());
const metaHtml = metaItems.length
? `<div class="info-card-news-meta-grid">${metaItems.map(([label, value]) => `
<div class="info-card-news-meta-item">
<span>${escapeInfoCardHtml(label)}</span>
<strong>${escapeInfoCardHtml(value)}</strong>
</div>
`).join('')}</div>`
: '';
content.innerHTML = `
<div class="info-card-news-layout">
<div class="info-card-news-kicker">新闻信号</div>
<div class="info-card-news-title">${escapeInfoCardHtml(title)}</div>
${metaHtml}
<div class="info-card-news-summary-shell">
<div class="info-card-news-summary-label">概要</div>
<div class="info-card-news-summary" data-news-summary></div>
@@ -305,10 +323,25 @@ function renderMobileNewsCardContent(content, data) {
if (!(content instanceof HTMLElement)) return;
const summary = getNewsCardSummary(data);
const title = getNewsCardTitle(data);
const metaItems = [
['来源', data?.source],
['RSS', data?.feedName],
['类型', data?.categoryLabel],
['区域', data?.regionLabel],
].filter(([, value]) => String(value ?? '').trim());
const metaHtml = metaItems.length
? `<div class="info-card-news-meta-grid info-card-news-meta-grid--mobile">${metaItems.map(([label, value]) => `
<div class="info-card-news-meta-item">
<span>${escapeInfoCardHtml(label)}</span>
<strong>${escapeInfoCardHtml(value)}</strong>
</div>
`).join('')}</div>`
: '';
content.innerHTML = `
<div class="earth-mobile-news-detail">
<div class="earth-mobile-news-detail-kicker">新闻信号</div>
<div class="earth-mobile-news-detail-title">${escapeInfoCardHtml(title)}</div>
${metaHtml}
<div class="earth-mobile-news-detail-summary-shell">
<div class="earth-mobile-news-detail-summary-label">概要</div>
<div class="earth-mobile-news-detail-summary" data-news-summary></div>

File diff suppressed because it is too large Load Diff

View File

@@ -15,12 +15,18 @@ import {
SCENE_LIGHT_CONFIG,
SURFACE_HOVER_INFO_MODES,
} from "./constants.js";
import {
canAttemptEarthRealtime,
getEarthRealtimeCooldownMs,
getEarthRealtimeUrl,
recordEarthRealtimeFailure,
recordEarthRealtimeOpen,
} from "./realtime.js";
import { vector3ToLatLon, screenToEarthCoords } from "./utils.js";
import {
showStatusMessage,
queueStatusMessage,
updateCoordinatesDisplay,
updateZoomDisplay,
updateEarthStats,
setEarthStatValue,
setLoading,
@@ -301,6 +307,8 @@ export let scene;
export let camera;
export let renderer;
const WEBGL_GPU_DIAGNOSTICS_EVENT = "earth:gpu-diagnostics";
let isDragging = false;
let previousMousePosition = { x: 0, y: 0 };
let targetRotation = { x: 0, y: 0 };
@@ -451,6 +459,29 @@ function getViewportAspect() {
return window.innerWidth / window.innerHeight;
}
function getWebGLGpuDiagnostics(activeRenderer) {
const gl = activeRenderer?.getContext?.();
if (!gl) return null;
const debugInfo = gl.getExtension?.("WEBGL_debug_renderer_info");
const attrs = gl.getContextAttributes?.() || {};
return {
requestedPowerPreference: "high-performance",
contextPowerPreference: attrs.powerPreference || null,
vendor: debugInfo ? gl.getParameter(debugInfo.UNMASKED_VENDOR_WEBGL) : gl.getParameter(gl.VENDOR),
renderer: debugInfo ? gl.getParameter(debugInfo.UNMASKED_RENDERER_WEBGL) : gl.getParameter(gl.RENDERER),
antialias: Boolean(attrs.antialias),
alpha: Boolean(attrs.alpha),
};
}
function publishWebGLGpuDiagnostics(activeRenderer) {
const diagnostics = getWebGLGpuDiagnostics(activeRenderer);
if (!diagnostics) return;
window.__planetEarthGpu = diagnostics;
console.info("[智能星球] WebGL GPU diagnostics", diagnostics);
window.dispatchEvent(new CustomEvent(WEBGL_GPU_DIAGNOSTICS_EVENT, { detail: diagnostics }));
}
function syncRendererViewport() {
if (!camera || !renderer) return;
camera.aspect = getViewportAspect();
@@ -793,12 +824,30 @@ function isSameVessel(marker1, marker2) {
return Boolean(marker1 && marker2 && marker1.userData?.mmsi === marker2.userData?.mmsi);
}
function getFirstObjectIntersection(intersections) {
return intersections.find((hit) => !hit?.cluster && hit?.object)?.object || null;
}
function getPrimaryClusterHit(...intersectionGroups) {
return intersectionGroups
.flat()
.filter((hit) => hit?.cluster && Number(hit.clusterCount) > 1)
.sort((a, b) => a.distancePxSq - b.distancePxSq)[0] || null;
}
function getClusterBriefHtml(hit) {
const count = Number(hit?.clusterCount || hit?.clusterMarkers?.length || 0);
return `<strong>共 ${count} 个对象</strong><br><span>放大后可查看单个图标</span>`;
}
function getPrimaryBGPHoverTarget(bgpAnomalyIntersects, bgpCollectorIntersects) {
if (bgpAnomalyIntersects.length > 0) {
return bgpAnomalyIntersects[0].object;
const anomalyMarker = getFirstObjectIntersection(bgpAnomalyIntersects);
if (anomalyMarker) {
return anomalyMarker;
}
if (bgpCollectorIntersects.length > 0) {
return bgpCollectorIntersects[0].object;
const collectorMarker = getFirstObjectIntersection(bgpCollectorIntersects);
if (collectorMarker) {
return collectorMarker;
}
return null;
}
@@ -808,8 +857,8 @@ function getPrimaryBGPClickTarget(
bgpAnomalyIntersects,
bgpCollectorIntersects,
) {
const anomalyMarker = bgpAnomalyIntersects[0]?.object || null;
const collectorMarker = bgpCollectorIntersects[0]?.object || null;
const anomalyMarker = getFirstObjectIntersection(bgpAnomalyIntersects);
const collectorMarker = getFirstObjectIntersection(bgpCollectorIntersects);
if (!anomalyMarker && !collectorMarker) return null;
if (!anomalyMarker) return collectorMarker;
if (!collectorMarker) return anomalyMarker;
@@ -952,19 +1001,23 @@ function getMotionIconCandidates() {
const candidates = [];
if (getShowBGP()) {
getBGPEventIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "bgp", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
getBGPCollectorIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "bgp_collector", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
}
if (getShowComputeCenters()) {
getComputeCenterIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "compute_center", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
}
if (getShowVessels()) {
getVesselIconPointerIntersections(sharedOptions).forEach((hit) => {
if (hit.cluster || !hit.object) return;
candidates.push({ type: "vessel", object: hit.object, screen: getMotionScreenPointFromWorld(hit.point), distancePxSq: hit.distancePxSq });
});
}
@@ -3961,7 +4014,7 @@ export function init() {
0.1,
5000,
);
camera.position.z = CONFIG.defaultCameraZ;
setZoomLevel(getDefaultEarthZoomLevel(), camera);
setSatelliteCamera(camera);
renderer = new THREE.WebGLRenderer({
@@ -3972,6 +4025,7 @@ export function init() {
syncRendererViewport();
renderer.setClearColor(0x02040a, 1);
renderer.setPixelRatio(window.devicePixelRatio);
publishWebGLGpuDiagnostics(renderer);
const container = document.getElementById("container");
if (container) {
@@ -4240,17 +4294,13 @@ async function hydrateAllSatellitesInBackground(guardFn) {
}
}
function getEarthUpdatesRealtimeUrl() {
const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
return `${protocol}//${window.location.host}/ws`;
}
function scheduleEarthUpdatesReconnect() {
if (earthUpdatesReconnectTimer || destroyed) return;
const delay = Math.max(EARTH_UPDATES_RECONNECT_DELAY_MS, getEarthRealtimeCooldownMs());
earthUpdatesReconnectTimer = window.setTimeout(() => {
earthUpdatesReconnectTimer = null;
connectEarthUpdatesRealtime();
}, EARTH_UPDATES_RECONNECT_DELAY_MS);
}, delay);
}
async function refreshBGPRealtimeLayer() {
@@ -4519,10 +4569,16 @@ function startEarthDataReconciliation() {
}
function connectEarthUpdatesRealtime() {
if (earthUpdatesSocket || typeof WebSocket === "undefined" || destroyed) return;
const socket = new WebSocket(getEarthUpdatesRealtimeUrl());
if (earthUpdatesSocket || destroyed) return;
if (!canAttemptEarthRealtime()) {
scheduleEarthUpdatesReconnect();
return;
}
const socket = new WebSocket(getEarthRealtimeUrl());
earthUpdatesSocket = socket;
socket.onopen = () => {
socket.__planetOpened = true;
recordEarthRealtimeOpen();
if (earthUpdatesReconnectTimer) {
window.clearTimeout(earthUpdatesReconnectTimer);
earthUpdatesReconnectTimer = null;
@@ -4552,6 +4608,9 @@ function connectEarthUpdatesRealtime() {
if (earthUpdatesSocket === socket) {
earthUpdatesSocket = null;
}
if (!socket.__planetOpened) {
recordEarthRealtimeFailure();
}
scheduleEarthUpdatesReconnect();
};
socket.onerror = () => {
@@ -5287,13 +5346,17 @@ function onMouseMove(event) {
bgpAnomalyIntersects,
bgpCollectorIntersects,
);
const hoveredClusterHit = getPrimaryClusterHit(
bgpAnomalyIntersects,
bgpCollectorIntersects,
computeCenterIntersects,
);
if (hoveredBGP && !isSameBGPMarker(hoveredBGP, hoveredBGPMarker)) {
clearTransientHoverState();
}
const hoveredComputeCenterMarker =
computeCenterIntersects.length > 0 ? computeCenterIntersects[0].object : null;
const hoveredComputeCenterMarker = getFirstObjectIntersection(computeCenterIntersects);
const hoveredVesselMarker =
vesselPick.checked && vesselIntersects.length > 0 ? vesselIntersects[0].object : null;
const earthPoint = screenToEarthCoords(
@@ -5357,6 +5420,20 @@ function onMouseMove(event) {
let objectTooltipShown = false;
if (
hoveredClusterHit &&
!hoveredBGPMarker &&
!hoveredComputeCenterMarker &&
lockedObjectType !== "bgp" &&
lockedObjectType !== "bgp_collector" &&
lockedObjectType !== "compute_center"
) {
showTooltip(
event.clientX + TOOLTIP_CURSOR_OFFSET,
event.clientY + TOOLTIP_CURSOR_OFFSET,
getClusterBriefHtml(hoveredClusterHit),
);
objectTooltipShown = true;
} else if (
hoveredBGPMarker &&
getShowBGP() &&
lockedObjectType !== "bgp" &&
@@ -5663,9 +5740,7 @@ function onClick(event) {
const clickedBGPMarker = getShowBGP()
? getPrimaryBGPClickTarget(event, bgpAnomalyIntersects, bgpCollectorIntersects)
: null;
const clickedComputeCenterMarker = computeCenterIntersects.length > 0
? computeCenterIntersects[0].object
: null;
const clickedComputeCenterMarker = getFirstObjectIntersection(computeCenterIntersects);
const clickedVesselMarker = vesselIntersects.length > 0
? vesselIntersects[0].object
: null;

View File

@@ -16,9 +16,12 @@ import {
import {
getNewsDisplaySummary,
getNewsDisplayTitle,
getNewsCategoryLabel,
getNewsFetchChannelLabel,
getNewsFeedLabel,
getNewsLocationSourceLabel,
getNewsRegionLabel,
getNewsSourceTypeLabel,
} from "./news-locale.js";
const CRUISE_PRESENTATION_HIDE_MS = 220;
@@ -89,6 +92,9 @@ function mapNewsItemToCruiseEvent(item) {
return null;
}
const rawFeedName = item.feed_name || "";
const sourceType = item.source_type || (String(rawFeedName).startsWith("Global Monitor /") ? "aggregated" : "rss");
return {
id: `news:${item.id}`,
sourceId: item.id,
@@ -96,7 +102,11 @@ function mapNewsItemToCruiseEvent(item) {
title: getNewsDisplayTitle(item),
summary: getNewsDisplaySummary(item),
source: item.source || "",
feedName: getNewsFeedLabel(item.feed_name),
feedName: getNewsFeedLabel(rawFeedName),
rawFeedName,
feedSourceTypeLabel: getNewsSourceTypeLabel(sourceType),
fetchChannelLabel: getNewsFetchChannelLabel(rawFeedName, sourceType),
categoryLabel: getNewsCategoryLabel(item.category),
region: item.region || "global",
regionLabel: item.display_region || getNewsRegionLabel(item.region),
url: item.url || "",
@@ -275,6 +285,13 @@ export function createNewsCruiseAdapter({ camera, earth, connector, focusView })
showInfoCard("news", {
title: item.title,
summary: item.summary || item.title || "",
source: item.source,
feedName: item.rawFeedName || item.feedName,
feedSourceTypeLabel: item.feedSourceTypeLabel,
fetchChannelLabel: item.fetchChannelLabel,
categoryLabel: item.categoryLabel,
regionLabel: item.regionLabel,
publishedAtDisplay: item.publishedAtDisplay,
}, {
x: placement.x,
y: placement.y,
@@ -320,6 +337,13 @@ export function createNewsCruiseAdapter({ camera, earth, connector, focusView })
showInfoCard("news", {
title: item.title,
summary: item.summary || item.title || "",
source: item.source,
feedName: item.rawFeedName || item.feedName,
feedSourceTypeLabel: item.feedSourceTypeLabel,
fetchChannelLabel: item.fetchChannelLabel,
categoryLabel: item.categoryLabel,
regionLabel: item.regionLabel,
publishedAtDisplay: item.publishedAtDisplay,
}, {
x: placement.x,
y: placement.y,

View File

@@ -9,11 +9,33 @@ const REGION_LABELS = {
};
const FEED_LABELS = {
"Global Monitor / World": "全球监测",
"Global Monitor / Americas": "美洲监测",
"Global Monitor / Europe": "欧洲监测",
"Global Monitor / MEA": "中东与非洲监测",
"Global Monitor / APAC": "亚太监测",
"Global Monitor / World": "区域监测",
"Global Monitor / Americas": "区域监测",
"Global Monitor / Europe": "区域监测",
"Global Monitor / MEA": "区域监测",
"Global Monitor / APAC": "区域监测",
};
const CATEGORY_LABELS = {
politics: "政治",
business: "商业",
ecommerce: "电商",
finance: "金融",
sports: "体育",
technology: "科技",
military: "军事",
disaster: "灾害",
energy: "能源",
society: "社会",
culture: "文化",
other: "其他",
};
const SOURCE_TYPE_LABELS = {
rss: "RSS",
atom: "Atom",
aggregated: "Aggregated",
reference: "Reference",
};
const LOCATION_SOURCE_LABELS = {
@@ -94,10 +116,7 @@ export function isNewsContentReady(item, locale = DEFAULT_LOCALE) {
const localized = getLocalization(item, locale);
const title = normalizeText(item?.display_title || localized.title);
const summary = normalizeText(item?.display_summary || localized.summary);
return (
(item?.enrichment_status === "success" || item?.enrichment_status === "content_only")
&& Boolean(title && summary)
);
return Boolean(title && summary);
}
export function getNewsRegionLabel(region, fallback = "") {
@@ -108,10 +127,39 @@ export function getNewsFeedLabel(feedName) {
return FEED_LABELS[feedName] || feedName || "聚合源";
}
export function getNewsCategoryLabel(category) {
return CATEGORY_LABELS[category] || category || CATEGORY_LABELS.other;
}
export function getNewsSourceTypeLabel(sourceType) {
const normalized = normalizeText(sourceType).toLowerCase();
return SOURCE_TYPE_LABELS[normalized] || sourceType || "RSS";
}
export function isRegionalMonitorFeed(feedName) {
return Object.prototype.hasOwnProperty.call(FEED_LABELS, feedName);
}
export function getNewsFetchChannelLabel(feedName, sourceType = "") {
const normalized = normalizeText(sourceType).toLowerCase();
if (isRegionalMonitorFeed(feedName) || normalized === "aggregated") return "区域监测";
if (normalized === "atom") return "单源 Atom";
if (normalized === "reference") return "配置保留";
return "单源 RSS";
}
export function getNewsLocationSourceLabel(source) {
return LOCATION_SOURCE_LABELS[source] || source || "位置来源";
}
export function getNewsEnrichmentStatusLabel(status) {
export function getNewsEnrichmentStatusLabel(statusOrItem) {
const item = statusOrItem && typeof statusOrItem === "object" ? statusOrItem : null;
const status = item ? item.enrichment_status : statusOrItem;
const language = String(item?.content_language || "").toLowerCase();
if (language.startsWith("zh") && status !== "success" && status !== "content_only") {
if (status === "queued" || status === "attempted") return "英文补译中";
if (status === "provider_error" || status === "parse_error" || status === "no_result") return "中文原文";
return "中文原文";
}
return ENRICHMENT_STATUS_LABELS[status] || status || "增强状态";
}

View File

@@ -2,11 +2,20 @@ import { showStatusMessage } from "./ui.js";
import {
getNewsDisplaySummary,
getNewsDisplayTitle,
getNewsCategoryLabel,
getNewsEnrichmentStatusLabel,
getNewsFeedLabel,
getNewsFetchChannelLabel,
getNewsRegionLabel,
getNewsSourceTypeLabel,
isNewsContentReady,
} from "./news-locale.js";
import {
canAttemptEarthRealtime,
getEarthRealtimeCooldownMs,
getEarthRealtimeUrl,
recordEarthRealtimeFailure,
recordEarthRealtimeOpen,
} from "./realtime.js";
// Desktop news has two surfaces:
// - a persistent top ticker
@@ -23,6 +32,9 @@ const NEWS_HUD_MIN_WIDTH_PX = 420;
const NEWS_HUD_MIN_HEIGHT_PX = 360;
const NEWS_HUD_RESIZE_MARGIN_PX = 12;
const NEWS_REALTIME_RECONNECT_MS = 5000;
const NEWS_SUMMARY_LIMIT = 12;
const NEWS_FULL_LIMIT = 50;
const NEWS_SOURCE_FILTER_STORAGE_KEY = "planet.earth.newsSourceFilters.v1";
let initialized = false;
let refreshPromise = null;
@@ -34,6 +46,54 @@ let selectedCruiseStoryId = null;
let morphTimer = null;
let newsRealtimeSocket = null;
let newsRealtimeReconnectTimer = null;
let activeNewsCategoryFilters = null;
let lastCategorySignature = "";
let activeNewsSourceFilters = loadNewsSourceFilters();
let lastSourceSignature = "";
let newsFullListMode = false;
let activeFilterPopover = null;
const NEWS_CATEGORY_ALIASES = {
politics: ["politics", "political", "policy", "政府", "政治", "政策", "政务"],
business: ["business", "economy", "economic", "commerce", "商业", "经济", "产业", "企业"],
ecommerce: ["ecommerce", "e-commerce", "online_retail", "retail_online", "电商", "电子商务", "网上零售", "跨境电商"],
finance: ["finance", "financial", "market", "stock", "金融", "财经", "市场", "证券"],
sports: ["sports", "sport", "体育"],
technology: ["technology", "tech", "science", "科技", "科学", "技术", "ai", "人工智能"],
military: ["military", "defense", "war", "军事", "防务", "战争"],
disaster: ["disaster", "emergency", "earthquake", "flood", "storm", "灾害", "灾难", "应急", "地震", "洪水"],
energy: ["energy", "oil", "gas", "power", "能源", "石油", "天然气", "电力"],
society: ["society", "social", "社会", "民生"],
culture: ["culture", "arts", "entertainment", "文化", "艺术", "娱乐"],
other: ["other", "general", "misc", "其他", "综合"],
};
function loadNewsSourceFilters() {
try {
const raw = window.localStorage?.getItem(NEWS_SOURCE_FILTER_STORAGE_KEY);
if (!raw) return null;
const parsed = JSON.parse(raw);
if (!Array.isArray(parsed?.enabled)) return null;
return parsed.enabled.map((id) => String(id || "").trim()).filter(Boolean);
} catch {
return null;
}
}
function persistNewsSourceFilters(enabledIds) {
try {
if (!Array.isArray(enabledIds)) {
window.localStorage?.removeItem(NEWS_SOURCE_FILTER_STORAGE_KEY);
return;
}
window.localStorage?.setItem(
NEWS_SOURCE_FILTER_STORAGE_KEY,
JSON.stringify({ enabled: enabledIds }),
);
} catch {
// Ignore localStorage failures; source filters are display-only preferences.
}
}
function getElements() {
const isMobile = document.body.classList.contains("layout-mode-mobile");
@@ -53,6 +113,9 @@ function getElements() {
tickerTrack: document.getElementById("news-ticker-track"),
hud: document.getElementById("news-hud-panel"),
hudCloseBtn: document.getElementById("news-hud-close"),
filterPopovers: document.querySelectorAll("[data-news-filter-popover]"),
filterToggles: document.querySelectorAll("[data-news-filter-toggle]"),
viewAllToggles: document.querySelectorAll("#news-view-all-toggle, #mobile-news-view-all-toggle"),
};
}
@@ -196,6 +259,11 @@ function closeNewsHud() {
setNewsHudOpen(false);
}
function isNewsHudOpen() {
const { hud } = getElements();
return hud instanceof HTMLElement && !hud.classList.contains("hud-panel-hidden");
}
function setupNewsHudResize() {
const { hud } = getElements();
const container = document.getElementById("container");
@@ -298,7 +366,221 @@ function escapeNewsHtml(value) {
}
function getDisplayableNewsItems(items) {
return Array.isArray(items) ? items.filter(isNewsContentReady) : [];
return Array.isArray(items)
? items.filter(isNewsContentReady)
: [];
}
function normalizeNewsSourceType(value) {
const normalized = String(value ?? "").trim().toLowerCase();
return normalized || "";
}
function getNewsSourceDescriptor(item, sourcesByName, sourcesById) {
const feedName = String(item?.feed_name || "").trim();
const sourceName = String(item?.source || feedName || "NEWS").trim();
const sourceId = String(item?.source_id || "").trim();
const sourceConfig = sourcesById.get(sourceId) || sourcesByName.get(feedName) || null;
const sourceType = normalizeNewsSourceType(item?.source_type || sourceConfig?.source_type)
|| (feedName.startsWith("Global Monitor /") ? "aggregated" : "rss");
const sourceTypeLabel = getNewsSourceTypeLabel(sourceType);
const channelLabel = getNewsFetchChannelLabel(feedName, sourceType);
const sourceGroupName = String(sourceConfig?.name || "").trim();
const originLabel = sourceGroupName
? `${sourceGroupName} · ${channelLabel}`
: feedName && feedName !== sourceName
? `${feedName} · ${sourceTypeLabel}`
: `${channelLabel} · ${sourceTypeLabel}`;
const tooltip = [
`媒体来源:${sourceName}`,
sourceGroupName ? `来源组:${sourceGroupName}` : "",
feedName ? `RSS 来源:${feedName}` : "",
`源类型:${sourceTypeLabel}`,
`抓取通道:${channelLabel}`,
].filter(Boolean).join("\n");
return {
sourceName,
feedName,
sourceType,
sourceTypeLabel,
channelLabel,
originLabel,
tooltip,
};
}
function getEnabledNewsCategoryKeys(filters = activeNewsCategoryFilters) {
if (!filters || typeof filters !== "object") return [];
return Object.entries(filters)
.filter(([, enabled]) => enabled !== false)
.map(([key]) => key)
.filter((key) => Object.prototype.hasOwnProperty.call(NEWS_CATEGORY_ALIASES, key))
.sort();
}
function getNewsCategorySignature(filters = activeNewsCategoryFilters) {
const enabled = getEnabledNewsCategoryKeys(filters);
const total = Object.keys(NEWS_CATEGORY_ALIASES).length;
if (enabled.length === 0) return "__none__";
if (enabled.length === total) return "";
return enabled.join(",");
}
function getAvailableSourceIds(nextPayload = payload) {
return (Array.isArray(nextPayload?.sources) ? nextPayload.sources : [])
.map((source) => String(source?.id || "").trim())
.filter(Boolean)
.sort();
}
function getEnabledNewsSourceIds(nextPayload = payload) {
const available = getAvailableSourceIds(nextPayload);
if (!available.length) return [];
if (!Array.isArray(activeNewsSourceFilters)) return available;
const allowed = new Set(activeNewsSourceFilters);
return available.filter((id) => allowed.has(id));
}
function getNewsSourceSignature(nextPayload = payload) {
const available = getAvailableSourceIds(nextPayload);
const enabled = getEnabledNewsSourceIds(nextPayload);
if (available.length > 0 && enabled.length === 0) return "__none__";
if (enabled.length === available.length) return "";
return enabled.join(",");
}
function getNewsLimit() {
return newsFullListMode ? NEWS_FULL_LIMIT : NEWS_SUMMARY_LIMIT;
}
function setNewsSourceFilters(enabledIds, { persist = true } = {}) {
const available = getAvailableSourceIds();
const next = Array.isArray(enabledIds)
? enabledIds.map((id) => String(id || "").trim()).filter((id) => available.includes(id))
: null;
activeNewsSourceFilters = next && next.length === available.length ? null : next;
if (persist) persistNewsSourceFilters(activeNewsSourceFilters);
}
function summarizeSelection(enabledCount, totalCount) {
if (totalCount <= 0) return "暂无";
if (enabledCount <= 0) return "未选";
if (enabledCount === totalCount) return "全部";
return `${enabledCount}`;
}
function syncFilterSummaries(nextPayload = payload) {
const categories = getEnabledNewsCategoryKeys();
const totalCategories = Object.keys(NEWS_CATEGORY_ALIASES).length;
const sources = getAvailableSourceIds(nextPayload);
const enabledSources = getEnabledNewsSourceIds(nextPayload);
document.querySelectorAll('[data-news-filter-summary="category"]').forEach((el) => {
el.textContent = summarizeSelection(categories.length, totalCategories);
});
document.querySelectorAll('[data-news-filter-summary="source"]').forEach((el) => {
el.textContent = summarizeSelection(enabledSources.length, sources.length);
});
document.querySelectorAll('[data-news-filter-summary="limit"]').forEach((el) => {
const total = Array.isArray(nextPayload?.items) ? nextPayload.items.length : 0;
el.textContent = newsFullListMode ? "全部" : `${Math.min(NEWS_SUMMARY_LIMIT, total || NEWS_SUMMARY_LIMIT)}`;
});
document.querySelectorAll("[data-news-view-mode-label]").forEach((el) => {
el.textContent = newsFullListMode ? "返回摘要" : "查看全部";
});
}
function closeNewsFilterPopover() {
activeFilterPopover = null;
document.querySelectorAll("[data-news-filter-popover]").forEach((popover) => {
if (popover instanceof HTMLElement) popover.hidden = true;
});
document.querySelectorAll("[data-news-filter-toggle]").forEach((toggle) => {
if (toggle instanceof HTMLElement) toggle.setAttribute("aria-expanded", "false");
});
}
function renderCategoryFilterChips() {
const enabled = new Set(getEnabledNewsCategoryKeys());
return Object.keys(NEWS_CATEGORY_ALIASES)
.map((key) => `
<button
class="news-filter-chip${enabled.has(key) ? " is-active" : ""}"
type="button"
data-news-category-toggle="${escapeNewsHtml(key)}"
aria-pressed="${enabled.has(key) ? "true" : "false"}"
>${escapeNewsHtml(getNewsCategoryLabel(key))}</button>
`)
.join("");
}
function renderSourceFilterChips() {
const sources = Array.isArray(payload?.sources) ? payload.sources : [];
const enabled = new Set(getEnabledNewsSourceIds());
if (!sources.length) return `<span class="news-filter-popover__hint">暂无可筛选来源。</span>`;
return sources
.map((source) => {
const id = String(source?.id || "").trim();
if (!id) return "";
const active = enabled.has(id);
return `
<button
class="news-filter-chip${active ? " is-active" : ""}"
type="button"
data-news-source-toggle="${escapeNewsHtml(id)}"
aria-pressed="${active ? "true" : "false"}"
>${escapeNewsHtml(source?.name || id)}</button>
`;
})
.join("");
}
function renderFilterPopover(kind) {
const title = kind === "source" ? "新闻来源" : "新闻类型";
const hint = kind === "source" ? "按大来源筛选,不影响后台抓取。" : "按新闻内容分类筛选。";
const content = kind === "source" ? renderSourceFilterChips() : renderCategoryFilterChips();
activeFilterPopover = kind;
document.querySelectorAll("[data-news-filter-popover]").forEach((popover) => {
if (!(popover instanceof HTMLElement)) return;
popover.hidden = false;
popover.innerHTML = `
<div class="news-filter-popover__header">
<div class="news-filter-popover__title">${escapeNewsHtml(title)}</div>
<div class="news-filter-popover__hint">${escapeNewsHtml(hint)}</div>
</div>
<div class="news-filter-chip-group">${content}</div>
`;
});
document.querySelectorAll("[data-news-filter-toggle]").forEach((toggle) => {
if (!(toggle instanceof HTMLElement)) return;
toggle.setAttribute("aria-expanded", toggle.dataset.newsFilterToggle === kind ? "true" : "false");
});
}
function toggleNewsSource(sourceId) {
const available = getAvailableSourceIds();
if (!available.includes(sourceId)) return;
const current = new Set(getEnabledNewsSourceIds());
if (current.has(sourceId)) current.delete(sourceId);
else current.add(sourceId);
setNewsSourceFilters([...current]);
syncFilterSummaries();
if (activeFilterPopover === "source") renderFilterPopover("source");
lastFetchAt = 0;
refreshNews(lastFocus?.lat, lastFocus?.lon, { silent: true }).catch(() => {});
}
function toggleNewsCategory(category, enabled) {
window.dispatchEvent(new CustomEvent("earth:set-news-category-enabled", {
detail: { category, enabled },
}));
}
function toggleNewsListMode() {
newsFullListMode = !newsFullListMode;
syncFilterSummaries();
lastFetchAt = 0;
refreshNews(lastFocus?.lat, lastFocus?.lon, { silent: true }).catch(() => {});
}
function renderTicker(nextPayload) {
@@ -320,7 +602,7 @@ function renderTicker(nextPayload) {
const visibleItems = getDisplayableNewsItems(items).slice(0, 6);
if (visibleItems.length === 0) {
tickerTrack.textContent = "正在等待中文新闻...";
tickerTrack.textContent = "当前新闻类型没有可显示新闻...";
tickerTrack.style.removeProperty("--news-ticker-duration");
return;
}
@@ -366,9 +648,20 @@ function renderPayload(nextPayload) {
const items = Array.isArray(nextPayload?.items) ? nextPayload.items : [];
const displayItems = getDisplayableNewsItems(items);
const sources = Array.isArray(nextPayload?.sources) ? nextPayload.sources : [];
const sourcesByName = new Map(
sources
.filter((source) => source && typeof source === "object" && source.name)
.map((source) => [String(source.name), source]),
);
const sourcesById = new Map(
sources
.filter((source) => source && typeof source === "object" && source.id)
.map((source) => [String(source.id), source]),
);
const focus = nextPayload?.focus || {};
renderTicker(nextPayload);
syncFilterSummaries(nextPayload);
if (!board || !status || !focusLabel || !focusCoords || !sourceCount || !regionChip) {
if (document.body.classList.contains("layout-mode-mobile")) {
@@ -400,8 +693,11 @@ function renderPayload(nextPayload) {
focusCoords.textContent = "跟随当前视角自动聚焦";
}
sourceCount.textContent = `${sources.length} 路聚合源`;
if (nextPayload?.stale) {
const enabledSourceCount = getEnabledNewsSourceIds(nextPayload).length;
sourceCount.textContent = `${enabledSourceCount || sources.length} / ${sources.length} 路来源`;
if (displayItems.length !== items.length) {
status.textContent = `展示 ${displayItems.length} / ${items.length} 条态势新闻`;
} else if (nextPayload?.stale) {
status.textContent = `当前显示最近一次可用新闻缓存,共 ${items.length}`;
} else {
status.textContent = nextPayload?.errors?.length
@@ -424,7 +720,7 @@ function renderPayload(nextPayload) {
empty.hidden = false;
empty.textContent = items.length === 0
? "当前未拉到可用新闻,请稍后刷新或切换视角区域。"
: "正在等待中文新闻内容完成处理。";
: "当前新闻类型没有可显示新闻。";
}
return;
}
@@ -440,22 +736,24 @@ function renderPayload(nextPayload) {
const summaryText = getNewsDisplaySummary(item);
const leadText = summaryText || title;
const regionLabel = item.display_region || getNewsRegionLabel(item.region);
const feedLabel = getNewsFeedLabel(item.feed_name);
const statusLabel = getNewsEnrichmentStatusLabel(item.enrichment_status);
const categoryLabel = getNewsCategoryLabel(item.category);
const statusLabel = getNewsEnrichmentStatusLabel(item);
const sourceDescriptor = getNewsSourceDescriptor(item, sourcesByName, sourcesById);
const summary = title && title !== leadText
? `<div class="news-story-summary">${escapeNewsHtml(title)}</div>`
: "";
return `
<a class="${cardClass}" data-news-id="${item.id}" href="${item.url}" target="_blank" rel="noreferrer noopener">
<a class="${cardClass}" data-news-id="${item.id}" href="${item.url}" target="_blank" rel="noreferrer noopener" title="${escapeNewsHtml(sourceDescriptor.tooltip)}">
<div class="news-story-meta">
<span class="news-story-source">${escapeNewsHtml(item.source || "NEWS")}</span>
<span class="news-story-source">${escapeNewsHtml(sourceDescriptor.sourceName)}</span>
<span class="news-story-time">${formatRelativeTime(item.published_at)}</span>
</div>
<div class="news-story-origin">${escapeNewsHtml(sourceDescriptor.originLabel)}</div>
<div class="news-story-title">${escapeNewsHtml(leadText)}</div>
${summary}
<div class="news-story-tags">
<span class="news-story-tag">${escapeNewsHtml(categoryLabel)}</span>
<span class="news-story-tag">${escapeNewsHtml(regionLabel)}</span>
<span class="news-story-tag">${escapeNewsHtml(feedLabel)}</span>
<span class="news-story-tag">${escapeNewsHtml(statusLabel)}</span>
</div>
</a>
@@ -472,11 +770,6 @@ function renderPayload(nextPayload) {
}));
}
function getNewsRealtimeUrl() {
const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
return `${protocol}//${window.location.host}/ws`;
}
function clearNewsRealtimeReconnectTimer() {
if (!newsRealtimeReconnectTimer) return;
window.clearTimeout(newsRealtimeReconnectTimer);
@@ -485,10 +778,11 @@ function clearNewsRealtimeReconnectTimer() {
function scheduleNewsRealtimeReconnect() {
if (newsRealtimeReconnectTimer) return;
const delay = Math.max(NEWS_REALTIME_RECONNECT_MS, getEarthRealtimeCooldownMs());
newsRealtimeReconnectTimer = window.setTimeout(() => {
newsRealtimeReconnectTimer = null;
connectNewsRealtime();
}, NEWS_REALTIME_RECONNECT_MS);
}, delay);
}
function applyNewsRealtimePatch(updatePayload) {
@@ -517,10 +811,15 @@ function applyNewsRealtimePatch(updatePayload) {
}
function connectNewsRealtime() {
if (newsRealtimeSocket || typeof WebSocket === "undefined") return;
const socket = new WebSocket(getNewsRealtimeUrl());
if (newsRealtimeSocket || !canAttemptEarthRealtime()) {
scheduleNewsRealtimeReconnect();
return;
}
const socket = new WebSocket(getEarthRealtimeUrl());
newsRealtimeSocket = socket;
socket.onopen = () => {
socket.__planetOpened = true;
recordEarthRealtimeOpen();
clearNewsRealtimeReconnectTimer();
socket.send(JSON.stringify({
type: "subscribe",
@@ -547,6 +846,9 @@ function connectNewsRealtime() {
if (newsRealtimeSocket === socket) {
newsRealtimeSocket = null;
}
if (!socket.__planetOpened) {
recordEarthRealtimeFailure();
}
scheduleNewsRealtimeReconnect();
};
socket.onerror = () => {
@@ -555,9 +857,34 @@ function connectNewsRealtime() {
}
async function fetchNews(lat, lon) {
const categorySignature = getNewsCategorySignature();
const sourceSignature = getNewsSourceSignature();
if (categorySignature === "__none__" || sourceSignature === "__none__") {
return {
...(payload || {}),
generated_at: new Date().toISOString(),
focus: payload?.focus || { lat, lon, region: "global", label: "全球焦点", display_region: "全球" },
sources: payload?.sources || [],
filters: {
region: payload?.focus?.region || "global",
categories: categorySignature === "__none__" ? [] : getEnabledNewsCategoryKeys(),
sources: sourceSignature === "__none__" ? [] : getEnabledNewsSourceIds(),
limit: getNewsLimit(),
locale: "zh-CN",
},
items: [],
cruise_items: [],
errors: [],
stale: false,
};
}
const url = new URL(EARTH_NEWS_API, window.location.origin);
if (typeof lat === "number") url.searchParams.set("lat", lat.toFixed(4));
if (typeof lon === "number") url.searchParams.set("lon", lon.toFixed(4));
if (categorySignature) url.searchParams.set("categories", categorySignature);
if (sourceSignature) url.searchParams.set("sources", sourceSignature);
url.searchParams.set("limit", String(getNewsLimit()));
url.searchParams.set("locale", "zh-CN");
const controller = new AbortController();
const timeoutId = window.setTimeout(() => controller.abort(), REQUEST_TIMEOUT_MS);
@@ -585,6 +912,8 @@ async function refreshNews(lat, lon, { silent = false } = {}) {
.then((nextPayload) => {
renderPayload(nextPayload);
lastFetchAt = Date.now();
lastCategorySignature = getNewsCategorySignature();
lastSourceSignature = getNewsSourceSignature(nextPayload);
if (Array.isArray(nextPayload?.items) && nextPayload.items.length === 0) {
const { status } = getElements();
if (status) {
@@ -618,6 +947,8 @@ export async function refreshEarthNews({ silent = true } = {}) {
function shouldRefreshForFocus(lat, lon, region) {
const now = Date.now();
if (getNewsCategorySignature() !== lastCategorySignature) return true;
if (getNewsSourceSignature() !== lastSourceSignature) return true;
if (!lastFocus) return true;
if (region !== lastFocus.region && now - lastRegionSwitchAt > MIN_REGION_SWITCH_INTERVAL_MS) {
lastRegionSwitchAt = now;
@@ -650,6 +981,7 @@ export function updateNewsViewFocus(coords) {
if (!coords || typeof coords.lat !== "number" || typeof coords.lon !== "number") return;
const region = inferRegion(coords.lat, coords.lon);
const previousRegion = lastFocus?.region || null;
const nextFocus = {
lat: coords.lat,
lon: coords.lon,
@@ -658,6 +990,9 @@ export function updateNewsViewFocus(coords) {
};
const shouldRefresh = shouldRefreshForFocus(coords.lat, coords.lon, region);
if (!shouldRefresh && previousRegion && region !== previousRegion) {
return;
}
lastFocus = nextFocus;
if (shouldRefresh) {
refreshNews(coords.lat, coords.lon, { silent: true }).catch(() => {});
@@ -743,6 +1078,57 @@ export function initNewsPanel() {
openNewsHud();
});
hudCloseBtn?.addEventListener("click", closeNewsHud);
document.addEventListener("keydown", (event) => {
if (event.key !== "Escape" || !isNewsHudOpen()) return;
event.preventDefault();
if (activeFilterPopover) {
closeNewsFilterPopover();
return;
}
closeNewsHud();
});
document.addEventListener("click", (event) => {
const target = event.target instanceof Element ? event.target : null;
if (!target) return;
const filterToggle = target.closest("[data-news-filter-toggle]");
if (filterToggle instanceof HTMLElement) {
const kind = filterToggle.dataset.newsFilterToggle || "";
if (activeFilterPopover === kind) closeNewsFilterPopover();
else renderFilterPopover(kind);
return;
}
const categoryToggle = target.closest("[data-news-category-toggle]");
if (categoryToggle instanceof HTMLElement && categoryToggle.closest("[data-news-filter-popover]")) {
const category = categoryToggle.dataset.newsCategoryToggle || "";
const active = categoryToggle.classList.contains("is-active");
toggleNewsCategory(category, !active);
return;
}
const sourceToggle = target.closest("[data-news-source-toggle]");
if (sourceToggle instanceof HTMLElement) {
toggleNewsSource(sourceToggle.dataset.newsSourceToggle || "");
return;
}
const viewAllToggle = target.closest("#news-view-all-toggle, #mobile-news-view-all-toggle");
if (viewAllToggle instanceof HTMLElement) {
toggleNewsListMode();
return;
}
if (activeFilterPopover && !target.closest("[data-news-filter-popover]")) {
closeNewsFilterPopover();
}
});
window.addEventListener("earth:news-category-filters-change", (event) => {
activeNewsCategoryFilters = event.detail?.categories || null;
syncFilterSummaries();
if (activeFilterPopover === "category") renderFilterPopover("category");
lastFetchAt = 0;
refreshNews(lastFocus?.lat, lastFocus?.lon, { silent: true }).catch(() => {});
});
setupNewsHudResize();
connectNewsRealtime();

View File

@@ -0,0 +1,40 @@
const WS_BACKOFF_BASE_MS = 5_000;
const WS_BACKOFF_MAX_MS = 60_000;
const wsBackoffState = {
failures: 0,
nextAttemptAt: 0,
};
export function getEarthRealtimeUrl() {
const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
const host = window.location.hostname;
const port = window.location.port;
if ((host === "localhost" || host === "127.0.0.1") && port === "3000") {
return `${protocol}//${host}:8000/ws`;
}
return `${protocol}//${window.location.host}/ws`;
}
export function getEarthRealtimeCooldownMs() {
return Math.max(0, wsBackoffState.nextAttemptAt - Date.now());
}
export function canAttemptEarthRealtime() {
return typeof WebSocket !== "undefined" && getEarthRealtimeCooldownMs() <= 0;
}
export function recordEarthRealtimeOpen() {
wsBackoffState.failures = 0;
wsBackoffState.nextAttemptAt = 0;
}
export function recordEarthRealtimeFailure() {
wsBackoffState.failures += 1;
const delay = Math.min(
WS_BACKOFF_MAX_MS,
WS_BACKOFF_BASE_MS * 2 ** Math.min(wsBackoffState.failures - 1, 5),
);
wsBackoffState.nextAttemptAt = Date.now() + delay;
return delay;
}

View File

@@ -39,12 +39,85 @@ const TV_PANEL_MIN_HEIGHT_PX = 340;
const HLS_MAX_RECOVERY_ATTEMPTS = 3;
const HLS_RETRY_CONFIG = {
maxNumRetry: 4,
retryDelayMs: 1500,
maxRetryDelayMs: 8000,
maxNumRetry: 1,
retryDelayMs: 1200,
maxRetryDelayMs: 3000,
backoff: "exponential",
};
const HLS_NON_PLAYBACK_ERROR_DETAILS = new Set([
"subtitleTrackLoadError",
"subtitleTrackParsingError",
"subtitleTrackSwitchError",
"audioTrackLoadError",
"audioTrackSwitchError",
"keyLoadError",
]);
const HLS_SOURCE_FAILURE_DETAILS = new Set([
"manifestLoadError",
"manifestLoadTimeOut",
"levelLoadError",
"levelLoadTimeOut",
"fragLoadError",
"fragLoadTimeOut",
]);
function isVideoActuallyPlaying(video) {
return (
video instanceof HTMLVideoElement
&& !video.paused
&& !video.ended
&& video.readyState >= HTMLMediaElement.HAVE_CURRENT_DATA
);
}
function getHlsErrorMessage(data) {
const details = data?.details || "";
const response = data?.response || {};
const status = response?.code || response?.status;
const url = data?.url || response?.url || "";
if (details === "manifestLoadError" || details === "manifestLoadTimeOut") {
return status
? `HLS 主播放列表加载失败HTTP ${status}`
: "HLS 主播放列表加载失败";
}
if (details === "levelLoadError" || details === "levelLoadTimeOut") {
return status
? `HLS 清晰度播放列表加载失败HTTP ${status}`
: "HLS 清晰度播放列表加载失败";
}
if (details === "fragLoadError" || details === "fragLoadTimeOut") {
return status
? `HLS 分片加载失败HTTP ${status}`
: "HLS 分片加载失败";
}
if (details === "bufferStalledError") return "直播流缓冲停滞,正在等待数据";
if (details === "bufferAppendError") return "直播流缓冲写入失败";
if (details === "manifestParsingError") return "HLS 播放列表格式无法解析";
if (details === "fragParsingError") return "HLS 分片格式无法解析";
if (url) return `HLS 资源加载失败:${url}`;
return "HLS 播放流不可用";
}
function logHlsDiagnostic(data, source) {
const payload = {
source_id: source?.id,
source_name: source?.name,
type: data?.type,
details: data?.details,
fatal: Boolean(data?.fatal),
url: data?.url || data?.response?.url,
status: data?.response?.code || data?.response?.status,
};
if (data?.fatal) {
console.error("HLS 播放失败:", payload, data);
} else {
console.debug("HLS 非致命事件:", payload, data);
}
}
function getElements() {
const isMobile = document.body.classList.contains("layout-mode-mobile");
return {
@@ -398,6 +471,8 @@ function tryStartPlayback(video) {
function attachVideoSource(video, source) {
const sourceUrl = getVideoUrl(source);
if (!(video instanceof HTMLVideoElement) || !sourceUrl) return;
let hlsNetworkErrorCount = 0;
let hlsHardFailureHandled = false;
destroyHlsPlayer();
video.autoplay = true;
@@ -414,16 +489,18 @@ function attachVideoSource(video, source) {
if (Hls.isSupported()) {
hlsPlayer = new Hls({
enableWorker: true,
enableWebVTT: false,
subtitleDisplay: false,
lowLatencyMode: false,
manifestLoadingTimeOut: 20000,
levelLoadingTimeOut: 20000,
fragLoadingTimeOut: 25000,
fragLoadingMaxRetry: 3,
fragLoadingRetryDelay: 1500,
levelLoadingMaxRetry: 3,
levelLoadingRetryDelay: 1500,
manifestLoadingMaxRetry: 2,
manifestLoadingRetryDelay: 1500,
fragLoadingMaxRetry: 1,
fragLoadingRetryDelay: 1200,
levelLoadingMaxRetry: 1,
levelLoadingRetryDelay: 1200,
manifestLoadingMaxRetry: 1,
manifestLoadingRetryDelay: 1200,
liveSyncDurationCount: 4,
liveMaxLatencyDurationCount: 10,
manifestLoadPolicy: {
@@ -432,11 +509,11 @@ function attachVideoSource(video, source) {
maxLoadTimeMs: 20000,
timeoutRetry: {
...HLS_RETRY_CONFIG,
maxNumRetry: 2,
maxNumRetry: 1,
},
errorRetry: {
...HLS_RETRY_CONFIG,
maxNumRetry: 2,
maxNumRetry: 1,
},
},
},
@@ -465,14 +542,35 @@ function attachVideoSource(video, source) {
tryStartPlayback(video);
});
hlsPlayer.on(Hls.Events.ERROR, (_event, data) => {
console.error("HLS 播放失败:", data);
logHlsDiagnostic(data, source);
const status = Number(data?.response?.code || data?.response?.status || 0);
const sourceFailure = HLS_SOURCE_FAILURE_DETAILS.has(data?.details);
if (sourceFailure && (status >= 500 || data?.details === "manifestLoadError")) {
hlsNetworkErrorCount += 1;
}
if (!hlsHardFailureHandled && !isVideoActuallyPlaying(video) && sourceFailure && hlsNetworkErrorCount >= 2) {
hlsHardFailureHandled = true;
const reasonMessage = getHlsErrorMessage(data);
hlsPlayer?.stopLoad();
if (!showEmbeddedFallback(source, reasonMessage) && !tryFallbackSource()) {
setPanelMessage(reasonMessage);
}
return;
}
if (
!data?.fatal
&& HLS_NON_PLAYBACK_ERROR_DETAILS.has(data?.details)
&& isVideoActuallyPlaying(video)
) {
return;
}
if (!data?.fatal) {
if (data?.type === Hls.ErrorTypes.NETWORK_ERROR) {
setPanelMessage("直播流网络波动,正在重试...");
setPanelMessage(isVideoActuallyPlaying(video) ? TV_STATUS_MESSAGE.videoReady : "直播流网络波动,正在重试...");
return;
}
if (data?.type === Hls.ErrorTypes.MEDIA_ERROR) {
setPanelMessage("直播流正在恢复...");
setPanelMessage(isVideoActuallyPlaying(video) ? TV_STATUS_MESSAGE.videoReady : "直播流正在恢复...");
return;
}
}
@@ -491,8 +589,9 @@ function attachVideoSource(video, source) {
}
}
if (!showEmbeddedFallback(source) && !tryFallbackSource()) {
setPanelMessage(TV_STATUS_MESSAGE.videoError);
const reasonMessage = getHlsErrorMessage(data);
if (!showEmbeddedFallback(source, reasonMessage) && !tryFallbackSource()) {
setPanelMessage(reasonMessage);
}
});
return;

View File

@@ -2,6 +2,13 @@ import * as THREE from "three";
import { CONFIG, PATHS, VESSEL_CONFIG } from "./constants.js";
import { createInteractableLayer } from "./interactable.js";
import {
canAttemptEarthRealtime,
getEarthRealtimeCooldownMs,
getEarthRealtimeUrl,
recordEarthRealtimeFailure,
recordEarthRealtimeOpen,
} from "./realtime.js";
import { latLonToVector3 } from "./utils.js";
let showVessels = false;
@@ -105,12 +112,6 @@ function applyVesselDeltas(earth, vessels = []) {
return changed;
}
function getVesselStreamUrl() {
if (typeof window === "undefined") return "ws://localhost:8000/ws";
const protocol = window.location.protocol === "https:" ? "wss:" : "ws:";
return `${protocol}//${window.location.host}/ws`;
}
function normalizeVesselType(value, code) {
const type = String(value || "").trim().toLowerCase();
const numericCode = Number(code);
@@ -257,6 +258,11 @@ const vesselIconLayer = createInteractableLayer({
vessel_kind: item.type,
baseScale: VESSEL_CONFIG.marker.baseScale,
}),
cluster: {
strategy: "dynamic-screen",
enabled: true,
maxMarkersPerDot: 10,
},
});
const DEFAULT_VESSEL_VIEWPORT = {
@@ -381,13 +387,19 @@ function normalizeVesselViewportOptions(options = {}) {
}
export function startVesselRealtime(earth, { onUpdate, bbox, zoom, limit } = {}) {
if (vesselStreamSocket || typeof WebSocket === "undefined") return;
if (vesselStreamSocket || vesselStreamReconnectTimer || typeof WebSocket === "undefined") return;
const subscriptionOptions = normalizeVesselViewportOptions({ bbox, zoom, limit });
const connect = () => {
if (!showVessels || vesselStreamSocket) return;
const socket = new WebSocket(getVesselStreamUrl());
if (!canAttemptEarthRealtime()) {
vesselStreamReconnectTimer = window.setTimeout(connect, Math.max(3000, getEarthRealtimeCooldownMs()));
return;
}
const socket = new WebSocket(getEarthRealtimeUrl());
vesselStreamSocket = socket;
socket.onopen = () => {
socket.__planetOpened = true;
recordEarthRealtimeOpen();
vesselRealtimeStats = {
...vesselRealtimeStats,
connected: true,
@@ -452,7 +464,10 @@ export function startVesselRealtime(earth, { onUpdate, bbox, zoom, limit } = {})
};
onUpdate?.({ totalCount: getVesselCount(), stream: getVesselRealtimeStats() });
if (showVessels) {
vesselStreamReconnectTimer = window.setTimeout(connect, 3000);
if (!socket.__planetOpened) {
recordEarthRealtimeFailure();
}
vesselStreamReconnectTimer = window.setTimeout(connect, Math.max(3000, getEarthRealtimeCooldownMs()));
}
};
socket.onerror = () => {

View File

@@ -4,6 +4,7 @@ import { Routes, Route, Navigate, useLocation } from 'react-router-dom'
import { useAuthStore } from './stores/auth'
import Login from './pages/Login/Login'
import { AdminErrorBoundary } from './admin/components/AdminErrorBoundary'
const Register = lazy(() => import('./pages/Register/Register'))
const VerifyEmail = lazy(() => import('./pages/VerifyEmail/VerifyEmail'))
@@ -51,7 +52,7 @@ function App() {
<Route path={DOCS_ROUTE} element={<Docs />} />
<Route path={DOCS_ROUTE_PATTERN} element={<Docs />} />
<Route path="/playground" element={<Navigate to="/ai?section=playground" replace />} />
<Route path="/*" element={<AdminRoutes />} />
<Route path="/*" element={<AdminErrorBoundary><AdminRoutes /></AdminErrorBoundary>} />
</Routes>
</Suspense>
)

View File

@@ -0,0 +1,39 @@
import { Component, type ErrorInfo, type ReactNode } from 'react'
import { reportAdminRuntimeLog } from '../runtimeLogs'
type AdminErrorBoundaryProps = {
children: ReactNode
}
type AdminErrorBoundaryState = {
hasError: boolean
}
export class AdminErrorBoundary extends Component<AdminErrorBoundaryProps, AdminErrorBoundaryState> {
state: AdminErrorBoundaryState = { hasError: false }
static getDerivedStateFromError() {
return { hasError: true }
}
componentDidCatch(error: Error, errorInfo: ErrorInfo) {
void reportAdminRuntimeLog({
level: 'error',
category: 'react-error-boundary',
module: 'admin',
message: error.message || '控制台渲染错误',
detail: `${error.stack || error.message}\n${errorInfo.componentStack || ''}`,
})
}
render() {
if (this.state.hasError) {
return (
<div className="app-route-loading">
<div className="app-route-loading__message"></div>
</div>
)
}
return this.props.children
}
}

View File

@@ -15,7 +15,7 @@ import Scrollbar from '../../../components/Scrollbar/Scrollbar'
import SegmentedControl from '../../../components/SegmentedControl/SegmentedControl'
import { useAuthStore } from '../../../stores/auth'
import { useAdminTheme, type AdminThemeMode } from '../../design/theme'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
import { adminRouteGroups, getVisibleAdminRoutes } from '../../routes/manifest'
import { useAdminSearch } from '../../search/AdminSearchContext'
import { Button } from '../ui/button'

View File

@@ -1,5 +1,5 @@
import { type HTMLAttributes } from 'react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
type BadgeTone = 'default' | 'blue' | 'green' | 'amber' | 'red' | 'purple' | 'cyan' | 'slate'

View File

@@ -1,5 +1,5 @@
import { type HTMLAttributes } from 'react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
export function Card({ className, ...props }: HTMLAttributes<HTMLDivElement>) {
return <div className={cn('an-card', className)} {...props} />

View File

@@ -1,5 +1,5 @@
import { forwardRef, type InputHTMLAttributes, type TextareaHTMLAttributes } from 'react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
export const Input = forwardRef<HTMLInputElement, InputHTMLAttributes<HTMLInputElement>>(
({ className, ...props }, ref) => <input ref={ref} className={cn('an-input', className)} {...props} />,

View File

@@ -1,6 +1,6 @@
import * as SelectPrimitive from '@radix-ui/react-select'
import { Check, ChevronDown } from 'lucide-react'
import { cn } from '../../lib/utils'
import { cn } from '../../utils'
export interface SelectOption {
value: string
@@ -33,7 +33,7 @@ export function Select({ value, onValueChange, options, placeholder, disabled, c
{options.map((option) => (
<SelectPrimitive.Item key={option.value} value={option.value} className="an-select__item">
<SelectPrimitive.ItemText>{option.label}</SelectPrimitive.ItemText>
<SelectPrimitive.ItemIndicator>
<SelectPrimitive.ItemIndicator className="an-select__indicator">
<Check size={14} />
</SelectPrimitive.ItemIndicator>
</SelectPrimitive.Item>

View File

@@ -23,7 +23,7 @@ import { Card, CardContent, CardDescription, CardHeader, CardTitle } from '../co
import { Dialog } from '../components/ui/dialog'
import { Select } from '../components/ui/select'
import { useToast } from '../components/ui/toast'
import { formatNumber } from '../lib/utils'
import { formatNumber } from '../utils'
interface Stats {
total_datasources: number

View File

@@ -1,7 +1,9 @@
import axios from 'axios'
import { Copy, RefreshCw, Search, X } from 'lucide-react'
import { useEffect, useMemo, useState } from 'react'
import { Copy, Pause, Play, RefreshCw, Search, X } from 'lucide-react'
import { useEffect, useMemo, useRef, useState } from 'react'
import { useLocation, useNavigate } from 'react-router-dom'
import Scrollbar from '../../components/Scrollbar/Scrollbar'
import { useWebSocket } from '../../hooks/useWebSocket'
import { useAuthStore } from '../../stores/auth'
import { AdminLayout } from '../components/layout/AdminLayout'
import { Badge } from '../components/ui/badge'
@@ -20,6 +22,13 @@ const LOG_LEVEL_OPTIONS = [
{ value: 'info', label: '信息' },
{ value: 'debug', label: '调试' },
]
const LOG_TAIL_CHANNEL = 'logs_tail'
const LOG_VIEW_OPTIONS = [
{ value: 'grouped', label: '重复统计' },
{ value: 'raw', label: '原始日志' },
{ value: 'audit', label: '审计日志' },
]
type LogViewMode = 'grouped' | 'raw' | 'audit'
interface LogSourceSummary {
source_id: string
@@ -48,30 +57,107 @@ interface LogSnapshot {
lines: string[]
}
interface ObservabilityGroup {
fingerprint: string
source: string
service?: string | null
module?: string | null
category?: string | null
event?: string | null
level: string
message: string
detail?: string | null
affected_sources: string[]
count: number
first_seen_at?: string | null
last_seen_at?: string | null
}
interface ObservabilityEvent {
id: number
source: string
service?: string | null
module?: string | null
category?: string | null
event?: string | null
level: string
message: string
fingerprint: string
occurred_at?: string | null
request_id?: string | null
trace_id?: string | null
task_id?: string | null
source_id?: string | null
provider?: string | null
context: Record<string, unknown>
occurrence_count: number
}
interface ObservabilityGroupsResponse {
groups: ObservabilityGroup[]
line_count: number
}
interface ObservabilityGroupEventsResponse {
fingerprint: string
group: ObservabilityGroup
events: ObservabilityEvent[]
line_count: number
}
type StoredLogFilters = {
selectedSource?: string
mode?: LogViewMode
lineLimit?: number
level?: string
startDate?: string
endDate?: string
searchQuery?: string
follow?: boolean
}
function readStoredFilters() {
if (typeof window === 'undefined') return null
try {
const rawValue = window.localStorage.getItem(LOG_FILTER_STORAGE_KEY)
return rawValue ? JSON.parse(rawValue) as {
selectedSource?: string
lineLimit?: number
level?: string
startDate?: string
endDate?: string
searchQuery?: string
} : null
return rawValue ? JSON.parse(rawValue) as StoredLogFilters : null
} catch {
return null
}
}
function readUrlFilters(search: string): StoredLogFilters {
const params = new URLSearchParams(search)
const lineLimit = Number(params.get('limit') || '')
const rawMode = params.get('mode')
const mode = rawMode === 'raw' || rawMode === 'audit' || rawMode === 'grouped' ? rawMode : undefined
return {
selectedSource: params.get('source') || undefined,
mode,
lineLimit: Number.isFinite(lineLimit) && lineLimit > 0 ? lineLimit : undefined,
level: params.get('level') || undefined,
startDate: params.get('start_date') || undefined,
endDate: params.get('end_date') || undefined,
searchQuery: params.get('search') || undefined,
follow: params.get('follow') === '1',
}
}
function statusTone(status: string) {
if (status === 'ok') return 'success'
if (status === 'missing' || status === 'empty') return 'warning'
if (status === 'missing') return 'warning'
if (status === 'empty') return 'neutral'
if (status.includes('unavailable')) return 'danger'
return 'neutral'
}
function levelTone(level: string) {
if (level === 'error') return 'danger'
if (level === 'warning') return 'warning'
if (level === 'info') return 'success'
return 'neutral'
}
function statusLabel(status: string) {
if (status === 'ok') return '可用'
if (status === 'missing') return '暂无日志'
@@ -87,29 +173,98 @@ function getErrorMessage(error: unknown, fallback: string) {
return typeof detail === 'string' ? detail : fallback
}
function formatDateTime(value?: string | null) {
if (!value) return '-'
try {
return new Date(value).toLocaleString()
} catch {
return value
}
}
function stringifyContext(context: Record<string, unknown>) {
try {
return JSON.stringify(context || {}, null, 2)
} catch {
return '{}'
}
}
export default function Logs() {
const storedFilters = readStoredFilters()
const location = useLocation()
const navigate = useNavigate()
const urlFilters = readUrlFilters(location.search)
const { user } = useAuthStore()
const { toast } = useToast()
const isSuperAdmin = user?.role === 'super_admin'
const [sources, setSources] = useState<LogSourceSummary[]>([])
const [selectedSource, setSelectedSource] = useState(storedFilters?.selectedSource || 'backend')
const [lineLimit, setLineLimit] = useState(storedFilters?.lineLimit || 200)
const [level, setLevel] = useState(storedFilters?.level || 'all')
const [startDate, setStartDate] = useState(storedFilters?.startDate || '')
const [endDate, setEndDate] = useState(storedFilters?.endDate || '')
const [searchQuery, setSearchQuery] = useState(storedFilters?.searchQuery || '')
const [submittedSearch, setSubmittedSearch] = useState(storedFilters?.searchQuery || '')
const [mode, setMode] = useState<LogViewMode>(urlFilters.mode || storedFilters?.mode || 'grouped')
const [selectedSource, setSelectedSource] = useState(urlFilters.selectedSource || storedFilters?.selectedSource || 'backend')
const [lineLimit, setLineLimit] = useState(urlFilters.lineLimit || storedFilters?.lineLimit || 200)
const [level, setLevel] = useState(urlFilters.level || storedFilters?.level || 'all')
const [startDate, setStartDate] = useState(urlFilters.startDate || storedFilters?.startDate || '')
const [endDate, setEndDate] = useState(urlFilters.endDate || storedFilters?.endDate || '')
const [searchQuery, setSearchQuery] = useState(urlFilters.searchQuery ?? storedFilters?.searchQuery ?? '')
const [submittedSearch, setSubmittedSearch] = useState(urlFilters.searchQuery ?? storedFilters?.searchQuery ?? '')
const [followEnabled, setFollowEnabled] = useState(Boolean(urlFilters.follow || storedFilters?.follow))
const [snapshot, setSnapshot] = useState<LogSnapshot | null>(null)
const [groups, setGroups] = useState<ObservabilityGroup[]>([])
const [selectedGroup, setSelectedGroup] = useState<ObservabilityGroup | null>(null)
const [groupEvents, setGroupEvents] = useState<ObservabilityEvent[]>([])
const [sourcesLoading, setSourcesLoading] = useState(false)
const [logLoading, setLogLoading] = useState(false)
const [errorMessage, setErrorMessage] = useState<string | null>(null)
const scrollContainerRef = useRef<HTMLDivElement | null>(null)
const shouldStickToBottomRef = useRef(true)
const selectedSourceInfo = useMemo(
() => sources.find((source) => source.source_id === selectedSource) || null,
[selectedSource, sources],
)
const hasActiveFilters = lineLimit !== 200 || level !== 'all' || Boolean(startDate || endDate || submittedSearch.trim() || searchQuery.trim())
const hasActiveFilters = lineLimit !== 200 || level !== 'all' || Boolean(startDate || endDate || submittedSearch.trim() || searchQuery.trim() || followEnabled)
const { connected: followConnected, sendMessage } = useWebSocket({
autoConnect: followEnabled && isSuperAdmin,
onMessage: (message) => {
if (message.channel !== LOG_TAIL_CHANNEL || !message.payload) return
const payload = message.payload as { mode?: string; source_id?: string; lines?: unknown; line_count?: number; status?: string }
if (payload.source_id !== selectedSource) return
const incomingLines = Array.isArray(payload.lines) ? payload.lines.filter((line): line is string => typeof line === 'string') : []
setSnapshot((current) => {
const base = current || {
source_id: selectedSource,
name: selectedSourceInfo?.name || selectedSource,
kind: selectedSourceInfo?.kind || 'unknown',
location: selectedSourceInfo?.location || '',
description: selectedSourceInfo?.description || '',
category: selectedSourceInfo?.category || '',
status: payload.status || 'ok',
level,
selected_levels: level === 'all' ? [] : [level],
search_query: submittedSearch,
available_levels: ['all', 'error', 'warning', 'info', 'debug'],
line_limit: lineLimit,
line_count: 0,
lines: [],
}
const nextLines = payload.mode === 'snapshot'
? incomingLines
: [...(base.lines || []), ...incomingLines].slice(-lineLimit)
return {
...base,
status: payload.status || base.status,
line_limit: lineLimit,
line_count: nextLines.length,
lines: nextLines,
}
})
setErrorMessage(null)
},
onError: () => {
setErrorMessage('日志跟随连接失败,可暂停后使用手动刷新。')
},
})
const fetchSources = async () => {
if (!isSuperAdmin) return
@@ -132,9 +287,10 @@ export default function Logs() {
const fetchSnapshot = async () => {
if (!isSuperAdmin || !selectedSource) return
const sourceForMode = mode === 'audit' ? 'audit-db' : selectedSource
setLogLoading(true)
try {
const response = await axios.get<LogSnapshot>(`/api/v1/system/logs/${encodeURIComponent(selectedSource)}`, {
const response = await axios.get<LogSnapshot>(`/api/v1/system/logs/${encodeURIComponent(sourceForMode)}`, {
params: {
limit: lineLimit,
level,
@@ -154,13 +310,55 @@ export default function Logs() {
}
}
const fetchGroups = async () => {
if (!isSuperAdmin) return
setLogLoading(true)
try {
const response = await axios.get<ObservabilityGroupsResponse>('/api/v1/system/logs/observability/groups', {
params: {
limit: lineLimit,
level,
levels: level === 'all' ? undefined : level,
start_date: startDate || undefined,
end_date: endDate || undefined,
search: submittedSearch.trim() || undefined,
},
})
setGroups(response.data.groups || [])
setErrorMessage(null)
} catch (error) {
setGroups([])
setErrorMessage(getErrorMessage(error, '加载重复日志统计失败'))
} finally {
setLogLoading(false)
}
}
const fetchGroupEvents = async (group: ObservabilityGroup) => {
setSelectedGroup(group)
try {
const response = await axios.get<ObservabilityGroupEventsResponse>(
`/api/v1/system/logs/observability/groups/${encodeURIComponent(group.fingerprint)}/events`,
{ params: { limit: lineLimit } },
)
setGroupEvents(response.data.events || [])
} catch (error) {
setGroupEvents([])
setErrorMessage(getErrorMessage(error, '加载重复日志详情失败'))
}
}
useEffect(() => {
void fetchSources()
}, [isSuperAdmin])
useEffect(() => {
void fetchSnapshot()
}, [isSuperAdmin, selectedSource, lineLimit, level, startDate, endDate, submittedSearch])
if (mode === 'grouped') {
void fetchGroups()
} else if (!followEnabled) {
void fetchSnapshot()
}
}, [isSuperAdmin, selectedSource, lineLimit, level, startDate, endDate, submittedSearch, followEnabled, mode])
useEffect(() => {
const timer = window.setTimeout(() => {
@@ -173,13 +371,80 @@ export default function Logs() {
if (typeof window === 'undefined') return
window.localStorage.setItem(LOG_FILTER_STORAGE_KEY, JSON.stringify({
selectedSource,
mode,
lineLimit,
level,
startDate,
endDate,
searchQuery: submittedSearch,
follow: followEnabled,
}))
}, [endDate, level, lineLimit, selectedSource, startDate, submittedSearch])
}, [endDate, followEnabled, level, lineLimit, mode, selectedSource, startDate, submittedSearch])
useEffect(() => {
const filters = readUrlFilters(location.search)
if (filters.mode && filters.mode !== mode) setMode(filters.mode)
if (filters.selectedSource && filters.selectedSource !== selectedSource) setSelectedSource(filters.selectedSource)
if (filters.lineLimit && filters.lineLimit !== lineLimit) setLineLimit(filters.lineLimit)
if (filters.level && filters.level !== level) setLevel(filters.level)
if ((filters.startDate || '') !== startDate) setStartDate(filters.startDate || '')
if ((filters.endDate || '') !== endDate) setEndDate(filters.endDate || '')
if (filters.searchQuery !== undefined && filters.searchQuery !== searchQuery) {
setSearchQuery(filters.searchQuery)
setSubmittedSearch(filters.searchQuery)
}
if (filters.follow !== followEnabled && new URLSearchParams(location.search).has('follow')) {
setFollowEnabled(Boolean(filters.follow))
}
}, [location.search])
useEffect(() => {
if (typeof window === 'undefined') return
const params = new URLSearchParams()
if (mode !== 'grouped') params.set('mode', mode)
if (selectedSource && selectedSource !== 'backend') params.set('source', selectedSource)
if (lineLimit !== 200) params.set('limit', String(lineLimit))
if (level !== 'all') params.set('level', level)
if (startDate) params.set('start_date', startDate)
if (endDate) params.set('end_date', endDate)
if (submittedSearch.trim()) params.set('search', submittedSearch.trim())
if (followEnabled) params.set('follow', '1')
const nextSearch = params.toString()
const currentSearch = location.search.replace(/^\?/, '')
if (nextSearch !== currentSearch) {
navigate({ pathname: location.pathname, search: nextSearch ? `?${nextSearch}` : '' }, { replace: true })
}
}, [endDate, followEnabled, level, lineLimit, location.pathname, location.search, mode, navigate, selectedSource, startDate, submittedSearch])
useEffect(() => {
if (!followEnabled || !followConnected || !isSuperAdmin || !selectedSource) return
sendMessage({
type: 'subscribe',
data: {
channel: LOG_TAIL_CHANNEL,
source_id: selectedSource,
limit: lineLimit,
level,
levels: level === 'all' ? undefined : level,
start_date: startDate || undefined,
end_date: endDate || undefined,
search: submittedSearch.trim() || undefined,
},
})
}, [endDate, followConnected, followEnabled, isSuperAdmin, level, lineLimit, selectedSource, sendMessage, startDate, submittedSearch])
useEffect(() => {
const viewport = scrollContainerRef.current
if (!viewport || !shouldStickToBottomRef.current) return
viewport.scrollTop = viewport.scrollHeight
}, [snapshot?.lines])
useEffect(() => {
const viewport = scrollContainerRef.current
if (!viewport) return
viewport.addEventListener('scroll', handleLogScroll, { passive: true })
return () => viewport.removeEventListener('scroll', handleLogScroll)
}, [snapshot?.source_id])
const resetFilters = () => {
setLevel('all')
@@ -188,10 +453,21 @@ export default function Logs() {
setSearchQuery('')
setSubmittedSearch('')
setLineLimit(200)
setFollowEnabled(false)
}
const handleLogScroll = () => {
const viewport = scrollContainerRef.current
if (!viewport) return
const distanceToBottom = viewport.scrollHeight - viewport.scrollTop - viewport.clientHeight
shouldStickToBottomRef.current = distanceToBottom < 32
}
const copyLogs = async () => {
await navigator.clipboard.writeText(snapshot?.lines?.join('\n') || '')
const text = mode === 'grouped'
? groupEvents.map((event) => `${event.occurred_at || ''} ${event.level.toUpperCase()} ${event.message} ${stringifyContext(event.context)}`).join('\n')
: snapshot?.lines?.join('\n') || ''
await navigator.clipboard.writeText(text)
toast({ tone: 'success', title: '日志已复制' })
}
@@ -212,26 +488,78 @@ export default function Logs() {
<PageFrame
title="系统日志"
description="查看日志源、按级别/日期/搜索条件读取快照,并复制原始输出。"
className="an-logs-page"
actions={(
<>
<Button size="icon" variant="subtle" onClick={() => void fetchSources()} loading={sourcesLoading} aria-label="刷新日志源" title="刷新日志源">
<RefreshCw size={15} />
</Button>
<Button size="icon" variant="primary" onClick={() => void fetchSnapshot()} loading={logLoading} aria-label="刷新日志" title="刷新日志">
{mode === 'raw' ? (
<Button
size="icon"
variant={followEnabled ? 'primary' : 'subtle'}
onClick={() => setFollowEnabled((enabled) => !enabled)}
aria-label={followEnabled ? '暂停日志跟随' : '跟随日志'}
title={followEnabled ? '暂停日志跟随' : '跟随日志'}
>
{followEnabled ? <Pause size={15} /> : <Play size={15} />}
</Button>
) : null}
<Button size="icon" variant="primary" onClick={() => mode === 'grouped' ? void fetchGroups() : void fetchSnapshot()} loading={logLoading} aria-label="刷新日志" title="刷新日志">
<RefreshCw size={15} />
</Button>
</>
)}
>
<div className="an-section-tabs" role="tablist" aria-label="日志视图">
{LOG_VIEW_OPTIONS.map((option) => {
const active = mode === option.value
return (
<button
key={option.value}
type="button"
role="tab"
aria-selected={active}
className={active ? 'an-section-tab is-active' : 'an-section-tab'}
onClick={() => {
setMode(option.value as LogViewMode)
setFollowEnabled(false)
setSelectedGroup(null)
setGroupEvents([])
}}
>
<span className={active ? 'an-section-tab__dot an-section-tab__dot--success' : 'an-section-tab__dot an-section-tab__dot--neutral'} />
<span>{option.label}</span>
</button>
)
})}
</div>
<div className="an-logs-layout">
<Panel className="an-logs-sidebar">
<div className="an-panel-heading">
<div>
<h2></h2>
<p>{sources.length} </p>
<h2>{mode === 'grouped' ? '重复统计' : mode === 'audit' ? '审计来源' : '日志源'}</h2>
<p>{mode === 'grouped' ? `${groups.length} 个聚合项` : `${sources.length} 个来源`}</p>
</div>
</div>
<Scrollbar className="an-logs-source-list">
{mode === 'grouped' ? (
<Scrollbar className="an-logs-source-list">
{groups.map((group) => (
<button
type="button"
key={group.fingerprint}
className={selectedGroup?.fingerprint === group.fingerprint ? 'an-logs-source is-active' : 'an-logs-source'}
onClick={() => void fetchGroupEvents(group)}
>
<strong>{group.message}</strong>
<span>{group.category || group.event || group.module || group.source}</span>
<StatusText tone={levelTone(group.level)}>{group.count} </StatusText>
</button>
))}
{!groups.length && !logLoading ? <EmptyState title="暂无重复日志" /> : null}
</Scrollbar>
) : (
<Scrollbar className="an-logs-source-list">
{sources.map((source) => (
<button
type="button"
@@ -245,18 +573,20 @@ export default function Logs() {
</button>
))}
{!sources.length && !sourcesLoading ? <EmptyState title="暂无日志源" /> : null}
</Scrollbar>
</Scrollbar>
)}
</Panel>
<Panel className="an-logs-main">
<div className="an-panel-heading">
<div>
<h2>{selectedSourceInfo?.name || selectedSource}</h2>
<p>{selectedSourceInfo?.description || '选择日志源后读取快照。'}</p>
<h2>{mode === 'grouped' ? (selectedGroup ? '重复日志详情' : '重复日志统计') : mode === 'audit' ? '审计事件' : (selectedSourceInfo?.name || selectedSource)}</h2>
<p>{mode === 'grouped' ? '点击左侧聚合项查看每次发生时间。' : mode === 'audit' ? '管理员敏感操作和安全审计记录。' : (selectedSourceInfo?.description || '选择日志源后读取快照。')}</p>
</div>
<div className="an-toolbar">
{snapshot ? <Badge tone="blue">{snapshot.line_count} </Badge> : null}
<Button size="icon" variant="subtle" onClick={copyLogs} disabled={!snapshot?.lines?.length} aria-label="复制日志" title="复制日志">
{mode === 'grouped' ? <Badge tone="blue">{selectedGroup ? groupEvents.length : groups.length} </Badge> : snapshot ? <Badge tone="blue">{snapshot.line_count} </Badge> : null}
{followEnabled ? <Badge tone={followConnected ? 'green' : 'amber'}>{followConnected ? '跟随中' : '连接中'}</Badge> : null}
<Button size="icon" variant="subtle" onClick={copyLogs} disabled={mode === 'grouped' ? !groupEvents.length : !snapshot?.lines?.length} aria-label="复制日志" title="复制日志">
<Copy size={15} />
</Button>
</div>
@@ -288,8 +618,32 @@ export default function Logs() {
<div className="an-log-reader">
{logLoading ? (
<div className="an-loading"><span className="an-spinner" /></div>
) : mode === 'grouped' && selectedGroup ? (
<Scrollbar className="an-log-reader__scroll" viewportRef={scrollContainerRef}>
<div className="an-log-group-detail">
<div className="an-log-group-summary">
<strong>{selectedGroup.message}</strong>
<span> {selectedGroup.fingerprint}</span>
<span> {formatDateTime(selectedGroup.first_seen_at)} · {formatDateTime(selectedGroup.last_seen_at)}</span>
</div>
{groupEvents.map((event) => (
<div className="an-log-occurrence" key={event.id}>
<div>
<strong>{formatDateTime(event.occurred_at)}</strong>
<StatusText tone={levelTone(event.level)}>{event.level}</StatusText>
{event.occurrence_count > 1 ? <Badge tone="blue">{event.occurrence_count} </Badge> : null}
</div>
<p>{event.message}</p>
<pre>{stringifyContext(event.context)}</pre>
</div>
))}
{!groupEvents.length ? <EmptyState title="暂无发生明细" /> : null}
</div>
</Scrollbar>
) : mode === 'grouped' ? (
<EmptyState title="选择一组重复日志" description="左侧展示按 fingerprint 聚合后的运行时错误。" />
) : snapshot?.lines?.length ? (
<Scrollbar className="an-log-reader__scroll">
<Scrollbar className="an-log-reader__scroll" viewportRef={scrollContainerRef}>
<pre>{snapshot.lines.join('\n')}</pre>
</Scrollbar>
) : (

File diff suppressed because it is too large Load Diff

View File

@@ -1,6 +1,6 @@
import { type HTMLAttributes, type ReactNode } from 'react'
import Scrollbar from '../../components/Scrollbar/Scrollbar'
import { cn } from '../lib/utils'
import { cn } from '../utils'
export function PageFrame({
title,

View File

@@ -48,7 +48,7 @@ export const adminRoutes: AdminRouteItem[] = [
{ path: '/alerts/bgp', label: 'BGP 告警', group: 'alerts', icon: Network, keywords: ['alert', 'bgp', '风险'] },
{ path: '/alerts/situational', label: '态势告警', group: 'alerts', icon: Globe2, keywords: ['situational', '态势', '研判'] },
{ path: '/ai', label: 'AI', group: 'ops', icon: Bot, keywords: ['ai', 'provider', 'playground', 'prompt'] },
{ path: '/earth-content', label: '智能星球内容', group: 'ops', icon: Globe2, keywords: ['earth', '地球', '智能星球', 'tv', 'boundary', 'brand'] },
{ path: '/earth-content', label: '智能星球内容', group: 'ops', icon: Globe2, keywords: ['earth', '地球', '智能星球', 'tv', 'boundary', 'brand', 'news', 'rss', '新闻源'] },
{ path: '/collection-management', label: '采集管理', group: 'ops', icon: Database, keywords: ['collector', 'mapping', 'custom source'] },
{ path: '/logs', label: '系统日志', group: 'ops', icon: FileText, keywords: ['log', '日志', 'tail'], superAdminOnly: true },
{ path: '/users', label: '用户管理', group: 'ops', icon: Users, keywords: ['users', 'role', 'gatekeeper'] },

View File

@@ -0,0 +1,167 @@
const RECENT_EVENT_TTL_MS = 15_000
const MAX_DETAIL_LENGTH = 4000
const recentEventMap = new Map<string, number>()
const pendingEventMap = new Map<string, {
level: string
message: string
category: string
module: string
detail: string
fingerprint: string
occurrenceCount: number
}>()
let pendingFlushTimer: number | null = null
const NON_ADMIN_PATH_PREFIXES = ['/earth', '/docs', '/login', '/register', '/verify-email', '/forgot-password']
type RuntimeLogLevel = 'error' | 'warning' | 'info' | 'debug'
type AdminRuntimeLogInput = {
level?: RuntimeLogLevel
message: string
category?: string
module?: string
detail?: unknown
}
function normalizeErrorDetail(detail: unknown) {
if (!detail) return ''
if (detail instanceof Error) {
return detail.stack || detail.message || String(detail)
}
if (typeof detail === 'string') return detail
try {
return JSON.stringify(detail)
} catch {
return String(detail)
}
}
function shouldSkip(level: string, message: string, detail: string, category: string) {
const key = `${level}::${category}::${message}::${detail}`
const now = Date.now()
const lastSeenAt = recentEventMap.get(key)
recentEventMap.set(key, now)
for (const [entryKey, entryTime] of recentEventMap.entries()) {
if (now - entryTime > RECENT_EVENT_TTL_MS) {
recentEventMap.delete(entryKey)
}
}
return Boolean(lastSeenAt && now - lastSeenAt < RECENT_EVENT_TTL_MS)
}
function normalizeFingerprintText(value: unknown) {
return String(value || '')
.replace(/[?&](m|t|token|expires|signature|X-Amz-[^=]+)=[^&\s]+/gi, '')
.replace(/\b[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}\b/gi, '<uuid>')
.replace(/\bconn_[A-Za-z0-9:._-]+\b/g, '<connection>')
.replace(/\b\d{5,}\b/g, '<number>')
.replace(/\s+/g, ' ')
.trim()
}
function hashFingerprint(value: string) {
let hash = 5381
for (let index = 0; index < value.length; index += 1) {
hash = ((hash << 5) + hash) ^ value.charCodeAt(index)
}
return `admin-${(hash >>> 0).toString(16).padStart(8, '0')}`
}
function buildFingerprint(level: string, message: string, detail: string, category: string, module: string) {
return hashFingerprint([level, category, module, message, detail].map(normalizeFingerprintText).join('|'))
}
function isAdminRoute() {
if (typeof window === 'undefined') return false
const pathname = window.location.pathname
return pathname !== '/' && !NON_ADMIN_PATH_PREFIXES.some((prefix) => pathname === prefix || pathname.startsWith(`${prefix}/`))
}
export async function reportAdminRuntimeLog({
level = 'error',
message,
category = 'runtime',
module = 'admin',
detail = '',
}: AdminRuntimeLogInput) {
if (!message || typeof window === 'undefined' || !isAdminRoute()) return
const normalizedDetail = normalizeErrorDetail(detail)
const fingerprint = buildFingerprint(level, message, normalizedDetail, category, module)
const key = `${level}::${category}::${message}::${normalizedDetail}`
const existing = pendingEventMap.get(key)
if (existing) {
existing.occurrenceCount += 1
} else {
pendingEventMap.set(key, {
level,
message,
category,
module,
detail: normalizedDetail,
fingerprint,
occurrenceCount: 1,
})
}
shouldSkip(level, message, normalizedDetail, category)
if (pendingFlushTimer) return
pendingFlushTimer = window.setTimeout(() => {
pendingFlushTimer = null
const pending = Array.from(pendingEventMap.values())
pendingEventMap.clear()
pending.forEach((entry) => {
void sendAdminRuntimeLog(entry)
})
}, 1000)
}
async function sendAdminRuntimeLog(entry: {
level: string
message: string
category: string
module: string
detail: string
fingerprint: string
occurrenceCount: number
}) {
try {
await fetch('/api/v1/system/logs/admin-client', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({
level: entry.level,
message: entry.message,
category: entry.category,
module: entry.module,
url: window.location.href,
detail: entry.detail.slice(0, MAX_DETAIL_LENGTH),
fingerprint: entry.fingerprint,
occurrence_count: entry.occurrenceCount,
}),
keepalive: true,
})
} catch {
// Runtime log reporting must never create more runtime noise.
}
}
export function registerAdminRuntimeErrorHandlers() {
if (typeof window === 'undefined') return
window.addEventListener('error', (event) => {
void reportAdminRuntimeLog({
level: 'error',
category: 'window-error',
module: 'admin',
message: event.message || '控制台发生未捕获错误',
detail: event.error || `${event.filename || ''}:${event.lineno || 0}:${event.colno || 0}`,
})
})
window.addEventListener('unhandledrejection', (event) => {
void reportAdminRuntimeLog({
level: 'error',
category: 'unhandledrejection',
module: 'admin',
message: '控制台发生未处理 Promise 错误',
detail: event.reason,
})
})
}

View File

@@ -81,6 +81,7 @@ const sectionTargets = [
{ key: 'brand', label: '品牌标识', terms: ['logo', '标题', 'subtitle'] },
{ key: 'earth_assets', label: '国界精度', terms: ['boundary', 'PMTiles', '边界'] },
{ key: 'tv', label: '电视直播', terms: ['TV', '直播源', '频道'] },
{ key: 'news_sources', label: '新闻源', terms: ['news', 'rss', '商业新闻', '电商', '财经', '新闻类型'] },
] },
{ routePath: '/collection-management', routeLabel: '采集管理', icon: Database, sections: [
{ key: 'collector_credentials', label: '采集器', terms: ['collector', 'credential', '凭证教程'] },

View File

@@ -444,8 +444,9 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
}
.an-collection-queue__item {
position: relative;
display: grid;
grid-template-columns: minmax(0, 1fr) auto auto;
grid-template-columns: minmax(0, 1fr) 44px;
align-items: center;
gap: 8px;
padding: 8px;
@@ -455,6 +456,12 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
}
.an-collection-queue__item-actions {
position: absolute;
top: 6px;
right: 6px;
z-index: 2;
display: flex;
gap: 6px;
opacity: 0;
transform: translateX(4px);
pointer-events: none;
@@ -470,6 +477,7 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
.an-collection-queue__item strong,
.an-collection-queue__item p {
max-width: 100%;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
@@ -522,6 +530,10 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
transform: none;
pointer-events: auto;
}
.an-collection-queue__item {
padding-right: 104px;
}
}
.an-task-summary {
@@ -577,6 +589,12 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
white-space: nowrap;
}
.an-task-summary__actions {
grid-column: 1 / -1;
display: flex;
justify-content: flex-end;
}
.admin__content {
min-width: 0;
height: 100vh;
@@ -1523,6 +1541,10 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
container-type: inline-size;
}
.an-hierarchy-list__header {
padding: 10px 10px 0;
}
.an-hierarchy-list__scroll,
.an-hierarchy-form-scroll {
min-height: 0;
@@ -1544,6 +1566,33 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
padding: 10px 0 4px;
}
.an-news-source-filters {
display: grid;
grid-template-columns: repeat(4, minmax(140px, 1fr));
gap: 8px;
align-items: center;
}
.an-hierarchy-workspace {
min-height: 0;
display: grid;
grid-template-rows: auto minmax(0, 1fr);
gap: var(--an-section-gap, 16px);
overflow: hidden;
}
@media (max-width: 900px) {
.an-news-source-filters {
grid-template-columns: repeat(2, minmax(0, 1fr));
}
}
@media (max-width: 560px) {
.an-news-source-filters {
grid-template-columns: 1fr;
}
}
.an-hierarchy-group-set {
display: grid;
gap: 6px;
@@ -2322,6 +2371,35 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
outline: none;
}
.an-news-feed-list {
display: grid;
gap: 10px;
}
.an-news-feed-card {
display: grid;
gap: 10px;
padding: 10px;
border: 1px solid var(--an-border);
border-radius: 7px;
background: var(--an-surface-alt);
}
.an-news-feed-card__header {
display: flex;
align-items: center;
justify-content: space-between;
gap: 12px;
}
.an-news-feed-card__actions {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
justify-content: flex-end;
}
.an-textarea {
height: auto;
min-height: 94px;
@@ -2385,7 +2463,7 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
.an-select__content {
z-index: 300;
min-width: var(--radix-select-trigger-width);
max-width: min(360px, calc(100vw - 24px));
max-width: min(420px, calc(100vw - 24px));
background: var(--an-surface, #ffffff);
color: var(--an-text, #0f172a);
border: 1px solid var(--an-border);
@@ -2404,21 +2482,32 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
min-height: 30px;
max-width: 100%;
border-radius: 4px;
padding: 0 8px;
display: flex;
padding: 6px 28px 6px 8px;
display: grid;
grid-template-columns: minmax(0, 1fr);
align-items: center;
gap: 8px;
cursor: pointer;
outline: none;
line-height: 1.25;
white-space: nowrap;
white-space: normal;
position: relative;
}
.an-select__item span:first-child {
min-width: 0;
overflow: hidden;
text-overflow: ellipsis;
white-space: nowrap;
max-width: 100%;
overflow: visible;
text-overflow: clip;
white-space: normal;
overflow-wrap: anywhere;
}
.an-select__indicator {
position: absolute;
top: 50%;
right: 8px;
display: inline-flex;
transform: translateY(-50%);
}
.an-select__item[data-highlighted] {
@@ -2455,34 +2544,64 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
background: currentColor;
}
.an-badge--green,
.an-status-pill--success {
.an-badge--green {
border-color: color-mix(in srgb, var(--an-success) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-success) 12%, var(--an-surface-alt));
color: var(--an-success);
}
.an-badge--amber,
.an-status-pill--warning {
.an-badge--amber {
border-color: color-mix(in srgb, var(--an-warning) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-warning) 12%, var(--an-surface-alt));
color: var(--an-warning);
}
.an-badge--red,
.an-status-pill--danger {
.an-badge--red {
border-color: color-mix(in srgb, var(--an-danger) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-danger) 12%, var(--an-surface-alt));
color: var(--an-danger);
}
.an-badge--blue,
.an-badge--cyan,
.an-badge--cyan {
border-color: color-mix(in srgb, var(--an-info) 34%, var(--an-border));
background: color-mix(in srgb, var(--an-info) 12%, var(--an-surface-alt));
color: var(--an-info);
}
.an-badge--purple {
border-color: color-mix(in srgb, #7c3aed 34%, var(--an-border));
background: color-mix(in srgb, #7c3aed 12%, var(--an-surface-alt));
color: #7c3aed;
}
.an-badge--slate {
border-color: color-mix(in srgb, var(--an-muted) 28%, var(--an-border));
background: color-mix(in srgb, var(--an-muted) 10%, var(--an-surface-alt));
color: var(--an-muted);
}
.an-status-pill--success {
color: var(--an-success);
}
.an-status-pill--warning {
color: var(--an-warning);
}
.an-status-pill--danger {
color: var(--an-danger);
}
.an-status-pill--info,
.an-status-pill--running {
color: var(--an-info);
}
.an-badge--purple,
.an-status-pill--ai {
color: #7c3aed;
}
.an-badge--slate,
.an-status-pill--neutral {
color: var(--an-muted);
}
@@ -3276,9 +3395,15 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
line-height: 1.65;
}
.an-logs-page .an-page__body {
display: grid;
grid-template-rows: auto minmax(0, 1fr);
gap: var(--an-section-gap);
}
.an-logs-layout {
min-height: 0;
height: min(760px, calc(100vh - 170px));
height: 100%;
display: grid;
grid-template-columns: 280px minmax(0, 1fr);
gap: var(--an-section-gap);
@@ -3323,7 +3448,7 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
background: var(--an-surface-alt);
}
.an-logs-source span {
.an-logs-source > span:not(.an-status-pill) {
color: var(--an-muted);
font-size: 12px;
}
@@ -3366,6 +3491,54 @@ body:has(.admin-theme-root[data-theme='dark']) .an-toast {
font-family: ui-monospace, SFMono-Regular, Menlo, Monaco, Consolas, monospace;
}
.an-log-group-detail {
display: grid;
gap: 10px;
padding: 10px;
color: #dbeafe;
}
.an-log-group-summary,
.an-log-occurrence {
border: 1px solid rgba(148, 163, 184, 0.28);
border-radius: 8px;
background: rgba(15, 23, 42, 0.72);
padding: 10px;
}
.an-log-group-summary {
display: grid;
gap: 4px;
}
.an-log-group-summary span,
.an-log-occurrence p {
color: #94a3b8;
}
.an-log-occurrence {
display: grid;
gap: 8px;
}
.an-log-occurrence > div {
display: flex;
align-items: center;
gap: 8px;
flex-wrap: wrap;
}
.an-log-occurrence pre {
min-width: 0;
border-radius: 6px;
background: rgba(2, 6, 23, 0.82);
padding: 10px;
max-height: 220px;
overflow: auto;
white-space: pre-wrap;
word-break: break-word;
}
.an-playground {
min-height: 0;
height: 100%;

View File

@@ -2,8 +2,11 @@ import React from 'react'
import ReactDOM from 'react-dom/client'
import { BrowserRouter } from 'react-router-dom'
import App from './App'
import { registerAdminRuntimeErrorHandlers } from './admin/runtimeLogs'
import './index.css'
registerAdminRuntimeErrorHandlers()
ReactDOM.createRoot(document.getElementById('root')!).render(
<React.StrictMode>
<BrowserRouter future={{ v7_startTransition: true, v7_relativeSplatPath: true }}>

View File

@@ -111,9 +111,17 @@ export const DOCS_METADATA: Record<string, DocsMetadataEntry> = {
zh: { title: '智能星球可交互图标接入', group: 'Earth', order: 16 },
en: { title: 'Intelligent Planet Interactable Usage', group: 'Earth', order: 16 },
},
'earth-interactable-clustering.md': {
zh: { title: '智能星球可交互图标聚类策略', group: 'Earth', order: 17 },
en: { title: 'Intelligent Planet Interactable Clustering', group: 'Earth', order: 17 },
},
'earth-toolbar-overlay-coordination.md': {
zh: { title: '智能星球工具栏与浮层协同', group: 'Earth', order: 17 },
en: { title: 'Intelligent Planet Toolbar and Overlay Coordination', group: 'Earth', order: 17 },
zh: { title: '智能星球工具栏与浮层协同', group: 'Earth', order: 18 },
en: { title: 'Intelligent Planet Toolbar and Overlay Coordination', group: 'Earth', order: 18 },
},
'earth-news-sources.md': {
zh: { title: '智能星球新闻源配置', group: 'Earth', order: 19 },
en: { title: 'Intelligent Planet News Source Configuration', group: 'Earth', order: 19 },
},
'frontend-admin-frontend-context.md': {
zh: { title: '控制台前端结构', group: 'Frontend', order: 20 },

View File

@@ -138,6 +138,7 @@ AI_PROVIDER_IMAGE_NAME="${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}"
AI_PROVIDER_CONTAINER_NAME="${AI_PROVIDER_CONTAINER_NAME:-planet_aiprovider}"
PLANET_AI_PROVIDER_RUNTIME_ENV_FILE="${PLANET_AI_PROVIDER_RUNTIME_ENV_FILE:-$PLANET_STATE_DIR/aiprovider_runtime.env}"
PLANET_EMPTY_UV_CONFIG_FILE="$PLANET_STATE_DIR/uv.empty.toml"
PLANET_TUNA_UV_CONFIG_FILE="$PLANET_STATE_DIR/uv.tuna.toml"
PLANET_UV_CONFIG_FILE="${PLANET_UV_CONFIG_FILE:-}"
AI_PROVIDER_RECREATE_REQUIRED=0
START_RUN_ACTIVE=0
@@ -1111,55 +1112,94 @@ run_with_retry() {
run_uv_sync_with_mirror_fallback() {
local log_file="$1"
local lock_digest_before
local sync_status
lock_digest_before="$(uv_lock_digest)"
if run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"uv sync 默认源失败,已重试 ${DEPENDENCY_INSTALL_MAX_RETRIES} 次,准备切换清华源" \
"uv sync --frozen" \
run_command_quiet_unless_verbose "$log_file" uv sync --frozen --group dev; then
assert_uv_lock_unchanged "$lock_digest_before" "uv sync --frozen"
return 0
fi
assert_uv_lock_unchanged "$lock_digest_before" "uv sync --frozen"
configure_uv_tuna_index
set_wait_detail "已写入 uv.toml 清华源,重新执行 uv sync"
set_wait_detail "已准备临时清华源配置,重新执行 uv sync"
lock_digest_before="$(uv_lock_digest)"
run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"uv 环境初始化失败,清华源重试 ${DEPENDENCY_INSTALL_MAX_RETRIES} 次后仍失败" \
"uv sync --frozen (清华源)" \
run_uv_sync_with_tuna_config "$log_file"
sync_status=$?
assert_uv_lock_unchanged "$lock_digest_before" "uv sync --frozen (清华源)"
return "$sync_status"
}
run_uv_sync_with_tuna_config() {
local log_file="$1"
UV_CONFIG_FILE="$PLANET_TUNA_UV_CONFIG_FILE" \
run_command_quiet_unless_verbose "$log_file" uv sync --frozen --group dev
}
configure_uv_tuna_index() {
local uv_config="$SCRIPT_DIR/uv.toml"
local uv_config="$PLANET_TUNA_UV_CONFIG_FILE"
if [ -f "$uv_config" ] &&
grep -Fq 'name = "tsinghua"' "$uv_config" 2>/dev/null &&
grep -Fq "$PLANET_UV_TUNA_INDEX_URL" "$uv_config" 2>/dev/null &&
grep -Eq '^[[:space:]]*default[[:space:]]*=[[:space:]]*true' "$uv_config" 2>/dev/null; then
log_note "uv.toml 已配置清华源,直接重试 uv sync"
return 0
fi
if [ -f "$uv_config" ] && grep -Eq '^[[:space:]]*default[[:space:]]*=[[:space:]]*true' "$uv_config" 2>/dev/null; then
log_warn "uv.toml 已存在默认 index未覆盖用户配置"
log_note "如需手动切换清华源,可添加 [[index]] name=\"tsinghua\" 并设置 default=true"
log_note "临时 uv 清华源配置已存在,直接重试 uv sync"
return 0
fi
{
if [ -s "$uv_config" ]; then
printf "\n"
fi
printf '[[index]]\n'
printf 'name = "tsinghua"\n'
printf 'url = "%s"\n' "$PLANET_UV_TUNA_INDEX_URL"
printf 'default = true\n'
} >> "$uv_config"
} > "$uv_config"
log_note "已向 uv.toml 添加清华 PyPI 源: ${PLANET_UV_TUNA_INDEX_URL}"
chmod 600 "$uv_config" 2>/dev/null || true
log_note "已写入临时 uv 清华源配置: ${uv_config}"
}
uv_lock_digest() {
local lock_file="$SCRIPT_DIR/uv.lock"
if [ ! -f "$lock_file" ]; then
printf "missing\n"
return 0
fi
if command -v sha256sum >/dev/null 2>&1; then
sha256sum "$lock_file" | awk '{print $1}'
return 0
fi
cksum "$lock_file" | awk '{print $1 ":" $2}'
}
assert_uv_lock_unchanged() {
local expected_digest="$1"
local action_label="$2"
local current_digest
current_digest="$(uv_lock_digest)"
if [ "$current_digest" = "$expected_digest" ]; then
return 0
fi
log_error "${action_label} 修改了 uv.lock新环境依赖安装不允许污染 lockfile"
log_note "请还原 uv.lock并只在明确升级依赖时手动运行 uv lock"
exit 1
}
install_system_package() {
@@ -1763,19 +1803,15 @@ ensure_frontend_deps() {
cd "$SCRIPT_DIR/frontend"
: > "$log_file"
set_wait_detail "检查 Vite Bun 入口是否已安装"
if [ ! -f "$FRONTEND_VITE_ENTRY" ]; then
log_warn "前端依赖缺失,正在执行 bun install (${FRONTEND_RUNTIME_SOURCE})"
set_wait_detail "执行 ${FRONTEND_RUNTIME_SOURCE} bun install"
if ! run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"前端依赖安装失败,已重试 ${DEPENDENCY_INSTALL_MAX_RETRIES}" \
"bun install" \
run_command_quiet_unless_verbose "$log_file" "$FRONTEND_RUNTIME_BIN" install; then
tail -20 "$log_file" 2>/dev/null || true
exit 1
fi
set_wait_detail "同步前端依赖"
if ! run_with_retry \
"$DEPENDENCY_INSTALL_MAX_RETRIES" \
"$DEPENDENCY_INSTALL_RETRY_INTERVAL" \
"前端依赖安装失败,已重试 ${DEPENDENCY_INSTALL_MAX_RETRIES}" \
"bun install" \
run_command_quiet_unless_verbose "$log_file" "$FRONTEND_RUNTIME_BIN" install; then
tail -20 "$log_file" 2>/dev/null || true
exit 1
fi
if [ ! -f "$FRONTEND_VITE_ENTRY" ]; then

View File

@@ -1,6 +1,6 @@
[project]
name = "planet"
version = "0.66.3"
version = "0.69.0"
description = "智能星球计划 - 态势感知系统"
requires-python = ">=3.14"
dependencies = [

2
uv.lock generated
View File

@@ -757,7 +757,7 @@ wheels = [
[[package]]
name = "planet"
version = "0.66.3"
version = "0.69.0"
source = { virtual = "." }
dependencies = [
{ name = "aiofiles" },