Compare commits

...

8 Commits
ci ... v0.75.2

Author SHA1 Message Date
rayd1o
a2025dbfe0 release: bump version to 0.75.2 2026-09-25 02:03:53 +08:00
rayd1o
003b316d42 release: bump version to 0.75.1 2026-09-25 00:16:30 +08:00
rayd1o
0c40764c58 feat: configure compute comparisons and restore adaptive globe dragging 2026-09-24 22:59:55 +08:00
rayd1o
5bfc790dec release: bump version to 0.75.0 2026-09-23 21:20:12 +08:00
linkong
60eefb19c9 release: bump version to 0.74.7 2026-09-17 17:30:54 +08:00
rayd1o
83a10a6c34 release: bump version to 0.74.6 2026-09-16 21:05:40 +08:00
rayd1o
cee1996809 release: bump version to 0.74.5 2026-09-13 14:04:34 +08:00
rayd1o
58671e7bc3 release: bump version to 0.74.4 2026-09-13 10:27:00 +08:00
287 changed files with 21354 additions and 2289 deletions

View File

@@ -5,6 +5,15 @@
!VERSION
!backend/
!backend/**
!config/
!config/earth-boundary-pov-policy.china-v1.json
!scripts/
!scripts/build_earth_boundary_china_pov_geojson.py
!scripts/build_earth_boundary_pmtiles.py
!scripts/earth_boundary_policy.py
!resources/
!resources/earth/
!resources/earth/boundary-pov-geometry.china-v1.json
!aiprovider/
!aiprovider/**

5
.gitignore vendored
View File

@@ -160,3 +160,8 @@ data/earth-boundary-sources/
# Git; publish PMTiles/MVT artifacts through release/deploy storage instead of
# committing thousands of generated tile files.
frontend/public/earth/data/boundaries/
# Offline comparison originals; runtime 8K maps and provenance remain versioned.
resources/earth/candidates/nasa-blue-marble/*.jpg
resources/earth/candidates/natural-earth-3/*.jpg
*:Zone.Identifier

View File

@@ -113,6 +113,9 @@ behavior and document the compatibility note in `docs/harness-audit.md` or
## Operating Mode
- Default to acting directly when the user gives a clear task.
- For authorized multi-agent work, follow the Coding Agent Model Strategy in
`rules.md` workflow: Sol for implementation, Luna for bounded supporting work,
and Astra for complex reasoning; the main agent owns final verification.
- Ask before acting only when the missing decision is risky, cannot be
discovered from repository context, and no conservative assumption is safe.
- Read relevant files before editing.

View File

@@ -17,6 +17,7 @@ are the source of detail for specific subsystems.
| `deploy/helm/planet/` | Helm chart for staging/deployment smoke paths | CI runs helm lint/template when delivery checks are available. |
| `.gitea/workflows/` | CI, release image build, staging deploy workflows | This repository uses Gitea workflow files, not `.github/workflows/`. |
| `planet.sh` | Main local lifecycle script | Owns init/start/restart/stop/health/log/createuser/destroy. |
| `planet.cmd` and `scripts/windows/` | Portable Windows entry, bound desktop launcher generation, and WSL restart bridge | Delegates service lifecycle to `planet.sh`; Windows and isolated WSL regression tests live under `scripts/harness/`. |
## Runtime Entry Points

View File

@@ -324,7 +324,7 @@ ipconfig
## 启动容错参数
`planet.sh` 现在为依赖安装、数据库、AI Provider 启动加入了有限次重试,并会在数据库与 `aiprovider` 启动后额外等待 Docker healthcheck。
`planet.sh` 为依赖安装、数据库、AI Provider 启动提供有限次重试。数据库先检查容器健康,再验证后端实际连接;`aiprovider` 直接以宿主机 `/health` 就绪为准。后端进程退出或应用初始化失败时立即停止等待,避免重复消耗健康检查预算。
可通过环境变量临时调整:

View File

@@ -1 +1 @@
0.74.3
0.75.2

View File

@@ -1,4 +1,4 @@
# syntax=docker/dockerfile:1.7
# Use BuildKit's bundled frontend to avoid a separate Docker Hub fetch.
ARG PYTHON_IMAGE=python:3.14-slim
ARG UV_IMAGE=ghcr.io/astral-sh/uv:latest
@@ -7,9 +7,6 @@ ARG AI_PROVIDER_BUILD_FINGERPRINT=unknown
FROM ${UV_IMAGE} AS uv
FROM ${PYTHON_IMAGE}
ARG AI_PROVIDER_BUILD_FINGERPRINT
LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
COPY --from=uv /uv /uvx /bin/
WORKDIR /app
@@ -28,13 +25,16 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
COPY pyproject.toml uv.lock /app/
RUN --mount=type=cache,target=/root/.cache/uv \
--mount=type=secret,id=planet_uv_config,target=/root/.config/uv/uv.toml,required=false \
uv sync --frozen --no-dev
uv sync --frozen --only-group aiprovider
COPY aiprovider /app/aiprovider
ARG AI_PROVIDER_BUILD_FINGERPRINT
LABEL planet.aiprovider.build-fingerprint="${AI_PROVIDER_BUILD_FINGERPRINT}"
EXPOSE 8010
HEALTHCHECK --interval=30s --timeout=5s --retries=3 \
CMD curl -fsS http://127.0.0.1:8010/health >/dev/null || exit 1
CMD ["uv", "run", "--frozen", "--no-dev", "--project", "/app", "python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]
CMD ["/app/.venv/bin/python", "-m", "uvicorn", "aiprovider.main:app", "--host", "0.0.0.0", "--port", "8010"]

View File

@@ -4,7 +4,7 @@
完整使用说明见:
- [docs/agents/aiprovider.md](/home/ray/dev/linkong/planet/docs/agents/aiprovider.md)
- [AI Provider 指南](../docs/technical/zh/agents-aiprovider.md)
当前支持:
@@ -15,6 +15,7 @@
- `AI_PROVIDER=ollama`
- request adapter:
- `AI_PROVIDER_API=openai-completions`
- `AI_PROVIDER_API=openai-responses`
- `AI_PROVIDER_API=anthropic-messages`
- `AI_PROVIDER_API=ollama-generate`

View File

@@ -3,6 +3,7 @@ from __future__ import annotations
import asyncio
import json
from typing import Any
from uuid import NAMESPACE_URL, uuid4, uuid5
import httpx
from fastapi import HTTPException, status
@@ -65,6 +66,7 @@ class ProviderService:
self.model_provider_apis = self._parse_model_provider_apis(
overrides.get("model_provider_apis")
)
self.session_id = str(uuid4())
def get_status(self) -> AIProviderStatusResponse:
enabled = self.provider != "disabled"
@@ -95,6 +97,8 @@ class ProviderService:
)
prompt = self._build_prompt(payload)
if payload.context.get("session_id") is not None:
self.session_id = str(uuid5(NAMESPACE_URL, f"planet:{payload.context['session_id']}"))
provider_api = self._resolve_model_provider_api(model)
@@ -102,6 +106,10 @@ class ProviderService:
data = await self._request_openai_compatible(model, prompt, payload.system_prompt)
content = self._extract_openai_content(data)
content_blocks = self._extract_openai_blocks(data)
elif provider_api == "openai-responses":
data = await self._request_openai_responses(model, prompt, payload.system_prompt)
content_blocks = self._extract_responses_blocks(data)
content = "".join(block.text for block in content_blocks if block.text)
elif provider_api == "anthropic-messages":
data = await self._request_anthropic_messages(
model,
@@ -200,6 +208,39 @@ class ProviderService:
request_body=request_body,
)
async def _request_openai_responses(
self, model: str, prompt: str, system_prompt: str | None = None,
) -> dict[str, Any]:
request_body: dict[str, Any] = {
"model": model, "input": prompt, "max_output_tokens": self.max_tokens, "store": False,
}
resolved_system_prompt = self._resolve_system_prompt(system_prompt)
if resolved_system_prompt:
request_body["instructions"] = resolved_system_prompt
return await self._post(
path="/responses",
headers={"Authorization": f"Bearer {self.api_key}", "Content-Type": "application/json"},
request_body=request_body,
)
def _extract_responses_blocks(self, payload: dict[str, Any]) -> list[AIContentBlock]:
blocks: list[AIContentBlock] = []
for item in payload.get("output") or []:
if not isinstance(item, dict):
continue
if item.get("type") == "message":
for part in item.get("content") or []:
if not isinstance(part, dict):
continue
text = part.get("text") or part.get("refusal")
if isinstance(text, str) and text:
blocks.append(AIContentBlock(type="text", text=text))
elif item.get("type") == "reasoning":
for part in item.get("summary") or []:
if isinstance(part, dict) and isinstance(part.get("text"), str):
blocks.append(AIContentBlock(type="thinking", thinking=part["text"]))
return blocks
async def _request_anthropic_messages(
self,
model: str,
@@ -226,7 +267,7 @@ class ProviderService:
resolved_system_prompt = self._resolve_system_prompt(system_prompt)
if resolved_system_prompt:
request_body["system"] = resolved_system_prompt
resolved_thinking = self._resolve_anthropic_thinking(thinking)
resolved_thinking = self._resolve_anthropic_thinking(thinking, model)
if resolved_thinking:
request_body["thinking"] = resolved_thinking
if self.provider == "minimax" and self.base_url.endswith("/anthropic"):
@@ -243,8 +284,12 @@ class ProviderService:
request_body=request_body,
)
def _resolve_anthropic_thinking(self, thinking: dict[str, Any] | None) -> dict[str, Any] | None:
def _resolve_anthropic_thinking(
self, thinking: dict[str, Any] | None, model: str,
) -> dict[str, Any] | None:
if thinking:
if model.casefold() == "minimax-m3" and thinking.get("type") == "enabled":
return {"type": "adaptive"}
return thinking
# OpenClaw treats MiniMax's Anthropic-compatible path specially:
@@ -295,6 +340,9 @@ class ProviderService:
headers: dict[str, str],
request_body: dict[str, Any],
) -> dict[str, Any]:
headers = {"User-Agent": "Planet/1.0", **headers}
if self.provider == "opencode-go":
headers["x-opencode-session"] = self.session_id
last_error: Exception | None = None
for attempt in range(1, self.http_retry_attempts + 1):
try:

View File

@@ -28,6 +28,11 @@ RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev
COPY backend /app/backend
COPY config/earth-boundary-pov-policy.china-v1.json /app/config/earth-boundary-pov-policy.china-v1.json
COPY scripts/build_earth_boundary_china_pov_geojson.py /app/scripts/build_earth_boundary_china_pov_geojson.py
COPY scripts/build_earth_boundary_pmtiles.py /app/scripts/build_earth_boundary_pmtiles.py
COPY scripts/earth_boundary_policy.py /app/scripts/earth_boundary_policy.py
COPY resources/earth/boundary-pov-geometry.china-v1.json /app/resources/earth/boundary-pov-geometry.china-v1.json
COPY VERSION /app/VERSION
EXPOSE 8000

View File

@@ -12,6 +12,7 @@ from app.api.v1 import (
alerts,
settings,
collected_data,
compute_intelligence,
data_products,
layers,
visualization,
@@ -43,6 +44,7 @@ api_router.include_router(alerts.router, prefix="/alerts", tags=["alerts"])
api_router.include_router(settings.router, prefix="/settings", tags=["settings"])
api_router.include_router(system_control.router, prefix="/system", tags=["system"])
api_router.include_router(data_products.router, prefix="/data-products", tags=["data-products"])
api_router.include_router(compute_intelligence.router, prefix="/data-products/compute", tags=["compute"])
api_router.include_router(layers.router, prefix="/layers", tags=["layers"])
api_router.include_router(visualization.router, prefix="/visualization", tags=["visualization"])
api_router.include_router(

View File

@@ -0,0 +1,231 @@
"""Compute observations, administrator configuration and independent facility samples."""
from datetime import UTC, datetime
from typing import Literal
from app.api.v1.visualization import convert_compute_centers_to_geojson
from app.core.earth_geography_policy import display_country
from app.core.enums import UserRole
from app.core.security import get_current_user
from app.db.session import get_db
from app.models.collected_data import CollectedData
from app.models.data_snapshot import DataSnapshot
from app.models.user import User
from app.schemas.compute_configuration import (
ComputeConfiguration,
ComputeConfigurationReset,
ComputeConfigurationWrite,
)
from app.services.compute_catalog import (
PARTICIPANTS,
participant_members,
source_country_names,
source_country_participants,
)
from app.services.compute_center_locations import refresh_compute_center_location_cache
from app.services.compute_configuration import (
DEFAULT_CONFIGURATION,
configuration_response,
publish_configuration,
read_configuration,
)
from app.services.compute_history import compute_history
from fastapi import APIRouter, Depends, HTTPException, Query, Response
from sqlalchemy import and_, case, func, or_, select
from sqlalchemy.ext.asyncio import AsyncSession
router = APIRouter()
SOURCES = ("top500", "epoch_ai_gpu")
def selected_countries(countries: str) -> list[str]:
codes = list(dict.fromkeys(code.strip().upper() for code in countries.split(",")))
if not codes or len(codes) > 12 or any(code not in PARTICIPANTS for code in codes):
raise HTTPException(422, "Select 1–12 participants from the compute catalog")
members = participant_members(codes)
flattened = [country for participant in members.values() for country in participant]
if len(flattened) != len(set(flattened)):
raise HTTPException(422, "Comparison participants cannot have overlapping country members")
return codes
@router.get("/series")
async def get_series(
response: Response,
countries: str = "CN,US",
mode: Literal["global_share", "bilateral_share", "absolute"] = "global_share",
db: AsyncSession = Depends(get_db),
):
codes = selected_countries(countries)
envelope = await read_configuration(db)
response.headers["Cache-Control"] = "no-store"
result = compute_history(codes, mode, ComputeConfiguration.model_validate(envelope["config"]))
result.update(
version=envelope["revision"],
provenance="bundled_reports" if envelope["is_default"] else "admin_curated_reports",
)
return result
async def configuration_admin(user: User = Depends(get_current_user)) -> User:
if user.role not in {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}:
raise HTTPException(403, "Administrator access required")
return user
@router.get("/configuration")
async def get_configuration(
response: Response,
_actor: User = Depends(configuration_admin),
db: AsyncSession = Depends(get_db),
):
response.headers["Cache-Control"] = "no-store"
return configuration_response(await read_configuration(db))
@router.put("/configuration")
async def update_configuration(
payload: ComputeConfigurationWrite,
actor: User = Depends(configuration_admin),
db: AsyncSession = Depends(get_db),
):
envelope = await publish_configuration(
db, revision=payload.revision, config=payload.config, actor=actor,
)
return configuration_response(envelope)
@router.post("/configuration/reset")
async def reset_configuration(
payload: ComputeConfigurationReset,
actor: User = Depends(configuration_admin),
db: AsyncSession = Depends(get_db),
):
envelope = await publish_configuration(
db, revision=payload.revision, config=DEFAULT_CONFIGURATION, actor=actor, reset=True,
)
return configuration_response(envelope)
async def sample_query(db: AsyncSession, countries: list[str], year: int | None):
now = datetime.now(UTC)
cutoff = min(datetime(year + 1, 1, 1, tzinfo=UTC), now) if year else now
ranked = (
select(
DataSnapshot.id,
func.row_number()
.over(
partition_by=DataSnapshot.source,
order_by=(DataSnapshot.completed_at.desc(), DataSnapshot.id.desc()),
)
.label("rank"),
)
.where(
DataSnapshot.source.in_(SOURCES),
DataSnapshot.status == "success",
DataSnapshot.completed_at < cutoff,
)
.subquery()
)
snapshots = select(ranked.c.id).where(ranked.c.rank == 1)
# Old collector rows have ambiguous units/identity and must not become
# national evidence merely because a new UI can display them.
query = select(CollectedData).where(
CollectedData.snapshot_id.in_(snapshots),
CollectedData.is_valid == 1,
CollectedData.extra_data["compute_contract"].as_string() == "v1",
or_(
CollectedData.source == "top500",
and_(
CollectedData.extra_data["status"].as_string() == "Existing",
func.coalesce(CollectedData.extra_data["superseded_by"].as_string(), "") == "",
CollectedData.extra_data["possible_duplicate"].as_boolean().is_not(True),
),
),
CollectedData.extra_data["country"].as_string().in_(source_country_names(countries)),
)
return query
@router.get("/facilities")
async def get_facilities(
countries: str = "CN,US",
year: int | None = Query(None, ge=2000, le=2100),
limit: int = Query(1000, ge=1, le=2000),
db: AsyncSession = Depends(get_db),
):
codes = selected_countries(countries)
query = await sample_query(db, codes, year)
country = CollectedData.extra_data["country"].as_string()
source_participants = source_country_participants(codes)
# Canonical source subdivisions retain their raw label on each record but
# share one owner in the comparison, using the same policy as the globe.
owner_cases = [
(country == name, display_country(name))
for name in source_country_names(codes)
if display_country(name) != name
]
display = case(*owner_cases, else_=country) if owner_cases else country
participant_cases = [(country == name, code) for name, code in source_participants.items()]
participant = case(*participant_cases, else_=None)
counts = (
(
await db.execute(
query.with_only_columns(
display.label("country"),
participant.label("participant_code"),
CollectedData.source,
func.count().label("count"),
).group_by(display, participant, CollectedData.source)
)
)
.mappings()
.all()
)
participant_counts = (
(
await db.execute(
query.with_only_columns(
participant.label("participant_code"),
CollectedData.source,
func.count().label("count"),
).group_by(participant, CollectedData.source)
)
)
.mappings()
.all()
)
records = (
(await db.execute(query.order_by(CollectedData.name, CollectedData.id).limit(limit)))
.scalars()
.all()
)
if year:
await refresh_compute_center_location_cache(db, as_of=datetime(year + 1, 1, 1, tzinfo=UTC))
else:
await refresh_compute_center_location_cache(db)
payload = convert_compute_centers_to_geojson(records)
for feature in payload["features"]:
feature["properties"]["historical_snapshot"] = year is not None
properties = feature["properties"]
properties["source_country"] = properties.get("country")
properties["country"] = display_country(properties.get("country"))
total = sum(row["count"] for row in counts)
payload.update(
{
"counts": [dict(row) for row in counts],
"participant_counts": [dict(row) for row in participant_counts],
"participants": [PARTICIPANTS[code] for code in codes],
"total_count": total,
"returned_count": len(records),
"truncated": total > limit,
"reference_year": year,
"scope": "public_source_records",
"history_basis": (
"snapshots_collected_by_year_end" if year else "latest_successful_snapshot"
),
"coverage": "available" if total else "no_verified_snapshot",
"cross_source_deduplicated": False,
}
)
return payload

View File

@@ -3,9 +3,9 @@ from datetime import UTC, datetime
import os
from pathlib import Path
from typing import Optional
from urllib.parse import urlsplit
from fastapi import APIRouter, Depends, HTTPException, Query, Request, status
import httpx
from pydantic import BaseModel, EmailStr, Field
from dotenv import dotenv_values
from sqlalchemy import select
@@ -65,6 +65,7 @@ from app.services.llm_provider_catalog import (
list_fallback_llm_provider_presets,
refresh_llm_provider_preset,
)
from app.services.llm_model_catalog import catalog_error_message, fetch_model_catalog
from app.services.scheduler import sync_datasource_job
from app.services.tv_streams import DEFAULT_TV_SETTINGS, get_tv_settings_payload, normalize_tv_settings
from app.services.persistent_logs import record_audit_log
@@ -75,6 +76,7 @@ logger = get_logger(__name__, service="api")
AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS = 5
AI_CONNECTION_TEST_PROMPT_KEY = "ai.connection_test"
SECRET_REVEAL_ROLES = {UserRole.ADMIN.value, UserRole.SUPER_ADMIN.value}
LLM_PROVIDER_PRESET_CATEGORY_PREFIX = "llm_provider_preset:"
DEFAULT_SETTINGS = {
"system": {
@@ -345,7 +347,13 @@ class ExternalIntegrationsUpdate(BaseModel):
def merge_with_defaults(category: str, payload: Optional[dict]) -> dict:
merged = deepcopy(DEFAULT_SETTINGS[category])
if category.startswith(LLM_PROVIDER_PRESET_CATEGORY_PREFIX):
defaults = get_fallback_llm_provider_preset(
category.removeprefix(LLM_PROVIDER_PRESET_CATEGORY_PREFIX)
)
else:
defaults = DEFAULT_SETTINGS[category]
merged = deepcopy(defaults)
if payload:
merged.update(payload)
return merged
@@ -382,6 +390,7 @@ async def get_setting_payload(db: AsyncSession, category: str) -> dict:
async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -> dict:
merged = merge_with_defaults(category, payload)
record = await get_setting_record(db, category)
if record is None:
record = SystemSetting(category=category, payload=payload)
@@ -391,7 +400,7 @@ async def save_setting_payload(db: AsyncSession, category: str, payload: dict) -
await db.commit()
await db.refresh(record)
return merge_with_defaults(category, record.payload)
return merged
AI_PROVIDER_ENV_FILE = Path(__file__).resolve().parents[4] / "aiprovider" / ".env"
@@ -655,6 +664,12 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
normalized_ai["providers"].get(default_provider) or _provider_defaults(default_provider)
)
api_key, _api_key_source = _resolve_provider_api_key(default_provider, provider_config)
model_provider_apis = {
**(provider_config.get("model_provider_apis") or {}),
**(_get_provider_preset(default_provider).get("model_provider_apis") or {}),
}
if default_provider == "openai" and urlsplit(provider_config.get("base_url") or "").hostname != "api.openai.com":
model_provider_apis = {}
return {
"service_url": normalized_ai.get("service_url") or app_settings.AI_PROVIDER_SERVICE_URL,
"service_token": _resolve_service_token(normalized_ai)[0],
@@ -672,8 +687,7 @@ def _runtime_config_from_ai_payload(ai_payload: dict) -> dict:
"api_key": api_key,
"max_tokens": int(provider_config.get("max_tokens") or 1200),
"anthropic_version": provider_config.get("anthropic_version") or "2023-06-01",
"model_provider_apis": provider_config.get("model_provider_apis") or {},
"preset_models": _get_provider_preset(default_provider).get("models") or [],
"model_provider_apis": model_provider_apis,
},
}
@@ -753,159 +767,35 @@ async def _validate_ai_provider_full_connection(ai_payload: dict) -> dict:
}
def _join_provider_url(base_url: str, path: str) -> str:
return f"{base_url.rstrip('/')}/{path.lstrip('/')}"
def _extract_model_ids(payload: dict) -> list[str]:
data = payload.get("data") if isinstance(payload, dict) else None
if isinstance(data, list):
return [
str(item.get("id"))
for item in data
if isinstance(item, dict) and item.get("id")
]
models = payload.get("models") if isinstance(payload, dict) else None
if isinstance(models, list):
return [
str(item.get("name") or item.get("model") or item.get("id") or item)
for item in models
if item
]
return []
def _contains_model(model_ids: list[str], model: str) -> bool:
normalized_model = model.strip().lower()
return any(str(item).strip().lower() == normalized_model for item in model_ids)
async def _check_ai_provider_lightweight(llm_config: dict, timeout_seconds: int) -> dict:
provider = _normalize_provider_id(llm_config.get("provider") or "")
configured_api = (
str(llm_config.get("provider_api") or "").strip()
or ProviderApi.OPENAI_COMPLETIONS.value
)
provider_api = str(llm_config.get("provider_api") or ProviderApi.OPENAI_COMPLETIONS.value)
model = str(llm_config.get("model") or "").strip()
base_url = str(llm_config.get("base_url") or "").strip().rstrip("/")
base_url = str(llm_config.get("base_url") or "").strip()
api_key = str(llm_config.get("api_key") or "").strip()
provider_api = configured_api
model_provider_apis = llm_config.get("model_provider_apis")
if isinstance(model_provider_apis, dict):
provider_api = str(model_provider_apis.get(model) or provider_api)
preset_models = [
str(item)
for item in (llm_config.get("preset_models") or [])
if str(item).strip()
]
if not provider or not base_url or not model:
return {
"success": False,
"connected": False,
"message": "当前 provider/base_url/model 未完整配置。",
"mode": "lightweight_config",
}
result = {
"success": False, "connected": False, "mode": "lightweight_models",
"provider": provider, "model": model,
}
if not base_url or not model:
return {**result, "message": "当前 provider/base_url/model 未完整配置。"}
if provider_api != ProviderApi.OLLAMA_GENERATE.value and not api_key:
return {
"success": False,
"connected": False,
"message": "当前 provider 未配置 API Key。",
"mode": "lightweight_config",
}
if provider == "opencode-go":
url = _join_provider_url(base_url, "/models")
headers = {"Authorization": f"Bearer {api_key}"}
elif provider_api == ProviderApi.OLLAMA_GENERATE.value:
url = _join_provider_url(base_url, "/api/tags")
headers: dict[str, str] = {}
elif provider_api == ProviderApi.OPENAI_COMPLETIONS.value:
url = _join_provider_url(base_url, "/models")
headers = {"Authorization": f"Bearer {api_key}"}
elif provider_api == ProviderApi.ANTHROPIC_MESSAGES.value:
url = _join_provider_url(base_url, "/models")
headers = {
"x-api-key": api_key,
"anthropic-version": str(llm_config.get("anthropic_version") or "2023-06-01"),
}
else:
return {
"success": False,
"connected": False,
"message": f"当前 provider_api 不支持轻量连通性测试: {provider_api}",
"mode": "lightweight_unsupported",
}
return {**result, "message": "当前 provider 未配置 API Key。"}
try:
async with httpx.AsyncClient(timeout=min(timeout_seconds, AI_PROVIDER_QUICK_CONNECT_TIMEOUT_SECONDS)) as client:
response = await client.get(url, headers=headers)
response.raise_for_status()
payload = response.json()
except httpx.HTTPStatusError as exc:
detail = exc.response.text or exc.response.reason_phrase
if exc.response.status_code == 404 and _contains_model(preset_models, model):
return {
"success": True,
"connected": True,
"message": "轻量连通性测试通过;当前 provider 不提供可用的模型目录,已按内置模型预设确认。",
"mode": "lightweight_preset",
"provider": provider,
"provider_api": provider_api,
"model": model,
"url": url,
}
return {
"success": False,
"connected": False,
"message": f"轻量连通性测试失败: HTTP {exc.response.status_code} {detail}",
"mode": "lightweight_models",
"url": url,
}
catalog = await fetch_model_catalog(
provider, base_url, provider_api, api_key,
str(llm_config.get("anthropic_version") or "2023-06-01"), timeout_seconds,
)
except Exception as exc:
return {
"success": False,
"connected": False,
"message": f"轻量连通性测试失败: {exc}",
"mode": "lightweight_models",
"url": url,
}
model_ids = _extract_model_ids(payload)
if model_ids and not _contains_model(model_ids, model):
if _contains_model(preset_models, model):
return {
"success": True,
"connected": True,
"message": "轻量连通性测试通过;provider 模型目录未返回当前别名,已按内置模型预设确认。",
"mode": "lightweight_models_with_preset_alias",
"provider": provider,
"provider_api": provider_api,
"model": model,
"models_count": len(model_ids),
"url": url,
}
return {
"success": False,
"connected": False,
"message": f"连接可用,但模型目录中没有当前模型: {model}",
"mode": "lightweight_models",
"provider": provider,
"model": model,
"models_count": len(model_ids),
"url": url,
}
return {**result, "message": catalog_error_message(exc)}
result.update({"url": catalog.url, "models_count": len(catalog.models)})
if model.casefold() not in {item.casefold() for item in catalog.models}:
return {**result, "message": f"模型目录查询成功,但当前模型不在目录中:{model}"}
model_apis = llm_config.get("model_provider_apis") or {}
return {
"success": True,
"connected": True,
"message": "轻量连通性测试通过",
"mode": "lightweight_models",
"provider": provider,
"provider_api": provider_api,
"model": model,
"models_count": len(model_ids),
"url": url,
**result, "success": True, "connected": True,
"provider_api": model_apis.get(model) or provider_api,
"message": "模型目录查询成功,当前模型已找到;尚未执行生成调用。",
}
@@ -1689,7 +1579,8 @@ async def connect_ai_provider_integration(
)
await emit_business_log(
logger,
event="settings.ai_provider.connect.success",
event=("settings.ai_provider.connect.success" if lightweight_result["success"]
else "settings.ai_provider.connect.failed"),
message="AI provider connection test completed",
category="ai",
service="api",
@@ -1699,7 +1590,7 @@ async def connect_ai_provider_integration(
"provider": payload.provider,
"model": payload.model,
"configured": True,
"lightweight_status": lightweight_result.get("status"),
"connected": lightweight_result["connected"],
},
)
return {
@@ -2044,8 +1935,18 @@ async def reset_provider_credential_guide(
@router.get("/integrations/ai-provider/presets")
async def get_ai_provider_presets(
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
return {"data": list_fallback_llm_provider_presets()}
presets = list_fallback_llm_provider_presets()
saved = await get_setting_payloads(
db, [f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}" for preset in presets]
)
return {
"data": [
{**preset, **saved[f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{preset['provider']}"]}
for preset in presets
]
}
@router.post("/integrations/ai-provider/presets/{provider}/refresh")
@@ -2053,22 +1954,52 @@ async def refresh_ai_provider_preset(
provider: str,
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
payload: AIProviderIntegrationUpdate | None = None,
):
try:
provider_id = _normalize_provider_id(provider)
api_key = None
if provider_id == "opencode-go":
current_payload = await get_setting_payload(db, "external_integrations")
ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
return {"data": await refresh_llm_provider_preset(provider_id, api_key=api_key)}
get_fallback_llm_provider_preset(provider_id)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
current_payload = await get_setting_payload(db, "external_integrations")
if payload is not None:
if _normalize_provider_id(payload.provider) != provider_id:
raise HTTPException(status_code=400, detail="刷新供应商与表单供应商不一致。")
ai_payload = _build_ai_provider_payload(current_payload, payload)
else:
ai_payload = _normalize_ai_provider_payload(current_payload.get("ai_provider") or {})
provider_config = ai_payload["providers"].get(provider_id) or _provider_defaults(provider_id)
api_key, _api_key_source = _resolve_provider_api_key(provider_id, provider_config)
if not api_key and provider_id not in {"ollama", "opencode-go", "openrouter"}:
raise HTTPException(
status_code=400, detail="请先配置当前供应商的 API Key,再刷新模型列表。",
)
try:
refreshed = await refresh_llm_provider_preset(
provider_id, api_key=api_key, base_url=provider_config.get("base_url"),
provider_api=provider_config.get("provider_api"),
anthropic_version=provider_config.get("anthropic_version") or "2023-06-01",
)
except Exception as exc:
fallback = get_fallback_llm_provider_preset(provider)
fallback["refresh_error"] = str(exc)
return {"data": fallback}
logger.warning(
"LLM provider catalog refresh failed",
extra={
"event": "settings.ai_provider.catalog.failed",
"context": {
"provider": provider_id,
"error_type": type(exc).__name__,
},
},
)
raise HTTPException(
status_code=502,
detail=f"{catalog_error_message(exc)} 已保留上次模型列表。",
) from exc
refreshed["refreshed_at"] = to_iso8601_utc(datetime.now(UTC))
await save_setting_payload(db, f"{LLM_PROVIDER_PRESET_CATEGORY_PREFIX}{provider_id}", refreshed)
return {"data": refreshed}
@router.put("/integrations")

View File

@@ -7,6 +7,7 @@ from fastapi.responses import Response
from sqlalchemy.ext.asyncio import AsyncSession
from app.db.session import get_db
from app.services.tv_catalog import get_tv_catalog_page
from app.services.tv_streams import get_public_tv_payload, is_allowed_tv_proxy_url
router = APIRouter()
@@ -34,9 +35,13 @@ def _should_strip_hls_metadata_line(line: str) -> bool:
@router.get("/streams")
async def list_public_tv_streams(
offset: int = Query(0, ge=0),
limit: int = Query(50, ge=1, le=100),
q: str = Query("", max_length=200),
selected_id: str | None = Query(None, max_length=200),
db: AsyncSession = Depends(get_db),
):
return await get_public_tv_payload(db)
return await get_tv_catalog_page(db, offset=offset, limit=limit, q=q, selected_id=selected_id)
@router.get("/proxy")

View File

@@ -644,7 +644,9 @@ def _normalize_capacity_band(capacity_value: Optional[float], capacity_unit: str
return "unknown"
unit = str(capacity_unit or "").strip().lower()
if unit in {"pflop/s", "pflops", "pflop"}:
if unit in {"flop/s", "flops"}:
normalized_tflops = capacity_value / 1e12
elif unit in {"pflop/s", "pflops", "pflop"}:
normalized_tflops = capacity_value * 1000
elif unit in {"gflop/s", "gflops", "gflop"}:
normalized_tflops = capacity_value
@@ -672,9 +674,13 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
"""
features: List[Dict[str, Any]] = []
unresolved: List[Dict[str, Any]] = []
withheld_count = 0
for record in records:
metadata = record.extra_data or {}
if metadata.get("compute_contract") == "v1" and metadata.get("anonymous"):
withheld_count += 1
continue
result = resolve_compute_center_location_full(record, metadata)
site_type = (
"supercomputer"
@@ -706,7 +712,10 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
latitude = location.latitude
longitude = location.longitude
if site_type == "supercomputer":
if metadata.get("compute_contract") == "v1":
capacity_value = _parse_float(get_record_field(record, "value"))
capacity_unit = str(get_record_field(record, "unit") or "FLOP/s")
elif site_type == "supercomputer":
capacity_value = _parse_float(get_record_field(record, "rmax"))
capacity_unit = "GFlops"
else:
@@ -767,7 +776,7 @@ def convert_compute_centers_to_geojson(records: List[CollectedData]) -> Dict[str
}
)
return {"type": "FeatureCollection", "features": features, "unresolved": unresolved}
return {"type": "FeatureCollection", "features": features, "unresolved": unresolved, "withheld_count": withheld_count}
VESSEL_TYPE_FILTERS = {
@@ -1121,6 +1130,7 @@ async def build_vessel_snapshot_response(
safe_limit = _safe_vessel_limit(limit)
safe_since_minutes = min(max(int(since_minutes or 60), 1), 1440)
observed_since = datetime.now(UTC) - timedelta(minutes=safe_since_minutes)
snapshot_started_at = datetime.now(UTC)
features, diagnostics = await _load_raw_vessel_snapshot_features(
db,
bbox=bbox,
@@ -1137,6 +1147,7 @@ async def build_vessel_snapshot_response(
"features": features,
"count": len(features),
"stats": _build_vessel_stats(features),
"generated_at": to_iso8601_utc(snapshot_started_at),
"diagnostics": {
**diagnostics,
"filtered_count": len(features),

View File

@@ -161,7 +161,7 @@ async def websocket_endpoint(
if is_anonymous:
channels = [channel for channel in channels if channel in supported_channels]
vessel_subscription = None
if "vessels" in channels and "bbox" in payload_data:
if "vessels" in channels:
try:
vessel_subscription = manager.subscribe_vessels(websocket, payload_data)
except ValueError as exc:

View File

@@ -0,0 +1,57 @@
"""Country display ownership for Earth comparisons and cards.
Keep source-country evidence separate; use this projection only for display
grouping so regional source records cannot become competing national keys.
"""
from __future__ import annotations
import json
from functools import lru_cache
from pathlib import Path
from app.core.countries import COUNTRY_ENTRIES, normalize_country
_DISPLAY_ISO_A3 = {
canonical: next(
(alias for alias in aliases if len(alias) == 3 and alias.isascii() and alias.isupper()),
None,
)
for canonical, aliases in COUNTRY_ENTRIES
}
_COUNTRY_BY_ISO_A3 = {code: country for country, code in _DISPLAY_ISO_A3.items() if code}
_POLICY_PATH = Path(__file__).resolve().parents[3] / "config/earth-boundary-pov-policy.china-v1.json"
@lru_cache(maxsize=1)
def _policy_country_by_code() -> dict[str, str]:
try:
content = _POLICY_PATH.read_text(encoding="utf-8")
except FileNotFoundError as error:
raise FileNotFoundError(f"EARTH_DISPLAY_POLICY_MISSING: {_POLICY_PATH}") from error
policy = json.loads(content)
if policy.get("profile") != "china-pov-v1":
raise ValueError("unsupported Earth display-policy profile")
mappings = {}
for rule in policy["rules"]:
target = rule.get("targetIsoA3") or rule.get("parentIsoA3")
if target not in _COUNTRY_BY_ISO_A3:
continue
for code in rule.get("sourceIsoA3", []):
mappings[code] = _COUNTRY_BY_ISO_A3[target]
return mappings
def display_country(value: str | None) -> str | None:
country = normalize_country(value)
code = _DISPLAY_ISO_A3.get(country) if country else str(value or "").strip().upper()
owner = _policy_country_by_code().get(code)
if owner:
return owner
return country
def display_iso_a3(value: str | None) -> str | None:
country = display_country(value)
return _DISPLAY_ISO_A3.get(country) if country else None

View File

@@ -138,6 +138,7 @@ class DatasourceRunStatus(StrEnum):
class ProviderApi(StrEnum):
ANTHROPIC_MESSAGES = "anthropic-messages"
OPENAI_COMPLETIONS = "openai-completions"
OPENAI_RESPONSES = "openai-responses"
OLLAMA_GENERATE = "ollama-generate"

View File

@@ -6,7 +6,7 @@ import redis
from fastapi import Depends, HTTPException, status
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from jose import JWTError, jwt
from sqlalchemy import text
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.config import settings
@@ -81,6 +81,21 @@ def decode_token(token: str) -> Optional[dict]:
return None
async def _load_active_user(user_id: str, db: AsyncSession) -> User:
try:
parsed_id = int(user_id)
except (TypeError, ValueError):
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="Invalid token")
result = await db.execute(select(User).where(User.id == parsed_id))
user = result.scalar_one_or_none()
if user is None or not user.is_active:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="User not found or inactive",
)
return user
async def get_current_user(
credentials: HTTPAuthorizationCredentials = Depends(oauth2_scheme),
db: AsyncSession = Depends(get_db),
@@ -103,27 +118,7 @@ async def get_current_user(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token",
)
result = await db.execute(
text(
"SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
),
{"id": int(user_id)},
)
row = result.fetchone()
if row is None or not row[5]:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="User not found or inactive",
)
user = User()
user.id = row[0]
user.username = row[1]
user.email = row[2]
user.password_hash = row[3]
user.role = row[4]
user.is_active = row[5]
user.gatekeeper_groups = row[6] or []
return user
return await _load_active_user(user_id, db)
async def get_current_user_refresh(
@@ -143,27 +138,7 @@ async def get_current_user_refresh(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid token",
)
result = await db.execute(
text(
"SELECT id, username, email, password_hash, role, is_active, gatekeeper_groups FROM users WHERE id = :id"
),
{"id": int(user_id)},
)
row = result.fetchone()
if row is None or not row[5]:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="User not found or inactive",
)
user = User()
user.id = row[0]
user.username = row[1]
user.email = row[2]
user.password_hash = row[3]
user.role = row[4]
user.is_active = row[5]
user.gatekeeper_groups = row[6] or []
return user
return await _load_active_user(user_id, db)
def blacklist_token(token: str) -> None:

View File

@@ -4,11 +4,14 @@ import asyncio
from datetime import UTC, datetime
from typing import Dict, Any
from app.core.logging import get_logger
from app.core.time import to_iso8601_utc
from app.core.websocket.manager import manager
EARTH_UPDATES_CHANNEL = "earth_updates"
VESSEL_STATE_QUERY_BATCH_SIZE = 1000
logger = get_logger(__name__, service="websocket")
class DataBroadcaster:
@@ -114,16 +117,22 @@ class DataBroadcaster:
return
pending = self._pending_vessel_updates
self._pending_vessel_updates = {}
vessels = []
for item in pending.values():
vessel = dict(item)
source = vessel.pop("_source", None)
action = vessel.pop("_action", "upsert")
created = vessel.pop("_created", None)
vessel["source"] = source
vessel["action"] = action
vessel["created"] = created
vessels.append(vessel)
try:
vessels = await self._load_current_vessel_updates(list(pending))
except Exception:
# Preserve newer updates that arrived during the failed database read.
self._pending_vessel_updates = {**pending, **self._pending_vessel_updates}
raise
if not vessels:
return
vessels = [
{
**vessel,
"action": vessel.get("action", "upsert"),
"created": pending.get(str(vessel["mmsi"]), {}).get("_created"),
}
for vessel in vessels
]
await manager.broadcast_vessels(
{
"action": "upsert",
@@ -133,12 +142,31 @@ class DataBroadcaster:
}
)
async def _load_current_vessel_updates(self, keys: list[str]) -> list[Dict[str, Any]]:
from app.db.session import async_session_factory
from app.services.vessel_ais_aggregation import get_current_vessels_by_mmsi
mmsis = [int(key) for key in keys if key.isdigit()]
vessels = []
async with async_session_factory() as db:
for offset in range(0, len(mmsis), VESSEL_STATE_QUERY_BATCH_SIZE):
vessels.extend(await get_current_vessels_by_mmsi(
db, mmsis[offset:offset + VESSEL_STATE_QUERY_BATCH_SIZE]
))
present = {int(vessel["mmsi"]) for vessel in vessels}
vessels.extend({"mmsi": mmsi, "action": "remove"} for mmsi in mmsis if mmsi not in present)
return vessels
async def broadcast_vessels_periodically(self):
while self.running:
try:
await self.flush_vessel_updates()
except Exception:
pass
except Exception as exc:
logger.exception_event(
"Failed to flush vessel updates",
event="vessels.broadcast.failed",
context={"error": str(exc)},
)
await asyncio.sleep(self._vessel_flush_interval)
async def broadcast_datasource_task_update(self, data: Dict[str, Any]):

View File

@@ -63,6 +63,8 @@ class ConnectionManager:
def unsubscribe(self, websocket: WebSocket, channels: list[str]):
for channel in {str(channel).strip() for channel in channels if str(channel).strip()}:
if channel == "vessels":
self.vessel_subscriptions.pop(websocket, None)
subscribers = self.channel_subscriptions.get(channel)
if subscribers is not None:
subscribers.discard(websocket)
@@ -88,7 +90,8 @@ class ConnectionManager:
return subscription
def _normalize_vessel_subscription(self, config: dict[str, Any]) -> dict[str, Any]:
bbox = config.get("bbox")
global_scope = config.get("scope") == "global"
bbox = [-180, -90, 180, 90] if global_scope else config.get("bbox")
if not isinstance(bbox, (list, tuple)) or len(bbox) != 4:
raise ValueError("vessels subscription requires bbox=[lon_min,lat_min,lon_max,lat_max]")
try:
@@ -103,7 +106,7 @@ class ConnectionManager:
raise ValueError("bbox longitude values must be between -180 and 180")
if not (-90 <= lat_min <= 90 and -90 <= lat_max <= 90):
raise ValueError("bbox latitude values must be between -90 and 90")
if (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
if not global_scope and (lon_max - lon_min) * (lat_max - lat_min) > MAX_VESSEL_BBOX_AREA:
raise ValueError("bbox is too large; zoom in or request a smaller viewport")
zoom = int(config.get("zoom") or 1)
@@ -116,10 +119,11 @@ class ConnectionManager:
if str(item).strip()
}
return {
"scope": "global" if global_scope else "viewport",
"bbox": (lon_min, lat_min, lon_max, lat_max),
"zoom": zoom,
"limit": limit,
"type": vessel_types,
"type": sorted(vessel_types),
"last_sent_at": None,
}
@@ -152,7 +156,9 @@ class ConnectionManager:
vessel
for vessel in vessels
if self._vessel_matches_subscription(vessel, subscription)
][: min(subscription["limit"], MAX_VESSEL_WS_MESSAGE_ITEMS)]
]
if subscription.get("scope") != "global":
matched = matched[:subscription["limit"]]
if not matched:
continue
subscription["last_sent_at"] = datetime.now(UTC)
@@ -162,16 +168,24 @@ class ConnectionManager:
"timestamp": subscription["last_sent_at"].isoformat(),
"payload": {
**data,
"vessels": matched,
"vessels": [],
"subscription": {
"bbox": list(subscription["bbox"]),
"zoom": subscription["zoom"],
"limit": subscription["limit"],
"scope": subscription.get("scope", "viewport"),
},
},
}
try:
await connection.send_json(message)
for offset in range(0, len(matched), MAX_VESSEL_WS_MESSAGE_ITEMS):
await connection.send_json({
**message,
"payload": {
**message["payload"],
"vessels": matched[offset:offset + MAX_VESSEL_WS_MESSAGE_ITEMS],
},
})
except Exception:
self.unsubscribe_all(connection)
@@ -180,6 +194,8 @@ class ConnectionManager:
vessel: dict[str, Any],
subscription: dict[str, Any],
) -> bool:
if vessel.get("action") == "remove" and subscription.get("scope") == "global":
return True
try:
lon = float(vessel.get("lon"))
lat = float(vessel.get("lat"))

View File

@@ -0,0 +1,94 @@
"""Validated, source-backed observations for the compute comparison cards."""
from datetime import date
from typing import Annotated, Literal, Self
from app.services.compute_catalog import COUNTRY_PARTICIPANTS
from pydantic import BaseModel, ConfigDict, Field, HttpUrl, field_validator, model_validator
Year = Annotated[int, Field(strict=True, ge=2000, le=2100)]
Value = Annotated[float, Field(strict=True, allow_inf_nan=False, ge=0, le=1e12)]
Identifier = Annotated[str, Field(min_length=1, max_length=80, pattern=r"^[a-zA-Z0-9_-]+$")]
Text = Annotated[str, Field(min_length=1, max_length=500)]
class ConfigurationModel(BaseModel):
model_config = ConfigDict(extra="forbid", str_strip_whitespace=True)
class ComputeSource(ConfigurationModel):
id: Identifier
title: Text
url: HttpUrl
publisher: Text
report_year: Year
published_at: date | None = None
locator: Text
@field_validator("url")
@classmethod
def public_reference(cls, value: HttpUrl) -> HttpUrl:
if value.username or value.password or len(str(value)) > 2000:
raise ValueError("Use a report URL without credentials (maximum 2000 characters)")
return value
class ComputeObservation(ConfigurationModel):
year: Year
metric: Literal["global_share", "absolute"]
source_id: Identifier
values: dict[str, Value] = Field(min_length=1, max_length=250)
@model_validator(mode="after")
def comparable_values(self) -> Self:
if any(code not in COUNTRY_PARTICIPANTS for code in self.values):
raise ValueError(
"Use countries from the canonical compute catalog, not regional totals"
)
if self.metric == "global_share" and sum(self.values.values()) > 100.000001:
raise ValueError("Global shares for one year must total at most 100 percent")
return self
class InnovationObservation(ConfigurationModel):
year: Year
value: Annotated[float, Field(strict=True, allow_inf_nan=False, ge=-100, le=100)]
source_id: Identifier
class ComputeConfiguration(ConfigurationModel):
reviewed_at: date
sources: list[ComputeSource] = Field(max_length=200)
observations: list[ComputeObservation] = Field(max_length=202)
innovation: list[InnovationObservation] = Field(max_length=101)
@model_validator(mode="after")
def consistent_evidence(self) -> Self:
ids = [source.id for source in self.sources]
if len(ids) != len(set(ids)):
raise ValueError("Report source IDs must be unique")
keys = [(row.metric, row.year) for row in self.observations]
if len(keys) != len(set(keys)):
raise ValueError("Each metric/year may have only one observation")
years = [row.year for row in self.innovation]
if len(years) != len(set(years)):
raise ValueError("Innovation observation years must be unique")
sources = {source.id: source for source in self.sources}
for row in [*self.observations, *self.innovation]:
source = sources.get(row.source_id)
if source is None:
raise ValueError(f"Missing report source: {row.source_id}")
if row.year > source.report_year:
raise ValueError("Observation year must not be after its report year")
if self.reviewed_at > date.today():
raise ValueError("Review date must not be in the future")
return self
class ComputeConfigurationWrite(ConfigurationModel):
revision: str = Field(min_length=1, max_length=80)
config: ComputeConfiguration
class ComputeConfigurationReset(ConfigurationModel):
revision: str = Field(min_length=1, max_length=80)

View File

@@ -36,6 +36,9 @@ class UserInDB(UserBase):
class UserResponse(UserBase):
# Bootstrap accounts have reserved .local email addresses. Validate email
# on writes, while allowing existing local identities to be serialized.
email: str
id: int
role: str
gatekeeper_groups: list[str] = Field(default_factory=list)

View File

@@ -31,6 +31,7 @@ class BaseCollector(ABC):
frequency_hours: int = 4
data_type: str = "generic"
fail_on_empty: bool = False
preserve_snapshot_on_failure: bool = False
def __init__(self):
self._current_task = None
@@ -361,6 +362,8 @@ class BaseCollector(ABC):
context={"task_id": task_id, "raw_count": len(raw_data)},
)
data = self.transform(raw_data)
if self.fail_on_empty and not data:
raise RuntimeError(f"Collector {self.name} produced no valid records")
await self._log_collection_event(
"collector.phase.transforming.success",
"Collector transform phase completed",
@@ -456,6 +459,10 @@ class BaseCollector(ABC):
raise
except Exception as e:
await db.rollback()
if snapshot_id is not None and self.preserve_snapshot_on_failure:
await self._rollback_incomplete_run(
db, task_id=task_id, snapshot_id=snapshot_id, reason=str(e)
)
task.status = JobStatus.FAILED.value
task.phase = "failed"
task.phase_message = str(e)
@@ -465,6 +472,8 @@ class BaseCollector(ABC):
snapshot = await db.get(DataSnapshot, snapshot_id)
if snapshot:
snapshot.status = SnapshotStatus.FAILED.value
if self.preserve_snapshot_on_failure:
snapshot.is_current = False
snapshot.completed_at = datetime.now(UTC)
snapshot.summary = {"error": str(e)}
await db.commit()

View File

@@ -0,0 +1,82 @@
"""Quality gates shared by the two compute sources."""
from datetime import datetime
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.countries import normalize_country
from app.models.compute_center_location import ComputeCenterLocationRecord
from app.models.data_snapshot import DataSnapshot
from app.services.collectors.base import BaseCollector
class ComputeCollector(BaseCollector):
fail_on_empty = True
preserve_snapshot_on_failure = True
async def _create_snapshot(
self, db: AsyncSession, task_id: int, data: list[dict], started_at: datetime
) -> int:
previous_count = await db.scalar(
select(DataSnapshot.record_count)
.where(
DataSnapshot.source == self.name,
DataSnapshot.is_current.is_(True),
DataSnapshot.status == "success",
)
.order_by(DataSnapshot.id.desc())
.limit(1)
)
if previous_count and len(data) < previous_count * 0.6:
raise ValueError("Compute source lost over 40% of rows; previous snapshot retained")
return await super()._create_snapshot(db, task_id, data, started_at)
async def _save_data(self, db: AsyncSession, data: list[dict], **kwargs) -> int:
if self.name == "top500":
await self._copy_verified_locations(db, data)
return await super()._save_data(db, data, **kwargs)
async def _copy_verified_locations(self, db: AsyncSession, data: list[dict]) -> None:
"""Keep historical location keys; copy only unambiguous exact identities.
Never migrate by rank: the machine occupying rank 1 changes over time.
"""
names = [item["name"] for item in data]
locations = (
(
await db.execute(
select(ComputeCenterLocationRecord).where(
ComputeCenterLocationRecord.source == self.name,
ComputeCenterLocationRecord.name.in_(names),
)
)
)
.scalars()
.all()
)
existing = {location.source_id for location in locations}
for item in data:
if item["source_id"] in existing:
continue
country = normalize_country(item["country"])
if not country:
continue
matches = [
location
for location in locations
if location.name == item["name"]
and normalize_country(location.country) == country
and (not location.site or location.site == item.get("metadata", {}).get("site"))
and not location.source_id.startswith("top500_system_")
]
if len(matches) != 1:
continue
old = matches[0]
values = {
column.name: getattr(old, column.name)
for column in old.__table__.columns
if column.name not in {"id", "source_id", "created_at", "updated_at"}
}
db.add(ComputeCenterLocationRecord(**values, source_id=item["source_id"]))
existing.add(item["source_id"])

View File

@@ -1,119 +1,135 @@
"""Epoch AI GPU Clusters Collector
Collects data from Epoch AI GPU clusters tracking.
https://epoch.ai/data/gpu-clusters
"""
"""Epoch's official CSV; preserve precision, source dates and anonymous records."""
import csv
import hashlib
import io
import json
import math
import re
from typing import Dict, Any, List
from datetime import UTC, datetime
from bs4 import BeautifulSoup
from urllib.parse import urlparse
import httpx
from app.services.collectors.base import BaseCollector
from app.core.countries import normalize_country
from app.services.collectors.compute_base import ComputeCollector
from app.services.epoch_location import assess_epoch_location
EPOCH_CSV_URL = "https://epoch.ai/data/gpu_clusters.csv"
class EpochAIGPUCollector(BaseCollector):
class EpochAIGPUCollector(ComputeCollector):
name = "epoch_ai_gpu"
priority = "P0"
module = "L1"
frequency_hours = 6
data_type = "gpu_cluster"
async def fetch(self) -> List[Dict[str, Any]]:
"""Fetch Epoch AI GPU clusters data from webpage"""
url = self._resolved_url or ""
async with httpx.AsyncClient(timeout=60.0) as client:
async def fetch(self) -> list[dict]:
configured = self._resolved_url or EPOCH_CSV_URL
parsed = urlparse(configured)
url = (
EPOCH_CSV_URL
if parsed.hostname == "epoch.ai" and not parsed.path.endswith(".csv")
else configured
)
async with httpx.AsyncClient(timeout=60, follow_redirects=True) as client:
response = await client.get(url)
response.raise_for_status()
return self.parse_response(response.text)
return self.parse_response(response.text)
def parse_response(self, html: str) -> List[Dict[str, Any]]:
"""Parse Epoch AI webpage to extract GPU cluster data"""
data = []
soup = BeautifulSoup(html, "html.parser")
def parse_response(self, text: str) -> list[dict]:
reader = csv.DictReader(io.StringIO(text.lstrip("\ufeff")))
required = {"Name", "Country", "Status", "First Operational Date", "16-bit OP/s (log)"}
if not required.issubset(reader.fieldnames or []):
raise ValueError(
"Epoch CSV schema changed or HTML returned; previous snapshot retained"
)
entries = []
seen = set()
anonymous_counts = {}
for row in reader:
name = (row.get("Name") or "").strip()
country = normalize_country(row.get("Country"))
if not name:
raise ValueError("Epoch row has missing identity")
source_id = f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', name.lower())}"
anonymous = "anonym" in name.lower()
if anonymous:
# Public anonymous rows have no facility identity. Preserve
# multiplicity within snapshots, without guessing names.
digest = hashlib.sha256(json.dumps(row, sort_keys=True).encode()).hexdigest()[:24]
anonymous_counts[digest] = anonymous_counts.get(digest, 0) + 1
source_id = f"epoch_anonymous_{digest}_{anonymous_counts[digest]}"
if source_id in seen:
raise ValueError("Epoch identity collision; previous snapshot retained")
seen.add(source_id)
raw_date = row.get("First Operational Date", "")
reference_date = raw_date if re.fullmatch(r"\d{4}-\d{2}-\d{2}", raw_date) else None
value = None
raw_value = row.get("16-bit OP/s (log)", "")
if raw_value:
exponent = float(raw_value)
if not math.isfinite(exponent) or not 0 <= exponent <= 30:
raise ValueError("Epoch performance exponent out of bounds")
value = 10**exponent
anonymous = anonymous or (country == "中国" and not row.get("Location"))
location = row.get("Location")
location_assessment = assess_epoch_location(
location,
country=country,
raw_country=row.get("Country"),
)
entries.append(
{
"source_id": source_id,
"name": name,
"country": country,
"value": value,
"unit": "FLOP/s",
"reference_date": reference_date,
"latitude": None if anonymous else self.coordinate(row.get("latitude"), 90),
"longitude": None if anonymous else self.coordinate(row.get("longitude"), 180),
"metadata": {
"compute_contract": "v1",
"precision": "FP16",
"measurement": "theoretical",
"source_url": EPOCH_CSV_URL,
"raw_reference_date": raw_date,
"status": row.get("Status"),
"certainty": row.get("Certainty"),
"operator": row.get("Owner"),
"gpu_type": row.get("Chip type (primary)"),
"gpu_count": row.get("Chip quantity (primary)"),
"location": location,
"location_precision": location_assessment.precision,
"location_precision_reason": location_assessment.reason,
"raw_country": row.get("Country"),
"raw_performance": raw_value,
"performance_encoding": "log10",
"builds_upon": row.get("Builds Upon"),
"superseded_by": row.get("Superseded by"),
"possible_duplicate": row.get("Possible Duplicate", "").lower() == "true",
"include_in_standard_analysis": row.get(
"Include in Standard Analysis", ""
).lower()
== "true",
"rounded": country == "中国",
"anonymous": anonymous,
"identity_scope": "snapshot_record" if anonymous else "named_cluster",
"country_unresolved": country is None,
"sources": [
row[f"Source {i}"] for i in range(1, 6) if row.get(f"Source {i}")
],
},
}
)
return entries
# Try to find data table on the page
tables = soup.find_all("table")
for table in tables:
rows = table.find_all("tr")
for row in rows[1:]: # Skip header
cells = row.find_all(["td", "th"])
if len(cells) >= 5:
try:
cluster_name = cells[0].get_text(strip=True)
if not cluster_name or cluster_name in ["Cluster", "System", "Name"]:
continue
location_cell = cells[1].get_text(strip=True) if len(cells) > 1 else ""
country, city = self._parse_location(location_cell)
perf_cell = cells[2].get_text(strip=True) if len(cells) > 2 else ""
entry = {
"source_id": f"epoch_{re.sub(r'[^a-zA-Z0-9]', '_', cluster_name.lower())}",
"name": cluster_name,
"country": country,
"city": city,
"latitude": "",
"longitude": "",
"value": self._parse_performance(perf_cell),
"unit": "TFlop/s",
"metadata": {
"raw_data": perf_cell,
},
"reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
}
data.append(entry)
except (ValueError, IndexError, AttributeError):
continue
# If no table found, return sample data
if not data:
data = self._get_sample_data()
return data
def _parse_location(self, location: str) -> tuple:
"""Parse location string into country and city"""
if not location:
return "", ""
if "," in location:
parts = location.rsplit(",", 1)
city = parts[0].strip()
country = parts[1].strip() if len(parts) > 1 else ""
return country, city
return location, ""
def _parse_performance(self, perf: str) -> str:
"""Parse performance string to extract value"""
if not perf:
return "0"
match = re.search(r"([\d,.]+)\s*(TFlop/s|PFlop/s|GFlop/s)?", perf, re.I)
if match:
return match.group(1).replace(",", "")
match = re.search(r"([\d,.]+)", perf)
if match:
return match.group(1).replace(",", "")
return "0"
def _get_sample_data(self) -> List[Dict[str, Any]]:
"""Return sample data for testing when scraping fails"""
return [
{
"source_id": "epoch_sample_1",
"name": "Sample GPU Cluster",
"country": "United States",
"city": "San Francisco, CA",
"latitude": "",
"longitude": "",
"value": "1000",
"unit": "TFlop/s",
"metadata": {
"note": "Sample data - Epoch AI page structure may vary",
},
"reference_date": datetime.now(UTC).strftime("%Y-%m-%d"),
},
]
@staticmethod
def coordinate(value: str | None, bound: int) -> float | None:
if not value:
return None
number = float(value)
if not math.isfinite(number) or abs(number) > bound:
raise ValueError("Epoch coordinate out of bounds")
return number

View File

@@ -35,7 +35,7 @@ class NewsLiveStreamsCollector(BaseCollector):
DEFAULT_IPTV_ORG_LOGOS_URL = "https://iptv-org.github.io/api/logos.json"
DEFAULT_IPTV_ORG_NEWS_CATEGORIES = ("news", "business", "weather")
DEFAULT_IPTV_ORG_EXCLUDE_CATEGORIES = ("music", "sports", "kids", "entertainment")
DEFAULT_IPTV_ORG_MAX_SOURCES = 120
DEFAULT_IPTV_ORG_MAX_SOURCES = 0 # Zero keeps the complete matching channel catalog.
async def fetch(self) -> list[dict[str, Any]]:
request_url = (self._resolved_url or "").strip()
@@ -445,7 +445,7 @@ class NewsLiveStreamsCollector(BaseCollector):
"reference_date": datetime.now(UTC).isoformat(),
}
)
if len(normalized) >= max_sources:
if max_sources > 0 and len(normalized) >= max_sources:
break
return normalized

View File

@@ -5,16 +5,18 @@ https://top500.org/lists/top500/
"""
import asyncio
import math
import re
from urllib.parse import urljoin
from typing import Dict, Any, List
from bs4 import BeautifulSoup
import httpx
from app.core.data_sources import get_data_sources_config
from app.services.collectors.base import BaseCollector
from app.services.collectors.compute_base import ComputeCollector
class TOP500Collector(BaseCollector):
class TOP500Collector(ComputeCollector):
name = "top500"
priority = "P0"
module = "L1"
@@ -28,7 +30,7 @@ class TOP500Collector(BaseCollector):
async with httpx.AsyncClient(timeout=60.0, follow_redirects=True) as client:
response = await client.get(url)
response.raise_for_status()
entries = self.parse_response(response.text)
entries = self.parse_response(response.text, str(response.url))
semaphore = asyncio.Semaphore(8)
@@ -55,7 +57,7 @@ class TOP500Collector(BaseCollector):
system_name = link.get_text(" ", strip=True) if link else system_cell.get_text(" ", strip=True)
detail_url = ""
if link and link.get("href"):
detail_url = f"{str(top500_base_url).rstrip('/')}{link.get('href')}"
detail_url = urljoin(str(top500_base_url), link.get("href"))
manufacturer = ""
if link and link.next_sibling:
@@ -127,10 +129,14 @@ class TOP500Collector(BaseCollector):
return detail_map
def parse_response(self, html: str) -> List[Dict[str, Any]]:
def parse_response(self, html: str, source_url: str = "") -> List[Dict[str, Any]]:
"""Parse TOP500 HTML response"""
data = []
soup = BeautifulSoup(html, "html.parser")
canonical = soup.find("link", rel="canonical")
date_url = source_url or (canonical.get("href", "") if canonical else "") or self._resolved_url or ""
date_match = re.search(r"/lists/top500/(\d{4})/(06|11)/", date_url)
reference_date = f"{date_match[1]}-{date_match[2]}-01" if date_match else None
# Find the table with TOP500 data
table = None
@@ -152,6 +158,9 @@ class TOP500Collector(BaseCollector):
table = soup.find("table", {"class": "top500-table"}) or soup.find("table", {"id": "top500"})
if table:
headers = [cell.get_text(" ", strip=True) for cell in table.select("tr")[0].find_all(["th", "td"])]
if not reference_date:
raise ValueError("TOP500 list date missing; previous snapshot retained")
rows = table.select("tr")
for row in rows[1:]: # Skip header row
cells = row.find_all(["td", "th"])
@@ -176,25 +185,32 @@ class TOP500Collector(BaseCollector):
cores = cells[2].get_text(strip=True).replace(",", "")
rmax_text = cells[3].get_text(strip=True)
rmax = self._parse_performance(rmax_text)
rmax = self._parse_performance(rmax_text, headers[3])
rpeak_text = cells[4].get_text(strip=True)
rpeak = self._parse_performance(rpeak_text)
rpeak = self._parse_performance(rpeak_text, headers[4])
power = ""
if len(cells) >= 6:
power = cells[5].get_text(strip=True).replace(",", "")
identity = re.search(r"/system/(\d+)", detail_url)
if not identity:
raise ValueError("TOP500 stable system ID missing")
entry = {
"source_id": f"top500_{rank}",
"source_id": f"top500_system_{identity[1]}",
"name": system_name,
"country": country,
"city": city,
"latitude": 0.0,
"longitude": 0.0,
"latitude": None,
"longitude": None,
"value": str(rmax),
"unit": "PFlop/s",
"unit": "FLOP/s",
"metadata": {
"compute_contract": "v1",
"precision": "FP64", "measurement": "LINPACK Rmax",
"source_url": detail_url, "list_url": date_url,
"status": "listed", "rmax_flops": rmax, "rpeak_flops": rpeak,
"rank": rank,
"cores": cores,
"rmax": rmax_text,
@@ -204,15 +220,11 @@ class TOP500Collector(BaseCollector):
"site": site,
},
"_detail_url": detail_url,
"reference_date": "2025-11-01",
"reference_date": reference_date,
}
data.append(entry)
except (ValueError, IndexError, AttributeError) as e:
continue
# If scraping failed, return sample data for testing
if not data:
data = self._get_sample_data()
raise ValueError("Invalid TOP500 row; previous snapshot retained") from e
return data
@@ -227,9 +239,14 @@ class TOP500Collector(BaseCollector):
return 0.0
return 0.0
def _parse_performance(self, text: str) -> float:
def _parse_performance(self, text: str, header: str = "") -> float:
"""Parse performance value from text (handles E, P, T suffixes)"""
text = text.strip().upper()
text = text.strip().upper().replace(",", "")
if not re.search(r"[EPTGMK]F", text):
unit = re.search(r"\b([EPTGMK])FLOP/?S", header.upper())
if not unit:
raise ValueError("TOP500 performance unit missing")
text = f"{text} {unit[1]}FLOP/S"
multipliers = {
"E": 1e18,
"P": 1e15,
@@ -249,73 +266,9 @@ class TOP500Collector(BaseCollector):
# Try simple float parsing
try:
return float(text.replace(",", ""))
value = float(text.replace(",", ""))
if not math.isfinite(value) or value < 0:
raise ValueError("Invalid TOP500 performance")
return value
except ValueError:
return 0.0
def _get_sample_data(self) -> List[Dict[str, Any]]:
"""Return sample data for testing when scraping fails"""
return [
{
"source_id": "top500_1",
"name": "El Capitan - HPE Cray EX255a, AMD 4th Gen EPYC 24C 1.8GHz, AMD Instinct MI300A",
"country": "United States",
"city": "Livermore, CA",
"latitude": 37.6819,
"longitude": -121.7681,
"value": "1742.00",
"unit": "PFlop/s",
"metadata": {
"rank": 1,
"cores": "11039616",
"rmax": "1742.00",
"rpeak": "2746.38",
"power": "29581",
"manufacturer": "HPE",
"site": "DOE/NNSA/LLNL",
"processor": "AMD 4th Gen EPYC 24C 1.8GHz",
"interconnect": "Slingshot-11",
"installation_year": "2025",
},
"reference_date": "2025-11-01",
},
{
"source_id": "top500_2",
"name": "Frontier - HPE Cray EX235a, AMD Optimized 3rd Generation EPYC 64C 2GHz, AMD Instinct MI250X",
"country": "United States",
"city": "Oak Ridge, TN",
"latitude": 36.0107,
"longitude": -84.2663,
"value": "1353.00",
"unit": "PFlop/s",
"metadata": {
"rank": 2,
"cores": "9066176",
"rmax": "1353.00",
"rpeak": "2055.72",
"power": "24607",
"manufacturer": "HPE",
"site": "DOE/SC/Oak Ridge National Laboratory",
},
"reference_date": "2025-11-01",
},
{
"source_id": "top500_3",
"name": "Aurora - HPE Cray EX - Intel Exascale Compute Blade, Xeon CPU Max 9470 52C 2.4GHz, Intel Data Center GPU Max",
"country": "United States",
"city": "Argonne, IL",
"latitude": 41.3784,
"longitude": -87.8600,
"value": "1012.00",
"unit": "PFlop/s",
"metadata": {
"rank": 3,
"cores": "9264128",
"rmax": "1012.00",
"rpeak": "1980.01",
"power": "38698",
"manufacturer": "Intel",
},
"reference_date": "2025-11-01",
},
]
raise ValueError("Invalid TOP500 performance") from None

View File

@@ -0,0 +1,200 @@
"""Canonical country and mutually exclusive regional compute participants."""
import re
from collections.abc import Iterable
from app.core.countries import COUNTRY_CENTROIDS, COUNTRY_ENTRIES
from app.core.earth_geography_policy import display_country
PALETTE = ("#91baff", "#f0c88a", "#89d2c2", "#c5abdf", "#eda9b3", "#b6c99a")
DEFAULT_PARTICIPANTS = ("CN", "US", "EUROPE", "APAC", "OTHER")
# Explicit membership prevents a regional total from double-counting a country.
# APAC deliberately excludes China.
COMPARISON_GROUPS = (
("CN", "中国", "China", ("CN",), {"latitude": 35.8617, "longitude": 104.1954}),
("US", "美国", "United States", ("US",), {"latitude": 39.8283, "longitude": -98.5795}),
(
"EUROPE",
"欧洲",
"Europe",
(
"GB",
"DE",
"FR",
"CH",
"ES",
"FI",
"HU",
"IE",
"IT",
"LU",
"MD",
"NL",
"PL",
"RO",
"RU",
"UA",
),
{"latitude": 54.0, "longitude": 20.0},
),
(
"APAC",
"亚太(不含中国)",
"Asia Pacific (excluding China)",
("JP", "KR", "IN", "AU", "AF", "ID", "NP", "PH", "PK", "SG", "TH"),
{"latitude": 36.0, "longitude": 130.5},
),
(
"OTHER",
"其他地区",
"Other",
(
"CA",
"AO",
"EG",
"KE",
"MZ",
"NG",
"UG",
"ZA",
"BR",
"CO",
"CR",
"DO",
"MX",
"AE",
"SY",
"TR",
),
{"latitude": 12.0, "longitude": 10.0},
),
)
def _country_code(canonical: str, aliases: list[str]) -> str | None:
code = next((alias for alias in aliases if re.fullmatch(r"[A-Z]{2}", alias)), None)
return "GB" if canonical == "英国" else code
def build_country_catalog() -> list[dict]:
"""Build the policy-projected canonical country drilldown catalog."""
countries = []
for canonical, aliases in COUNTRY_ENTRIES:
if display_country(canonical) != canonical or canonical not in COUNTRY_CENTROIDS:
continue
code = _country_code(canonical, aliases)
if code:
countries.append(
{
"code": code,
"name": canonical,
"name_en": aliases[0],
"kind": "country",
"members": [code],
"anchor": dict(COUNTRY_CENTROIDS[canonical]),
"anchor_kind": "regional_label",
}
)
return countries
COUNTRY_CATALOG = build_country_catalog()
COUNTRIES = {entry["code"]: entry["name"] for entry in COUNTRY_CATALOG}
COUNTRY_PARTICIPANTS = {entry["code"]: entry for entry in COUNTRY_CATALOG}
def _validate_group_members() -> None:
members = [
member for _code, _name, _name_en, codes, _anchor in COMPARISON_GROUPS for member in codes
]
if len(members) != len(set(members)) or set(members) != set(COUNTRIES):
raise ValueError("compute comparison groups must cover country catalog exactly once")
_validate_group_members()
GROUP_BY_MEMBER = {
member: code
for code, _name, _name_en, members, _anchor in COMPARISON_GROUPS
for member in members
}
def build_catalog() -> list[dict]:
"""Return the five default participants before country drilldown entries."""
catalog: list[dict] = []
for index, (code, name, name_en, members, anchor) in enumerate(COMPARISON_GROUPS):
entry = (
dict(COUNTRY_PARTICIPANTS[code])
if code in COUNTRY_PARTICIPANTS
else {
"code": code,
"name": name,
"name_en": name_en,
"kind": "region",
"members": list(members),
"anchor": dict(anchor),
"anchor_kind": "regional_label",
}
)
entry.update(
{
"members": list(members),
"parent_group": name,
"parent_group_code": code,
"order": index,
"default_selected": True,
"color": PALETTE[index % len(PALETTE)],
}
)
catalog.append(entry)
for entry in COUNTRY_CATALOG:
if entry["code"] in DEFAULT_PARTICIPANTS:
continue
country = dict(entry)
group_code = GROUP_BY_MEMBER[country["code"]]
group_name = next(group[1] for group in COMPARISON_GROUPS if group[0] == group_code)
country.update(
{
"parent_group": group_name,
"parent_group_code": group_code,
"order": len(catalog),
"default_selected": False,
"color": PALETTE[len(catalog) % len(PALETTE)],
}
)
catalog.append(country)
return catalog
CATALOG = build_catalog()
PARTICIPANTS = {entry["code"]: entry for entry in CATALOG}
def participant_members(codes: Iterable[str]) -> dict[str, tuple[str, ...]]:
"""Resolve country or region participant codes to canonical country codes."""
return {code: tuple(PARTICIPANTS[code]["members"]) for code in codes}
def source_country_names(codes: Iterable[str]) -> list[str]:
"""Return unique canonical source labels, respecting China and Serbia ownership."""
selected_members = {
member for members in participant_members(codes).values() for member in members
}
selected_names = {COUNTRIES[code] for code in selected_members}
return [name for name, _aliases in COUNTRY_ENTRIES if display_country(name) in selected_names]
def source_country_participants(codes: Iterable[str]) -> dict[str, str]:
"""Map stored source-country labels to their requested participant code."""
owner_by_country = {
member: participant
for participant, members in participant_members(codes).items()
for member in members
}
code_by_name = {name: code for code, name in COUNTRIES.items()}
return {
name: owner_by_country[code_by_name[display_country(name)]]
for name in source_country_names(codes)
}

View File

@@ -3,7 +3,12 @@
This module is a thin domain wrapper that wires up
:mod:`app.services.location` for compute centers:
SourceCoordinates
Confirmed stored location -> trustworthy source coordinates -> exact
official facility registration -> source city/site coordinates.
Epoch AI coordinates are admitted only at the grain asserted by its official
``Location`` column. Country/region/empty locations remain unresolved even
when the CSV happens to contain a latitude/longitude pair.
The online Nominatim step is intentionally reserved for the user-triggered
``collect-location`` flow. The regular GeoJSON endpoint runs during Earth
@@ -31,6 +36,8 @@ from sqlalchemy.ext.asyncio import AsyncSession
from app.core.collected_data_fields import get_record_field
from app.models.collected_data import CollectedData
from app.models.compute_center_location import ComputeCenterLocationRecord
from app.services.compute_center_verified_sites import get_verified_compute_site
from app.services.epoch_location import assess_epoch_location
from app.services.location import (
LocationCandidate,
@@ -166,8 +173,13 @@ def set_compute_center_location_cache(
async def refresh_compute_center_location_cache(
session: AsyncSession,
*,
as_of: datetime | None = None,
) -> dict[str, dict[str, Any]]:
result = await session.execute(select(ComputeCenterLocationRecord))
query = select(ComputeCenterLocationRecord)
if as_of is not None:
query = query.where(ComputeCenterLocationRecord.verified_at < as_of)
result = await session.execute(query)
records = result.scalars().all()
cache = {}
for record in records:
@@ -350,6 +362,58 @@ class ROROrganizationResolver:
)
class ComputeCenterSourceCoordinatesResolver:
"""Use source coordinates only at the grain asserted by the source text."""
name = "source_coordinates"
def __init__(self) -> None:
self._default = SourceCoordinatesResolver()
def resolve(self, query: LocationQuery) -> ResolverOutput:
extra = query.extra or {}
if extra.get("source") != "epoch_ai_gpu":
return self._default.resolve(query)
if query.source_latitude in (None, 0.0) or query.source_longitude in (None, 0.0):
return ResolverOutput()
assessment = assess_epoch_location(
extra.get("source_location"),
country=query.country,
raw_country=extra.get("source_raw_country"),
)
if not assessment.is_renderable:
return ResolverOutput(
attempted_queries=(f"source_coordinates:{assessment.precision}",)
)
output = self._default.resolve(query)
if not output.candidates:
return output
candidate = output.candidates[0]
source_location = extra.get("source_location")
source_location_text = (
source_location.strip() if isinstance(source_location, str) else ""
)
source_note = assessment.reason
if source_location_text:
source_note = f"{source_note}; Epoch Location: {source_location_text}"
return ResolverOutput(
candidates=(
LocationCandidate(
**{
**candidate.__dict__,
"precision": assessment.precision,
"confidence": 1.0 if assessment.precision == "precise" else 0.9,
"source_note": source_note,
"needs_confirmation": assessment.precision != "precise",
"matched_location_name": source_location_text
or candidate.matched_location_name,
}
),
),
attempted_queries=output.attempted_queries,
)
class StoredComputeCenterLocationResolver:
"""Resolve a compute center through the DB-backed current-location cache."""
@@ -367,13 +431,26 @@ class StoredComputeCenterLocationResolver:
longitude = parse_float(stored.get("longitude"))
if latitude in (None, 0.0) or longitude in (None, 0.0):
return ResolverOutput()
precision = stored.get("precision") or "city"
if (
extra.get("source") == "epoch_ai_gpu"
and stored.get("location_source") == "source_coordinates"
):
assessment = assess_epoch_location(
extra.get("source_location"),
country=query.country,
raw_country=extra.get("source_raw_country"),
)
if not assessment.is_renderable:
return ResolverOutput()
precision = assessment.precision
return ResolverOutput(
candidates=(
LocationCandidate(
latitude=latitude,
longitude=longitude,
display_name=stored.get("name") or query.name or "Compute center",
precision=stored.get("precision") or "city",
precision=precision,
confidence=float(stored.get("confidence") or 0.85),
query=f"stored_compute_center_location::{stored.get('source')}:{stored.get('source_id')}",
source=self.name,
@@ -386,6 +463,60 @@ class StoredComputeCenterLocationResolver:
matched_location_name=stored.get("site") or stored.get("name") or query.name,
location_verified_at=stored.get("verified_at"),
suggested_registry_entry=None,
raw_payload={
"stored_location_source": stored.get("location_source"),
"verification_status": stored.get("verification_status"),
},
),
)
)
class VerifiedComputeSiteResolver:
"""Resolve an exact source identity to an evidence-backed facility site."""
name = "official_facility_registry"
def resolve(self, query: LocationQuery) -> ResolverOutput:
extra = query.extra or {}
source = coerce_str(extra.get("source"))
source_id = coerce_str(extra.get("source_id"))
if not source or not source_id:
return ResolverOutput()
site = get_verified_compute_site(source, source_id)
if not site:
return ResolverOutput()
source_urls = [coerce_str(url) for url in site.get("sources", []) if coerce_str(url)]
source_note = coerce_str(site.get("source_note"))
if source_urls:
source_note = f"{source_note} Sources: {'; '.join(source_urls)}".strip()
return ResolverOutput(
candidates=(
LocationCandidate(
latitude=float(site["latitude"]),
longitude=float(site["longitude"]),
display_name=coerce_str(site.get("matched_location_name"))
or query.name
or "Compute center",
precision=coerce_str(site.get("precision")) or "site",
confidence=1.0,
query=f"official_facility_registry::{source}:{source_id}",
source=self.name,
source_note=source_note or None,
matched_fields=("source", "source_id"),
needs_confirmation=False,
city=coerce_str(site.get("city")) or None,
region=None,
country=coerce_str(site.get("country")) or query.country,
matched_location_name=coerce_str(site.get("matched_location_name"))
or None,
location_verified_at=coerce_str(site.get("verified_at")) or None,
suggested_registry_entry=None,
raw_payload={
"address": site.get("address"),
"coordinate_basis": site.get("coordinate_basis"),
"sources": source_urls,
},
),
)
)
@@ -400,7 +531,7 @@ def _short_system_name(name: Any) -> str:
return head or text
def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, Any]:
name = coerce_str(getattr(record, "name", None))
return {
"source": coerce_str(getattr(record, "source", None)),
@@ -417,11 +548,13 @@ def _record_context(record: Any, metadata: dict[str, Any]) -> dict[str, str]:
or metadata.get("manufacturer")
),
"organization": coerce_str(metadata.get("organization")),
"source_location": metadata.get("location"),
"source_raw_country": metadata.get("raw_country"),
}
def _context_to_query(
context: dict[str, str],
context: dict[str, Any],
*,
source_lat: float | None = None,
source_lon: float | None = None,
@@ -444,6 +577,8 @@ def _context_to_query(
"site": context.get("site") or "",
"operator": context.get("operator") or "",
"organization": context.get("organization") or "",
"source_location": context.get("source_location"),
"source_raw_country": context.get("source_raw_country"),
},
)
@@ -497,11 +632,36 @@ def _compute_center_query_plan(
return plan
def _compute_center_candidate_sort_key(candidate: LocationCandidate) -> tuple[int, int, float]:
stored_origin = (candidate.raw_payload or {}).get("stored_location_source")
stored_confirmed = (
candidate.source == "stored_compute_center_location"
and not candidate.needs_confirmation
and stored_origin != "source_coordinates"
)
if stored_confirmed:
source_rank = 0
elif candidate.source == "source_coordinates" and candidate.precision == "precise":
source_rank = 1
elif candidate.source == "official_facility_registry":
source_rank = 2
elif candidate.source == "source_coordinates":
source_rank = 3
elif candidate.source == "stored_compute_center_location":
source_rank = 4
else:
source_rank = 5
precision_rank = {"precise": 0, "site": 1, "city": 2}.get(candidate.precision, 9)
return (source_rank, precision_rank, -float(candidate.confidence or 0))
COMPUTE_CENTER_PIPELINE = LocationPipeline(
[
SourceCoordinatesResolver(),
ComputeCenterSourceCoordinatesResolver(),
StoredComputeCenterLocationResolver(),
VerifiedComputeSiteResolver(),
],
sort_key=_compute_center_candidate_sort_key,
failure_reason=(
"Could not resolve to city-level coordinates from source coords"
" or stored compute-center location."
@@ -510,7 +670,8 @@ COMPUTE_CENTER_PIPELINE = LocationPipeline(
COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
[
SourceCoordinatesResolver(),
ComputeCenterSourceCoordinatesResolver(),
VerifiedComputeSiteResolver(),
ROROrganizationResolver(),
NominatimResolver(
query_plan_builder=_compute_center_query_plan,
@@ -518,6 +679,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
geocoder=lambda q: _geocode_online(q),
),
],
sort_key=_compute_center_candidate_sort_key,
failure_reason=(
"Could not resolve to city-level coordinates from source coords"
", ROR organization lookup, or online geocoding."
@@ -531,6 +693,7 @@ COMPUTE_CENTER_COLLECTION_PIPELINE = LocationPipeline(
_GEOGRAPHY_MODE_BY_SOURCE = {
"source_coordinates": "source_coordinates",
"stored_compute_center_location": "stored_compute_center_location",
"official_facility_registry": "official_facility_registry",
"ror_organization_registry": "ror_organization",
"nominatim_online_geocode": "online_geocode",
}
@@ -539,7 +702,7 @@ _GEOGRAPHY_MODE_BY_SOURCE = {
def _candidate_to_location(
candidate: LocationCandidate,
*,
context: dict[str, str],
context: dict[str, Any],
) -> ComputeCenterLocation:
geography_mode = _GEOGRAPHY_MODE_BY_SOURCE.get(candidate.source, "online_geocode")
is_estimated = candidate.needs_confirmation or candidate.source.startswith(
@@ -594,7 +757,7 @@ def _candidate_to_location(
def _diagnostic_for(
record: Any,
context: dict[str, str],
context: dict[str, Any],
*,
failure_reason: str,
attempted_queries: tuple[str, ...] = (),
@@ -738,7 +901,7 @@ def build_compute_center_location_query(
organization: str | None = None,
) -> LocationQuery:
name_value = coerce_str(name)
context: dict[str, str] = {
context: dict[str, Any] = {
"source": coerce_str(source),
"source_id": coerce_str(source_id),
"name": name_value,
@@ -783,6 +946,23 @@ async def seed_compute_center_locations_from_source_coords(
longitude = parse_float(get_record_field(record, "longitude"))
if latitude in (None, 0.0) or longitude in (None, 0.0):
continue
metadata = record.extra_data or {}
precision = "precise"
confidence = 1.0
needs_confirmation = False
source_note = "Seeded from source-provided compute-center coordinates"
if source_value == "epoch_ai_gpu":
assessment = assess_epoch_location(
metadata.get("location"),
country=get_record_field(record, "country"),
raw_country=metadata.get("raw_country"),
)
if not assessment.is_renderable:
continue
precision = assessment.precision
confidence = 1.0 if precision == "precise" else 0.9
needs_confirmation = precision != "precise"
source_note = assessment.reason
existing = await session.scalar(
select(ComputeCenterLocationRecord)
.where(ComputeCenterLocationRecord.source == source_value)
@@ -790,7 +970,6 @@ async def seed_compute_center_locations_from_source_coords(
)
if existing:
continue
metadata = record.extra_data or {}
session.add(
ComputeCenterLocationRecord(
source=source_value,
@@ -802,16 +981,16 @@ async def seed_compute_center_locations_from_source_coords(
country=coerce_str(get_record_field(record, "country")) or None,
latitude=latitude,
longitude=longitude,
precision="precise",
confidence=1.0,
precision=precision,
confidence=confidence,
location_source="source_coordinates",
source_note="Seeded from source-provided compute-center coordinates",
source_note=source_note,
raw_payload={
"record_id": getattr(record, "id", None),
"source": source_value,
"source_id": source_id,
},
needs_confirmation=False,
needs_confirmation=needs_confirmation,
verification_status="source_provided",
verified_at=None,
)

View File

@@ -0,0 +1,44 @@
{
"version": 1,
"sites": [
{
"source": "epoch_ai_gpu",
"source_ids": ["epoch_jamstec_zettascaler_2_0_gyoukou", "epoch_jamstec_zettascaler_2_2_gyoukou"],
"latitude": 35.380924,
"longitude": 139.625008,
"precision": "site",
"location_source": "official_facility_registry",
"country": "日本",
"city": "Yokohama",
"matched_location_name": "JAMSTEC Yokohama Institute for Earth Sciences",
"address": "3173-25 Showa-machi, Kanazawa-ku, Yokohama, Kanagawa, Japan",
"source_note": "Gyoukou deployment confirmed by JAMSTEC's 2017 release. Campus-level anchor from its official access map center, not a surveyed machine-room or building coordinate.",
"verified_at": "2026-09-21T17:06:31Z",
"coordinate_basis": "official_access_map_center",
"sources": [
"https://www.jamstec.go.jp/j/about/press_release/archive/2017/20171114.pdf",
"https://www.jamstec.go.jp/e/about/access/yokohama.html"
]
},
{
"source": "epoch_ai_gpu",
"source_ids": ["epoch_argonne_nl_sophia__formerly_theta_"],
"latitude": 41.718109383287235,
"longitude": -87.984498648496,
"precision": "site",
"location_source": "official_facility_registry",
"country": "美国",
"city": "Argonne",
"matched_location_name": "Argonne Leadership Computing Facility",
"address": "9700 South Cass Avenue, Building 240, Argonne, IL 60439, USA",
"source_note": "Sophia is an ALCF system. Campus-level anchor from ALCF's official visitor map center; the published address identifies Building 240 but the coordinate is not a surveyed building or rack location.",
"verified_at": "2026-09-21T17:06:31Z",
"coordinate_basis": "official_access_map_center",
"sources": [
"https://www.alcf.anl.gov/systems/sophia",
"https://docs.alcf.anl.gov/sophia/",
"https://www.alcf.anl.gov/about/visiting-alcf"
]
}
]
}

View File

@@ -0,0 +1,26 @@
"""Evidence-backed facility anchors; never match an operator's headquarters by name."""
import json
from copy import deepcopy
from functools import lru_cache
from pathlib import Path
@lru_cache(maxsize=1)
def _site_index() -> dict[tuple[str, str], dict]:
payload = json.loads(Path(__file__).with_suffix(".json").read_text(encoding="utf-8"))
index = {}
for site in payload["sites"]:
if not (-90 <= site["latitude"] <= 90 and -180 <= site["longitude"] <= 180):
raise ValueError("Invalid verified compute facility coordinate")
for source_id in site["source_ids"]:
key = (site["source"], source_id)
if key in index:
raise ValueError("Duplicate verified compute facility identity")
index[key] = site
return index
def get_verified_compute_site(source: str, source_id: str) -> dict | None:
site = _site_index().get((source, source_id))
return deepcopy(site) if site else None

View File

@@ -0,0 +1,99 @@
"""Database overrides with bundled defaults and atomic, revision-checked publishing."""
import hashlib
from pathlib import Path
from uuid import uuid4
from app.core.request_context import get_request_id
from app.models.system_log import AuditLog
from app.models.system_setting import SystemSetting
from app.models.user import User
from app.schemas.compute_configuration import ComputeConfiguration
from app.services.compute_catalog import COUNTRY_CATALOG
from fastapi import HTTPException
from sqlalchemy import select
from sqlalchemy.dialects.postgresql import insert
from sqlalchemy.ext.asyncio import AsyncSession
CATEGORY = "compute_comparison"
DEFAULT_PATH = Path(__file__).with_name("compute_history_defaults.json")
DEFAULT_CONFIGURATION = ComputeConfiguration.model_validate_json(DEFAULT_PATH.read_text())
DEFAULT_REVISION = hashlib.sha256(DEFAULT_PATH.read_bytes()).hexdigest()
def default_envelope() -> dict:
return {
"config": DEFAULT_CONFIGURATION.model_dump(mode="json"),
"revision": DEFAULT_REVISION,
"is_default": True,
}
async def read_configuration(db: AsyncSession) -> dict:
record = (
await db.execute(select(SystemSetting).where(SystemSetting.category == CATEGORY))
).scalar_one_or_none()
return dict(record.payload) if record else default_envelope()
def configuration_response(envelope: dict) -> dict:
return {
**envelope,
"countries": [
{key: country[key] for key in ("code", "name", "name_en")}
for country in COUNTRY_CATALOG
],
}
async def publish_configuration(
db: AsyncSession,
*,
revision: str,
config: ComputeConfiguration,
actor: User,
reset: bool = False,
) -> dict:
# The unique category plus row lock also serializes concurrent first writes.
await db.execute(
insert(SystemSetting)
.values(category=CATEGORY, payload=default_envelope())
.on_conflict_do_nothing(index_elements=[SystemSetting.category])
)
record = (
await db.execute(
select(SystemSetting).where(SystemSetting.category == CATEGORY).with_for_update()
)
).scalar_one()
if record.payload["revision"] != revision:
await db.rollback()
raise HTTPException(409, "Configuration changed. Reload before saving your edits.")
envelope = (
default_envelope()
if reset
else {
"config": config.model_dump(mode="json"),
"revision": uuid4().hex,
"is_default": reset,
}
)
before = record.payload
if reset:
await db.delete(record)
else:
record.payload = envelope
# Audit and configuration commit together: a published edit must be attributable.
db.add(
AuditLog(
actor_id=actor.id,
actor_name=actor.username,
action="compute_comparison.reset" if reset else "compute_comparison.update",
target_type="system_setting",
target_id=CATEGORY,
result="success",
request_id=get_request_id(),
details={"before": before, "after": envelope},
)
)
await db.commit()
return envelope

View File

@@ -0,0 +1,122 @@
"""Versioned, source-backed annual observations for compute comparisons."""
from copy import deepcopy
from datetime import date
from app.schemas.compute_configuration import ComputeConfiguration
from app.services.compute_catalog import (
CATALOG,
DEFAULT_PARTICIPANTS,
PARTICIPANTS,
participant_members,
)
from app.services.compute_configuration import DEFAULT_CONFIGURATION
def _participant_value(
members: tuple[str, ...], observations: dict[str, float | int], mode: str
) -> tuple[float | int | None, dict]:
covered = [member for member in members if member in observations]
missing = [member for member in members if member not in observations]
coverage = {
"status": "complete" if not missing else "incomplete",
"required_members": list(members),
"covered_members": covered,
"missing_members": missing,
}
if missing:
return None, coverage
if mode == "bilateral_share":
if (
not set(members).issubset({"CN", "US"})
or not {"CN", "US"}.issubset(observations)
or observations["US"] + observations["CN"] <= 0
):
coverage.update(
{
"status": "incompatible",
"reason": "bilateral_share_requires_CN_and_US_observations",
}
)
return None, coverage
return (
round(
sum(observations[member] for member in members)
/ (observations["US"] + observations["CN"])
* 100,
2,
),
coverage,
)
return sum(observations[member] for member in members), coverage
def compute_history(
countries: list[str],
mode: str = "global_share",
config: ComputeConfiguration | None = None,
) -> dict:
"""Aggregate only fully covered, same-year compatible observations."""
config = config if config is not None else DEFAULT_CONFIGURATION
metric = "global_share" if mode == "bilateral_share" else mode
annual = {row.year: row for row in config.observations if row.metric == metric}
gaps = {row.year: row for row in config.innovation}
observed_years = [row.year for row in config.observations] + list(gaps)
years = tuple(
range(
min(observed_years, default=date.today().year),
max([date.today().year, *observed_years]) + 1,
)
)
members_by_participant = participant_members(countries)
rows = []
for year in years:
record = annual.get(year)
observations = record.values if record else {}
values, coverage = {}, {}
for code, members in members_by_participant.items():
values[code], coverage[code] = _participant_value(members, observations, mode)
evidence_id = record.source_id if record else None
rows.append(
{"year": year, "values": values, "coverage": coverage, "evidence_id": evidence_id}
)
selected_members = {member for members in members_by_participant.values() for member in members}
has_innovation_scope = {"CN", "US"}.issubset(selected_members)
return {
"version": f"compute-history-{config.reviewed_at}",
"provenance": "curated_reports",
"reviewed_at": config.reviewed_at.isoformat(),
"time_basis": "retrospective",
"years": years,
"mode": mode,
"unit": "EFlop/s (FP32)" if mode == "absolute" else "%",
"denominator": {"global_share": "global", "bilateral_share": "US+CN", "absolute": None}[
mode
],
"catalog": deepcopy(CATALOG),
"default_participants": list(DEFAULT_PARTICIPANTS),
"countries": [deepcopy(PARTICIPANTS[code]) for code in countries],
"compute": rows,
"innovation": [
{
"year": year,
"value": gaps[year].value if year in gaps and has_innovation_scope else None,
"evidence_id": gaps[year].source_id
if year in gaps and has_innovation_scope
else None,
}
for year in years
],
"evidence": {
source.id: source.model_dump(mode="json", exclude={"id"}) for source in config.sources
},
"innovation_scope": ["US", "CN"],
"limitations": [
"公开设施样本不代表全国总算力;FP64、FP16 与 FP32 不可直接相加。",
"区域值只在全部成员有同年、同指标、可比较观测时汇总;缺失成员时返回 null。",
"全球份额不从中美或其他局部成员反推剩余地区。",
"年度资料按已配置报告引用展示,观测年与报告年份分列;跨年方法可比性仍需核验。",
"创新分差为美国减中国;只有分差,不反推单国得分,不证明算力与创新的因果关系。",
],
}

View File

@@ -0,0 +1,125 @@
{
"reviewed_at": "2026-09-22",
"sources": [
{
"id": "caict-2021",
"title": "中国算力发展指数白皮书(2021)",
"url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202109/P020210918521091309950.pdf",
"publisher": "中国信息通信研究院",
"report_year": 2021,
"published_at": null,
"locator": "正文 11 页图 4(2020 年全球份额)"
},
{
"id": "caict-2022",
"title": "中国算力发展指数白皮书(2022)",
"url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202211/P020221105727522653499.pdf",
"publisher": "中国信息通信研究院",
"report_year": 2022,
"published_at": null,
"locator": "正文 13 页图 5(2021 年全球份额)"
},
{
"id": "caict-2023",
"title": "中国算力发展指数白皮书(2023)",
"url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202309/P020240326630458153765.pdf",
"publisher": "中国信息通信研究院",
"report_year": 2023,
"published_at": null,
"locator": "正文 13 页图 5(2022 年全球份额);16 页(中国总规模)"
},
{
"id": "caict-2024",
"title": "先进计算暨算力发展指数蓝皮书(2024年)",
"url": "https://www.caict.ac.cn/kxyj/qwfb/bps/202501/P020250117503621662777.pdf",
"publisher": "中国信息通信研究院",
"report_year": 2024,
"published_at": null,
"locator": "正文 10 页图 3(2023 年全球份额);12 页(中国总规模)"
},
{
"id": "innovation-2026",
"title": "全球人工智能创新指数报告 2026:科技日报报道",
"url": "https://finance.sina.com.cn/tech/roll/2026-07-17/doc-iniickaz8224252.shtml",
"publisher": "科技日报(新浪转载)",
"report_year": 2026,
"published_at": "2026-07-17",
"locator": "2023–2025 年美国减中国的指数分差"
}
],
"observations": [
{
"year": 2020,
"metric": "global_share",
"source_id": "caict-2021",
"values": {
"US": 36,
"CN": 31,
"JP": 6
}
},
{
"year": 2021,
"metric": "global_share",
"source_id": "caict-2022",
"values": {
"US": 34,
"CN": 33,
"JP": 5
}
},
{
"year": 2022,
"metric": "global_share",
"source_id": "caict-2023",
"values": {
"US": 34,
"CN": 33,
"JP": 4
}
},
{
"year": 2023,
"metric": "global_share",
"source_id": "caict-2024",
"values": {
"US": 41,
"CN": 31,
"JP": 4
}
},
{
"year": 2022,
"metric": "absolute",
"source_id": "caict-2023",
"values": {
"CN": 302
}
},
{
"year": 2023,
"metric": "absolute",
"source_id": "caict-2024",
"values": {
"CN": 435
}
}
],
"innovation": [
{
"year": 2023,
"value": 22.02,
"source_id": "innovation-2026"
},
{
"year": 2024,
"value": 19.96,
"source_id": "innovation-2026"
},
{
"year": 2025,
"value": 17.95,
"source_id": "innovation-2026"
}
]
}

View File

@@ -6,8 +6,10 @@ import asyncio
import hashlib
import json
import shutil
import subprocess
import sys
from datetime import UTC, datetime
from functools import lru_cache
from pathlib import Path
from typing import Any
from uuid import uuid4
@@ -25,8 +27,11 @@ BOUNDARY_MANIFEST_PATH = BOUNDARY_OUTPUT_DIR / "manifest.json"
PMTILES_ARTIFACT_PATH = (
REPO_ROOT / "frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles"
)
LEGACY_GEOJSON_PATH = REPO_ROOT / "frontend/public/earth/data/countries-admin0.min.geojson"
POLICY_FALLBACK_PATH = (
REPO_ROOT / "frontend/public/earth/data/countries-admin0.china-pov-v1.geojson"
)
POV_POLICY_PATH = REPO_ROOT / "config/earth-boundary-pov-policy.china-v1.json"
POV_GEOMETRY_PATH = REPO_ROOT / "resources/earth/boundary-pov-geometry.china-v1.json"
LOCAL_CONFIG_PATH = REPO_ROOT / "config/earth-boundary-sources.local.json"
EXAMPLE_CONFIG_PATH = REPO_ROOT / "config/earth-boundary-sources.example.json"
@@ -150,6 +155,21 @@ def _stable_json_hash(payload: Any) -> str:
return hashlib.sha256(encoded.encode("utf-8")).hexdigest()
@lru_cache(maxsize=16)
def _read_versioned_asset_cached(
path: Path, mtime_ns: int, size_bytes: int
) -> tuple[dict[str, Any], str]:
payload = _read_json(path)
return payload, _stable_json_hash(payload)
def _versioned_asset(path: Path) -> tuple[dict[str, Any], str]:
if not path.exists():
return {}, ""
stat = path.stat()
return _read_versioned_asset_cached(path, stat.st_mtime_ns, stat.st_size)
def _artifact_extension(endpoint: str, content_type: str, payload: bytes) -> str:
suffix = Path(endpoint.split("?", 1)[0]).suffix.lower()
if suffix in {".json", ".geojson", ".zip", ".pbf"}:
@@ -176,62 +196,37 @@ def _directory_stats(path: Path) -> dict[str, int]:
return {"file_count": len(files), "size_bytes": sum(item.stat().st_size for item in files)}
def _load_source_feature_collection(source: dict[str, Any]) -> dict[str, Any]:
path = REPO_ROOT / source["path"]
payload = _read_json(path)
features = payload.get("features") if isinstance(payload, dict) else None
return {
"type": "FeatureCollection",
"features": features if isinstance(features, list) else [],
}
def _write_high_precision_geojson_manifest(
sources: list[dict[str, Any]],
build_input_hash: str,
missing_tools: list[str],
) -> dict[str, Any]:
BOUNDARY_OUTPUT_DIR.mkdir(parents=True, exist_ok=True)
admin0 = next(source for source in sources if source["kind"] == "admin0-boundaries")
coastline = next(source for source in sources if source["kind"] == "coastline")
claim_lines = next(source for source in sources if source["kind"] == "claim-lines")
admin0_payload = _load_source_feature_collection(admin0)
coastline_payload = _load_source_feature_collection(coastline)
claim_payload = _load_source_feature_collection(claim_lines)
for feature in coastline_payload["features"]:
props = feature.setdefault("properties", {})
if isinstance(props, dict):
props["PLANET_LAYER"] = "coastline"
base_payload = {
"type": "FeatureCollection",
"features": [*admin0_payload["features"], *coastline_payload["features"]],
}
base_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-base.geojson"
hover_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-hover.geojson"
claim_path = BOUNDARY_OUTPUT_DIR / "earth-boundaries-high-precision-claims.geojson"
_write_json(base_path, base_payload)
_write_json(hover_path, admin0_payload)
_write_json(claim_path, claim_payload)
manifest = {
"version": "natural-earth-v1",
"builtAt": _utc_now_iso(),
"tileProvider": "geojson-high-precision",
"format": "geojson-directory",
"buildInputHash": build_input_hash,
"base": base_path.name,
"hoverIndex": hover_path.name,
"claimLine": claim_path.name,
"sourceFeatureCount": {
"admin0": len(admin0_payload["features"]),
"coastline": len(coastline_payload["features"]),
"claimLines": len(claim_payload["features"]),
},
"pmtiles": None,
"missingTools": missing_tools,
}
result = subprocess.run(
[
sys.executable,
str(Path(__file__).resolve().parents[3] / "scripts/build_earth_boundary_china_pov_geojson.py"),
"--admin0-source", admin0["path"],
"--coastline-source", coastline["path"],
"--claims-source", claim_lines["path"],
"--output-dir", _relative(BOUNDARY_OUTPUT_DIR),
"--build-input-hash", build_input_hash,
"--pov-policy", _relative(POV_POLICY_PATH),
"--pov-geometry", str(POV_GEOMETRY_PATH),
],
cwd=REPO_ROOT,
capture_output=True,
text=True,
)
if result.returncode:
raise EarthBoundaryBuildError(
result.stderr.strip() or "Earth boundary policy build failed",
code="policy_build_failed",
)
manifest = _read_json(BOUNDARY_MANIFEST_PATH)
manifest["missingTools"] = missing_tools
_write_json(BOUNDARY_MANIFEST_PATH, manifest)
return manifest
@@ -291,6 +286,7 @@ def _build_input_hash(source_manifest: dict[str, Any]) -> str:
for source in source_manifest.get("sources", [])
],
"pov_policy": source_manifest.get("povPolicy"),
"pov_geometry_sha256": _versioned_asset(POV_GEOMETRY_PATH)[1],
"build_config": BUILD_CONFIG,
}
)
@@ -301,7 +297,47 @@ def _has_current_artifacts(boundary_manifest: dict[str, Any], build_input_hash:
bool(boundary_manifest)
and boundary_manifest.get("buildInputHash") == build_input_hash
and boundary_manifest.get("tileProvider") == "pmtiles-mvt"
and _policy_verified(boundary_manifest)
and PMTILES_ARTIFACT_PATH.exists()
and _pmtiles_hover_exists(boundary_manifest)
)
def _policy_verified(manifest: dict[str, Any]) -> bool:
policy, policy_hash = _versioned_asset(POV_POLICY_PATH)
geometry, geometry_hash = _versioned_asset(POV_GEOMETRY_PATH)
coverage = manifest.get("povPolicy") or {}
return (
policy.get("profile") == "china-pov-v1"
and coverage.get("profile") == policy["profile"]
and coverage.get("sha256") == policy_hash
and coverage.get("reviewedGeometrySourceCommit")
== geometry.get("sourceCommit")
and coverage.get("geometrySha256") == geometry_hash
and coverage.get("applied") is True
and {"china-taiwan-penghu", "china-hong-kong", "china-macao", "kosovo"}
<= set(coverage.get("appliedRuleIds") or [])
)
def _geojson_artifacts_exist(manifest: dict[str, Any]) -> bool:
paths = [manifest.get("base"), manifest.get("hoverIndex")]
if manifest.get("chinaClaims", {}).get("available"):
paths.append(manifest.get("claimLine"))
for asset in paths:
if not isinstance(asset, str) or Path(asset).name != asset:
return False
if not (BOUNDARY_OUTPUT_DIR / asset).is_file():
return False
return True
def _pmtiles_hover_exists(manifest: dict[str, Any]) -> bool:
hover = manifest.get("hoverIndex")
return (
isinstance(hover, str)
and Path(hover).name == hover
and (BOUNDARY_OUTPUT_DIR / hover).is_file()
)
@@ -313,24 +349,34 @@ def get_boundary_status() -> dict[str, Any]:
manifest_exists = BOUNDARY_MANIFEST_PATH.exists()
high_precision_ready = (
manifest_exists
and _policy_verified(boundary_manifest)
and (
(
boundary_manifest.get("tileProvider") == "pmtiles-mvt"
and pmtiles_exists
and _pmtiles_hover_exists(boundary_manifest)
)
or (
boundary_manifest.get("tileProvider") == "geojson-high-precision"
and _geojson_artifacts_exist(boundary_manifest)
)
or boundary_manifest.get("tileProvider") == "geojson-high-precision"
)
)
legacy_exists = LEGACY_GEOJSON_PATH.exists()
legacy_exists = POLICY_FALLBACK_PATH.exists() and _policy_verified(
_versioned_asset(POLICY_FALLBACK_PATH)[0]
)
provider = (
boundary_manifest.get("tileProvider")
if high_precision_ready
else "legacy-geojson" if legacy_exists else "missing"
else "policy-geojson-fallback" if legacy_exists else "missing"
)
return {
"provider": provider,
"high_precision_ready": high_precision_ready,
"fallback_available": legacy_exists,
"policy_publishable": bool(
high_precision_ready and boundary_manifest.get("povPolicy", {}).get("publicationReady")
),
"config_source": config_source,
"config_path": _relative(LOCAL_CONFIG_PATH),
"config_exists": LOCAL_CONFIG_PATH.exists(),
@@ -347,6 +393,8 @@ def get_boundary_status() -> dict[str, Any]:
"tileProvider": boundary_manifest.get("tileProvider"),
"buildInputHash": boundary_manifest.get("buildInputHash"),
"builtAt": boundary_manifest.get("builtAt"),
"povPolicy": boundary_manifest.get("povPolicy"),
"publicationReady": boundary_manifest.get("povPolicy", {}).get("publicationReady", False),
},
"pmtiles": {
"path": _relative(PMTILES_ARTIFACT_PATH),
@@ -354,9 +402,9 @@ def get_boundary_status() -> dict[str, Any]:
"size_bytes": PMTILES_ARTIFACT_PATH.stat().st_size if pmtiles_exists else 0,
},
"legacy": {
"path": _relative(LEGACY_GEOJSON_PATH),
"path": _relative(POLICY_FALLBACK_PATH),
"exists": legacy_exists,
"size_bytes": LEGACY_GEOJSON_PATH.stat().st_size if legacy_exists else 0,
"size_bytes": POLICY_FALLBACK_PATH.stat().st_size if legacy_exists else 0,
},
"source_manifest": {
"path": _relative(SOURCE_MANIFEST_PATH),
@@ -577,6 +625,8 @@ async def build_boundary_assets(progress_callback: Any = None) -> dict[str, Any]
build_input_hash,
"--pov-policy",
_relative(POV_POLICY_PATH),
"--pov-geometry",
str(POV_GEOMETRY_PATH),
]
)
if progress_callback:

View File

@@ -56,6 +56,8 @@ def build_earth_update_from_db_payload(payload: dict[str, Any]) -> dict[str, Any
source_has_adapter = bool(get_earth_update_layers_for_source(source))
effective_source = source if source_has_adapter else (table_name if table_name else source)
operation = payload.get("operation")
if "vessels" in layers and operation in {"DELETE", "TRUNCATE"}:
refresh_strategy = "reload"
update: dict[str, Any] = {
"event": "earth.layer.changed",
"action": "database_changed",
@@ -113,6 +115,8 @@ class PendingEarthDbChange:
operation = payload.get("operation")
if operation:
self.operations.add(str(operation))
if "vessels" in self.layers and operation in {"DELETE", "TRUNCATE"}:
self.refresh_strategy = "reload"
entity_keys = payload.get("entity_keys")
if not isinstance(entity_keys, list):
entity_key = payload.get("entity_key")
@@ -176,6 +180,17 @@ class EarthDbChangeDispatcher:
if not update:
return False
if payload.get("table") == "vessel_current_state" and payload.get("operation") == "DELETE":
from app.core.websocket.broadcaster import broadcaster
keys = payload.get("entity_keys")
if not isinstance(keys, list):
keys = [payload.get("entity_key")]
broadcaster.enqueue_vessel_update({
"action": "remove",
"vessels": [{"mmsi": key} for key in keys if key is not None],
})
source = update["source"]
pending = self._pending.get(source)
if pending is None:

View File

@@ -25,6 +25,7 @@ EARTH_LAYER_ADAPTERS: tuple[EarthLayerAdapter, ...] = (
layers=("vessels",),
cache_patterns=("vessels*", "summary*"),
derived_models=("ais_raw_observations", "ais_conflict_records", "ais_source_health"),
refresh_strategy="delta",
),
EarthLayerAdapter(
sources=frozenset(

View File

@@ -0,0 +1,198 @@
"""Interpret the geographic grain of Epoch AI's official ``Location`` field."""
from __future__ import annotations
from dataclasses import dataclass
import math
import re
from typing import Any
from app.core.countries import normalize_country
_COUNTRY_LABEL_OVERRIDES = {
"korea republic of": "韩国",
}
_REGION_ONLY_LABELS = {
"europe",
"iowa",
"maryland",
"new mexico",
"texas",
"the nordics",
"virginia",
"washington",
}
_UNLOCATABLE_LABELS = {'"on prem" for tesla', "on prem for tesla"}
_HEMISPHERE_COORDINATE_PATTERN = re.compile(
r"(?P<degrees>\d{1,3})(?:[°º]|\s)\s*(?P<minutes>\d{1,2})(?:['′]|\s)"
r"(?P<seconds>\d{1,2}(?:\.\d+)?)?[\"″]?\s*(?P<hemisphere>[NSEW])",
re.IGNORECASE,
)
_LABELED_COORDINATE_PAIR_PATTERN = re.compile(
r"\b(?:lat|latitude)\b\s*[:=]?\s*(?P<latitude>-?\d{1,2}(?:\.\d+)?)"
r".{0,40}\b(?:lon|lng|longitude)\b\s*[:=]?\s*"
r"(?P<longitude>-?\d{1,3}(?:\.\d+)?)",
re.IGNORECASE | re.DOTALL,
)
_DECIMAL_COORDINATE_PAIR_PATTERN = re.compile(
r"^\s*[\[(]?\s*(-?\d{1,2}\.\d+)\s*[,;]\s*"
r"(-?\d{1,3}\.\d+)\s*[\])]?\s*$"
)
_NUMBERED_STREET_PATTERN = re.compile(
r"(?:\b(?:no\.?\s*)?\d+[a-z]?\s+(?:[\w.'’\-]+\s+){0,6}"
r"(?:road|rd|street|st|avenue|ave|boulevard|blvd|drive|dr|lane|ln|"
r"highway|hwy|way|parkway|pkwy|chome)\b|"
r"\b(?:via|rue|calle|plaça|strasse|straße)\b[^,\n]{0,80},?\s*\d)",
re.IGNORECASE,
)
_POSTAL_CODE_PATTERN = re.compile(
r"(?:\b\d{5}(?:-\d{4})?\b|\b\d{3}-\d{4}\b|"
r"\b[A-Z]{1,2}\d[A-Z\d]?\s*\d[A-Z]{2}\b)",
re.IGNORECASE,
)
_SITE_CONTEXT_PATTERN = re.compile(
r"\b(?:campus|building|cluster|data\s+cent(?:er|re)|facility|institute|"
r"laborator(?:y|ies)|science\s+park|supercomputing\s+cent(?:er|re))\b",
re.IGNORECASE,
)
_COORDINATE_CUE_PATTERN = re.compile(
r"(?:\b(?:lat|latitude|lon|lng|longitude)\b|[°º].*[NSEW])",
re.IGNORECASE | re.DOTALL,
)
@dataclass(frozen=True)
class EpochLocationAssessment:
precision: str
reason: str
@property
def is_renderable(self) -> bool:
return self.precision in {"precise", "site", "city"}
def _normalized_label(value: Any) -> str:
return re.sub(r"[^a-z0-9]+", " ", str(value or "").casefold()).strip()
def _country_from_label(value: Any) -> str | None:
return normalize_country(value) or _COUNTRY_LABEL_OVERRIDES.get(
_normalized_label(value)
)
def _coordinate(mapping: dict[str, Any], keys: tuple[str, ...], bound: float) -> float | None:
raw = next((mapping.get(key) for key in keys if mapping.get(key) not in (None, "")), None)
if isinstance(raw, bool):
return None
try:
number = float(raw)
except (TypeError, ValueError):
return None
return number if math.isfinite(number) and abs(number) <= bound else None
def _has_explicit_coordinate_pair(text: str) -> bool:
hemispheres = {}
for match in _HEMISPHERE_COORDINATE_PATTERN.finditer(text):
hemisphere = match.group("hemisphere").upper()
degrees = int(match.group("degrees"))
minutes = int(match.group("minutes"))
seconds = float(match.group("seconds") or 0)
bound = 90 if hemisphere in {"N", "S"} else 180
if degrees <= bound and minutes < 60 and seconds < 60:
hemispheres[hemisphere] = degrees
if ({"N", "S"} & hemispheres.keys()) and ({"E", "W"} & hemispheres.keys()):
return True
labeled_pair = _LABELED_COORDINATE_PAIR_PATTERN.search(text)
if labeled_pair:
latitude = float(labeled_pair.group("latitude"))
longitude = float(labeled_pair.group("longitude"))
return abs(latitude) <= 90 and abs(longitude) <= 180
decimal_pair = _DECIMAL_COORDINATE_PAIR_PATTERN.match(text)
if not decimal_pair:
return False
latitude, longitude = map(float, decimal_pair.groups())
return abs(latitude) <= 90 and abs(longitude) <= 180
def _has_address_evidence(text: str) -> bool:
if _NUMBERED_STREET_PATTERN.search(text):
return True
return bool(
_POSTAL_CODE_PATTERN.search(text)
and ("\n" in text or text.count(",") >= 2)
and _SITE_CONTEXT_PATTERN.search(text)
)
def assess_epoch_location(
location: Any,
*,
country: Any = None,
raw_country: Any = None,
) -> EpochLocationAssessment:
"""Classify source coordinates without upgrading their source text.
Epoch's CSV supplies coordinates even when ``Location`` contains only a
country. Those coordinates are useful source evidence, but are not a
facility point and must not enter the renderable location pipeline.
"""
if isinstance(location, dict):
latitude = _coordinate(location, ("lat", "latitude"), 90)
longitude = _coordinate(location, ("lon", "lng", "longitude"), 180)
has_coordinates = latitude is not None and longitude is not None
return EpochLocationAssessment(
"precise" if has_coordinates else "unknown",
"complete explicit coordinate object"
if has_coordinates
else "coordinate object lacks a complete valid latitude/longitude pair",
)
text = str(location or "").strip()
if not text:
return EpochLocationAssessment("unknown", "Epoch Location is empty")
normalized = _normalized_label(text)
normalized_country = normalize_country(country)
source_country = normalize_country(raw_country)
location_country = _country_from_label(text)
if location_country or (
raw_country and normalized == _normalized_label(raw_country)
):
matched = location_country or source_country or normalized_country
return EpochLocationAssessment(
"country",
f"Epoch Location identifies country only: {matched or text}",
)
if normalized in _REGION_ONLY_LABELS:
return EpochLocationAssessment(
"region", f"Epoch Location identifies region/state only: {text}"
)
comma_parts = [part.strip() for part in text.split(",") if part.strip()]
if (
len(comma_parts) == 2
and _normalized_label(comma_parts[0]) in _REGION_ONLY_LABELS
and _country_from_label(comma_parts[1])
):
return EpochLocationAssessment(
"region", f"Epoch Location identifies region/state only: {text}"
)
if normalized in _UNLOCATABLE_LABELS:
return EpochLocationAssessment("unknown", "Epoch Location has no geographic place")
if normalized.isdigit():
return EpochLocationAssessment("unknown", "Epoch Location is only a numeric label")
if _has_explicit_coordinate_pair(text):
return EpochLocationAssessment(
"precise", "Epoch Location contains a complete valid coordinate pair"
)
if _COORDINATE_CUE_PATTERN.search(text):
return EpochLocationAssessment(
"unknown", "Epoch Location contains an incomplete or invalid coordinate pair"
)
if _has_address_evidence(text):
return EpochLocationAssessment("precise", "Epoch Location contains an address")
if "\n" in text or _SITE_CONTEXT_PATTERN.search(text):
return EpochLocationAssessment("site", "Epoch Location names a facility or campus")
return EpochLocationAssessment("city", "Epoch Location identifies a locality")

View File

@@ -0,0 +1,186 @@
"""Authenticated model discovery shared by refresh and connection checks."""
from __future__ import annotations
import asyncio
from dataclasses import dataclass
from datetime import datetime
from urllib.parse import urlsplit, urlunsplit
import httpx
CATALOG_TIMEOUT_SECONDS = 30
CATALOG_MAX_PAGES = 100
CATALOG_PAGE_SIZE = 100
CATALOG_REQUEST_ATTEMPTS = 2
CATALOG_RETRY_DELAY_SECONDS = 0.2
SUPPORTED_PROVIDER_APIS = {
"openai-completions",
"openai-responses",
"anthropic-messages",
"ollama-generate",
}
class LLMProviderCatalogError(RuntimeError):
"""A safe, user-facing catalog failure with no upstream response or credentials."""
@dataclass(frozen=True)
class ModelCatalog:
url: str
models: list[str]
def model_catalog_url(provider: str, base_url: str, provider_api: str) -> str:
parts = urlsplit(base_url.strip())
if parts.scheme not in {"http", "https"} or not parts.hostname:
raise LLMProviderCatalogError("请填写有效的 HTTP(S) 模型基础地址。")
if parts.username or parts.password or parts.query or parts.fragment:
raise LLMProviderCatalogError("模型基础地址不能包含账号、密码、查询参数或片段。")
if provider_api not in SUPPORTED_PROVIDER_APIS:
raise LLMProviderCatalogError("当前接口协议不支持模型目录查询。")
path = parts.path.rstrip("/")
if provider_api == "ollama-generate":
path = path.removesuffix("/api").removesuffix("/v1") + "/api/tags"
elif provider == "alibaba" and parts.hostname.endswith(".aliyuncs.com"):
path = "/api/v1/models"
else:
if not path or (provider_api == "anthropic-messages" and path.endswith("/anthropic")):
path += "/v1"
path += "/models"
return urlunsplit((parts.scheme, parts.netloc, path, "", ""))
def catalog_error_message(exc: Exception) -> str:
if isinstance(exc, LLMProviderCatalogError):
return str(exc)
if isinstance(exc, (httpx.TimeoutException, TimeoutError)):
return "模型目录请求超时,请检查网络后重试。"
if isinstance(exc, httpx.HTTPStatusError):
code = exc.response.status_code
messages = {
401: "API Key 验证失败,请检查当前供应商的凭证。",
403: "当前 API Key 无权访问该模型目录,请检查账号权限和服务地域。",
404: "模型目录接口不存在,请检查基础地址、地域和接口协议。",
429: "供应商请求限流,请稍后重试。",
}
return messages.get(code, f"供应商模型目录返回 HTTP {code},请稍后重试。")
if isinstance(exc, httpx.RequestError):
return "无法连接模型目录,请检查基础地址和网络。"
return "模型目录响应无效,请稍后重试。"
def _model_rows(payload: object) -> tuple[list[dict[str, object]], dict[str, object]]:
if not isinstance(payload, dict):
raise LLMProviderCatalogError("供应商返回了无效的模型目录。")
envelope = payload.get("output", payload)
if not isinstance(envelope, dict):
raise LLMProviderCatalogError("供应商返回了无效的模型目录。")
rows = envelope.get("data", envelope.get("models"))
if not isinstance(rows, list):
raise LLMProviderCatalogError("供应商响应中没有模型列表。")
if any(not isinstance(row, dict) for row in rows):
raise LLMProviderCatalogError("供应商返回了无效的模型条目。")
return rows, envelope
def _model_id(row: dict[str, object]) -> str:
value = row.get("id") or row.get("model") or row.get("name")
if not isinstance(value, str) or not value.strip():
raise LLMProviderCatalogError("供应商返回了缺少 ID 的模型条目。")
return value.strip()
def _model_date(row: dict[str, object]) -> float:
value = row.get("created_at") or row.get("published_time") or row.get("created")
if isinstance(value, (int, float)):
return float(value)
if isinstance(value, str):
try:
return datetime.fromisoformat(value.replace("Z", "+00:00")).timestamp()
except ValueError:
pass
return 0
async def _get_catalog_page(
client: httpx.AsyncClient,
url: str,
headers: dict[str, str],
params: dict[str, str | int],
) -> object:
for attempt in range(CATALOG_REQUEST_ATTEMPTS):
try:
response = await client.get(url, headers=headers, params=params)
response.raise_for_status()
return response.json()
except httpx.HTTPStatusError as exc:
if attempt or exc.response.status_code not in {502, 503, 504}:
raise
except httpx.TransportError:
if attempt:
raise
await asyncio.sleep(CATALOG_RETRY_DELAY_SECONDS)
raise LLMProviderCatalogError("模型目录请求失败。")
async def fetch_model_catalog(
provider: str,
base_url: str,
provider_api: str,
api_key: str = "",
anthropic_version: str = "2023-06-01",
timeout_seconds: int = CATALOG_TIMEOUT_SECONDS,
) -> ModelCatalog:
url = model_catalog_url(provider, base_url, provider_api)
public_catalog = provider in {"opencode-go", "openrouter"}
if not api_key and provider_api != "ollama-generate" and not public_catalog:
raise LLMProviderCatalogError("请先配置当前供应商的 API Key,再刷新模型列表。")
headers = {"User-Agent": "Planet/1.0", "Accept": "application/json"}
if provider_api == "anthropic-messages" and provider not in {
"opencode-go",
"openrouter",
"alibaba",
"moonshotai",
}:
headers.update({"x-api-key": api_key, "anthropic-version": anthropic_version})
elif api_key:
headers["Authorization"] = f"Bearer {api_key}"
native_dashscope = provider == "alibaba" and urlsplit(url).hostname.endswith(".aliyuncs.com")
params: dict[str, str | int] = {}
if native_dashscope:
params = {"page_no": 1, "page_size": CATALOG_PAGE_SIZE, "capabilities": "TG"}
rows_by_id: dict[str, dict[str, object]] = {}
timeout = max(1, min(timeout_seconds, CATALOG_TIMEOUT_SECONDS))
# Bound the complete pagination/retry cycle, not just each individual request.
async with asyncio.timeout(timeout), httpx.AsyncClient(timeout=timeout) as client:
for page in range(CATALOG_MAX_PAGES):
payload = await _get_catalog_page(client, url, headers, params)
rows, envelope = _model_rows(payload)
previous_count = len(rows_by_id)
for row in rows:
rows_by_id[_model_id(row)] = row
has_more = envelope.get("has_more") is True
if native_dashscope:
total = envelope.get("total")
if not isinstance(total, int) or total < 0:
raise LLMProviderCatalogError("供应商返回了无效的模型目录分页信息。")
has_more = len(rows_by_id) < total
params["page_no"] = page + 2
elif has_more:
cursor = envelope.get("last_id")
if not isinstance(cursor, str) or not cursor or cursor == params.get("after_id"):
raise LLMProviderCatalogError("供应商返回了无效的模型目录分页信息。")
params["after_id"] = cursor
if not has_more:
models = sorted(
rows_by_id, key=lambda key: _model_date(rows_by_id[key]), reverse=True
)
# An empty Ollama catalog is valid: no models have been installed yet.
if not models and provider_api != "ollama-generate":
raise LLMProviderCatalogError("供应商返回了空模型目录,已保留上次模型列表。")
return ModelCatalog(url=url, models=models)
if len(rows_by_id) == previous_count:
raise LLMProviderCatalogError("供应商模型目录分页没有进展,已保留上次模型列表。")
raise LLMProviderCatalogError("供应商模型目录分页超过限制,已保留上次模型列表。")

View File

@@ -3,17 +3,30 @@
from __future__ import annotations
from typing import Any
from urllib.parse import urlsplit
import httpx
MODELS_DEV_URL = "https://models.dev/api.json"
OPENCODE_GO_MODELS_URL = "https://opencode.ai/zen/go/v1/models"
from app.services.llm_model_catalog import fetch_model_catalog
OPENCODE_GO_MODEL_PROVIDER_APIS = {
"minimax-m3": "anthropic-messages",
"qwen3.8-max": "anthropic-messages",
"qwen3.8-flash": "anthropic-messages",
"qwen3.7-max": "anthropic-messages",
"qwen3.7-plus": "anthropic-messages",
"qwen3.6-plus": "anthropic-messages",
"grok-4.6": "openai-responses",
"gpt-5.6-luna": "openai-responses",
"muse-spark-1.3-contributor": "openai-responses",
"muse-spark-1.2-contributor": "openai-responses",
"minimax-m2.7": "anthropic-messages",
"minimax-m2.5": "anthropic-messages",
}
OPENCODE_GO_FALLBACK_MODELS = [
"minimax-m3",
"kimi-k3",
"glm-5.3",
"qwen3.8-max",
"gpt-5.6-luna",
"minimax-m2.7",
"minimax-m2.5",
"kimi-k2.6",
@@ -29,24 +42,47 @@ OPENCODE_GO_FALLBACK_MODELS = [
]
OPENAI_MODEL_PROVIDER_APIS = {
model: "openai-responses"
for model in [
"gpt-6-astra",
"gpt-5.6-sol",
"gpt-5.6-terra",
"gpt-5.6-luna",
"gpt-5.1",
"gpt-5.1-codex",
]
}
FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
"minimax": {
"provider": "minimax",
"label": "MiniMax",
"provider_api": "anthropic-messages",
"base_url": "https://api.minimaxi.com/anthropic",
"model": "MiniMax-M2.7",
"models": ["MiniMax-M2.7", "MiniMax-M2.7-highspeed", "MiniMax-M2.5", "MiniMax-M2"],
"model": "MiniMax-M3",
"models": [
"MiniMax-M3",
"MiniMax-M2.7",
"MiniMax-M2.7-highspeed",
"MiniMax-M2.5",
"MiniMax-M2.5-highspeed",
"MiniMax-M2.1",
"MiniMax-M2.1-highspeed",
"MiniMax-M2",
],
"api_key_env": "MINIMAX_API_KEY",
"source": "fallback",
},
"openai": {
"provider": "openai",
"label": "OpenAI",
"provider_api": "openai-completions",
"provider_api": "openai-responses",
"base_url": "https://api.openai.com/v1",
"model": "gpt-5.1",
"models": ["gpt-5.1", "gpt-5.1-codex", "gpt-4.1", "gpt-4o"],
"model": "gpt-6-astra",
"models": ["gpt-6-astra", "gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna", "gpt-4.1"],
"model_provider_apis": OPENAI_MODEL_PROVIDER_APIS,
"api_key_env": "OPENAI_API_KEY",
"source": "fallback",
},
@@ -55,8 +91,8 @@ FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
"label": "Anthropic",
"provider_api": "anthropic-messages",
"base_url": "https://api.anthropic.com/v1",
"model": "claude-sonnet-4-6",
"models": ["claude-sonnet-4-6", "claude-opus-4-5", "claude-3-5-haiku-20241022"],
"model": "claude-opus-5",
"models": ["claude-opus-5", "claude-sonnet-4-6"],
"api_key_env": "ANTHROPIC_API_KEY",
"source": "fallback",
},
@@ -65,8 +101,8 @@ FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
"label": "DeepSeek",
"provider_api": "openai-completions",
"base_url": "https://api.deepseek.com/v1",
"model": "deepseek-chat",
"models": ["deepseek-chat", "deepseek-reasoner"],
"model": "deepseek-flash",
"models": ["deepseek-flash", "deepseek-v4-pro"],
"api_key_env": "DEEPSEEK_API_KEY",
"source": "fallback",
},
@@ -75,8 +111,8 @@ FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
"label": "Alibaba Qwen / DashScope",
"provider_api": "openai-completions",
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
"model": "qwen3-max",
"models": ["qwen3-max", "qwen3.5-plus", "qwen-max", "qwen-plus"],
"model": "qwen3.8-max",
"models": ["qwen3.8-max", "qwen3-max", "qwen-plus"],
"api_key_env": "DASHSCOPE_API_KEY",
"source": "fallback",
},
@@ -85,8 +121,8 @@ FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
"label": "Moonshot AI / Kimi",
"provider_api": "openai-completions",
"base_url": "https://api.moonshot.ai/v1",
"model": "kimi-k2.5",
"models": ["kimi-k2.5", "kimi-k2-thinking", "kimi-k2-turbo-preview"],
"model": "kimi-k3",
"models": ["kimi-k3", "kimi-k2.5"],
"api_key_env": "MOONSHOT_API_KEY",
"source": "fallback",
},
@@ -123,16 +159,6 @@ FALLBACK_LLM_PROVIDER_PRESETS: dict[str, dict[str, Any]] = {
},
}
MODELS_DEV_PROVIDER_KEYS = {
"minimax": "minimax",
"openai": "openai",
"anthropic": "anthropic",
"deepseek": "deepseek",
"alibaba": "alibaba",
"moonshotai": "moonshotai",
"openrouter": "openrouter",
}
def list_fallback_llm_provider_presets() -> list[dict[str, Any]]:
return [dict(value) for value in FALLBACK_LLM_PROVIDER_PRESETS.values()]
@@ -152,65 +178,40 @@ def _opencode_go_model_provider_apis(model_ids: list[str]) -> dict[str, str]:
}
async def refresh_llm_provider_preset(provider: str, api_key: str | None = None) -> dict[str, Any]:
async def refresh_llm_provider_preset(
provider: str,
api_key: str | None = None,
*,
base_url: str | None = None,
provider_api: str | None = None,
anthropic_version: str = "2023-06-01",
) -> dict[str, Any]:
fallback = get_fallback_llm_provider_preset(provider)
if fallback["provider"] == "opencode-go":
headers = {"User-Agent": "Planet/1.0"}
if api_key:
headers["Authorization"] = f"Bearer {api_key}"
async with httpx.AsyncClient(timeout=15.0, follow_redirects=True) as client:
response = await client.get(
OPENCODE_GO_MODELS_URL,
headers=headers,
)
response.raise_for_status()
payload = response.json()
data = payload.get("data") if isinstance(payload, dict) else []
model_ids = [
str(item.get("id"))
for item in data
if isinstance(item, dict) and item.get("id")
][:120]
if not model_ids:
model_ids = fallback["models"]
return {
**fallback,
"model": fallback["model"] if fallback["model"] in model_ids else model_ids[0],
"models": model_ids,
"model_provider_apis": _opencode_go_model_provider_apis(model_ids),
"source": OPENCODE_GO_MODELS_URL,
}
models_dev_key = MODELS_DEV_PROVIDER_KEYS.get(fallback["provider"])
if not models_dev_key:
return fallback
async with httpx.AsyncClient(timeout=15.0, follow_redirects=True) as client:
response = await client.get(
MODELS_DEV_URL,
headers={"User-Agent": "Planet/1.0"},
)
response.raise_for_status()
catalog = response.json()
upstream = catalog.get(models_dev_key)
if not isinstance(upstream, dict):
return fallback
upstream_models = upstream.get("models") if isinstance(upstream.get("models"), dict) else {}
model_ids = list(upstream_models.keys())[:80]
base_url = upstream.get("api") or fallback["base_url"]
if fallback["provider"] == "deepseek" and base_url == "https://api.deepseek.com":
base_url = "https://api.deepseek.com/v1"
refreshed = {
resolved_base_url = base_url or fallback["base_url"]
resolved_api = provider_api or fallback["provider_api"]
catalog = await fetch_model_catalog(
fallback["provider"],
resolved_base_url,
resolved_api,
api_key or "",
anthropic_version,
)
model_provider_apis = (
_opencode_go_model_provider_apis(catalog.models)
if fallback["provider"] == "opencode-go"
else fallback.get("model_provider_apis", {})
)
if (
fallback["provider"] == "openai"
and urlsplit(resolved_base_url).hostname != "api.openai.com"
):
model_provider_apis = {}
return {
**fallback,
"label": upstream.get("name") or fallback["label"],
"base_url": base_url,
"model": model_ids[0] if model_ids else fallback["model"],
"models": model_ids or fallback["models"],
"api_key_env": (upstream.get("env") or [fallback["api_key_env"]])[0],
"source": MODELS_DEV_URL,
"base_url": resolved_base_url,
"provider_api": resolved_api,
"model": catalog.models[0] if catalog.models else "",
"models": catalog.models,
"model_provider_apis": model_provider_apis,
"source": catalog.url,
}
return refreshed

View File

@@ -635,6 +635,7 @@ async def _run_assistant_message(
constraints=_collect_constraints(payload.constraints),
context={
"source": "playground",
"session_id": session_id,
"preset": payload.selected_preset_key,
"conversation_history": conversation_history,
"history_size": len(conversation_history),

View File

@@ -0,0 +1,148 @@
"""Search and paginate the public live TV catalog at the database boundary."""
from typing import Any
from sqlalchemy import Select, func, select
from sqlalchemy.ext.asyncio import AsyncSession
from app.core.time import to_iso8601_utc
from app.models.collected_data import CollectedData
from app.services.tv_streams import (
TV_LIVE_SOURCE_COLLECTOR,
TV_LIVE_SOURCE_DATA_TYPE,
_build_collected_tv_source,
build_public_tv_payload,
get_tv_settings_payload,
)
def _source_key():
return func.coalesce(
func.nullif(CollectedData.extra_data["id"].as_string(), ""),
func.nullif(CollectedData.source_id, ""),
CollectedData.entity_key,
)
def _collected_catalog_query(configured_ids: list[str]) -> Select[tuple[CollectedData]]:
metadata = CollectedData.extra_data
source_id = _source_key()
enabled = func.lower(func.trim(func.coalesce(metadata["is_enabled"].as_string(), "true")))
ranked = (
select(
CollectedData.id,
func.row_number()
.over(
partition_by=source_id,
order_by=CollectedData.id.desc(),
)
.label("source_rank"),
)
.where(
CollectedData.source == TV_LIVE_SOURCE_COLLECTOR,
CollectedData.data_type == TV_LIVE_SOURCE_DATA_TYPE,
CollectedData.is_current.is_(True),
CollectedData.is_valid == 1,
CollectedData.deleted_at.is_(None),
enabled.notin_(("false", "0", "no", "off")),
source_id.notin_(configured_ids),
)
.subquery()
)
return (
select(CollectedData)
.join(ranked, ranked.c.id == CollectedData.id)
.where(ranked.c.source_rank == 1)
)
def _filter_catalog_query(
query: Select[tuple[CollectedData]], terms: list[str]
) -> Select[tuple[CollectedData]]:
metadata = CollectedData.extra_data
searchable = func.lower(
func.concat_ws(
" ",
CollectedData.name,
CollectedData.title,
CollectedData.source_id,
metadata["name"].as_string(),
metadata["provider"].as_string(),
metadata["region"].as_string(),
metadata["country"].as_string(),
metadata["language"].as_string(),
)
)
for term in terms:
query = query.where(searchable.contains(term, autoescape=True))
return query
def _matches_source(source: dict[str, Any], terms: list[str]) -> bool:
searchable = " ".join(
str(source.get(key) or "") for key in ("id", "name", "provider", "region", "language")
).lower()
return all(term in searchable for term in terms)
async def get_tv_catalog_page(
db: AsyncSession,
*,
offset: int = 0,
limit: int = 50,
q: str = "",
selected_id: str | None = None,
) -> dict[str, Any]:
settings = await get_tv_settings_payload(db)
payload = build_public_tv_payload(settings, [])
configured = payload["sources"]
query = _collected_catalog_query([source["id"] for source in settings["sources"]])
if selected_id:
selected = next((source for source in configured if source["id"] == selected_id), None)
if selected is None:
record = await db.scalar(query.where(_source_key() == selected_id).limit(1))
selected = _build_collected_tv_source(record, 0) if record else None
if selected:
payload["selected_source"] = selected
summary = (
await db.execute(
select(func.count(), func.max(CollectedData.collected_at))
.select_from(CollectedData)
.where(CollectedData.id.in_(query.with_only_columns(CollectedData.id)))
)
).one()
total_collected, latest_update = summary
terms = q.lower().split()
matched_configured = [source for source in configured if _matches_source(source, terms)]
filtered_query = _filter_catalog_query(query, terms)
matched_collected = (
await db.scalar(select(func.count()).select_from(filtered_query.subquery()))
if terms
else total_collected
)
sources = matched_configured[offset : offset + limit]
remaining = limit - len(sources)
if remaining:
rows = await db.scalars(
filtered_query.order_by(func.lower(CollectedData.name), CollectedData.id)
.offset(max(0, offset - len(matched_configured)))
.limit(remaining)
)
sources.extend(
_build_collected_tv_source(record, index) for index, record in enumerate(rows)
)
total = len(matched_configured) + matched_collected
next_offset = offset + len(sources)
return {
**payload,
"sources": sources,
"source_count": len(configured) + total_collected,
"latest_updated_at": (
to_iso8601_utc(latest_update) if latest_update else payload["latest_updated_at"]
),
"total": total,
"offset": offset,
"limit": limit,
"has_more": next_offset < total,
"next_offset": next_offset if next_offset < total else None,
}

View File

@@ -11,10 +11,16 @@ from app.core.time import to_iso8601_utc
from app.models.collected_data import CollectedData
from app.models.system_setting import SystemSetting
DEFAULT_TV_SOURCE_ID = "cgtn-en"
DEFAULT_TV_SOURCE_ID = "aljazeera-mubasher"
TV_SETTINGS_CATEGORY = "tv"
TV_LIVE_SOURCE_COLLECTOR = "news_live_streams"
TV_LIVE_SOURCE_DATA_TYPE = "news_live_stream"
CCTV4_HOMEPAGE = "https://tv.cctv.com/live/cctv4/"
CCTV4_STREAM = "https://h5cul1yar48um3t.wcetv.com/hls/cctv4.m3u8"
UNSUPPORTED_CCTV4_STREAM = (
"https://ldocctvwbcdtxy.liveplay.myqcloud.com/ldocctvwbcd/cdrmldcctv4_1_td.m3u8"
)
CCTV4_PLAYBACK_NOTE = "CCTV-4 美洲版,使用 WCETV HLS 直播源;节目编排可能与亚洲版不同。"
DEFAULT_TV_SETTINGS = {
"default_source_id": DEFAULT_TV_SOURCE_ID,
@@ -22,20 +28,21 @@ DEFAULT_TV_SETTINGS = {
"sources": [
{
"id": "cctv4",
"name": "CCTV-4 中文国际",
"name": "CCTV-4 中文国际(美洲版)",
"provider": "CCTV",
"region": "China",
"language": "zh-CN",
"source_type": "hls",
"embed_url": "https://tv.cctv.com/live/cctv4/",
"stream_url": "https://ldocctvwbcdtxy.liveplay.myqcloud.com/ldocctvwbcd/cdrmldcctv4_1_td.m3u8",
"homepage_url": "https://tv.cctv.com/live/cctv4/",
"embed_url": "",
"stream_url": CCTV4_STREAM,
"use_proxy": False,
"homepage_url": CCTV4_HOMEPAGE,
"poster_url": "",
"is_enabled": True,
"is_fallback": True,
"sort_order": 10,
"collector_source": None,
"notes": "默认兜底新闻直播源。优先尝试 CCTV-4 官方 HLS 播放流,若直播放失败则回退到央视官网直播页。",
"notes": CCTV4_PLAYBACK_NOTE,
},
{
"id": "reuters-tv",
@@ -237,6 +244,17 @@ def _clean_int(value: Any, *, default: int) -> int:
def normalize_tv_source(source: dict[str, Any] | None, *, index: int = 0) -> dict[str, Any]:
payload = dict(source or {})
# Retire only the formerly bundled CDRM URL, including saved copies of that default.
# HTTP 200 and valid TS framing do not mean this stream is decodable by hls.js.
if _clean_text(payload.get("id")) == "cctv4" and _clean_url(
payload.get("stream_url")
) == UNSUPPORTED_CCTV4_STREAM:
payload.update(
name="CCTV-4 中文国际(美洲版)",
source_type="hls", stream_url=CCTV4_STREAM, embed_url="",
use_proxy=False,
homepage_url=CCTV4_HOMEPAGE, notes=CCTV4_PLAYBACK_NOTE,
)
source_id = _clean_text(payload.get("id")) or f"tv-source-{index + 1}"
source_type = _clean_text(payload.get("source_type")).lower()
youtube_video_id = _clean_text(payload.get("youtube_video_id"))
@@ -259,6 +277,7 @@ def normalize_tv_source(source: dict[str, Any] | None, *, index: int = 0) -> dic
"source_type": source_type,
"embed_url": _clean_url(payload.get("embed_url")),
"stream_url": _clean_url(payload.get("stream_url")),
"use_proxy": _clean_bool(payload.get("use_proxy"), default=True),
"homepage_url": _clean_url(payload.get("homepage_url")),
"poster_url": _clean_url(payload.get("poster_url")),
"youtube_video_id": youtube_video_id,
@@ -300,8 +319,12 @@ def normalize_tv_settings(payload: dict[str, Any] | None) -> dict[str, Any]:
]
if not any(source["id"] == DEFAULT_TV_SOURCE_ID for source in normalized_sources):
default_source = next(
source for source in DEFAULT_TV_SETTINGS["sources"]
if source["id"] == DEFAULT_TV_SOURCE_ID
)
normalized_sources.append(
normalize_tv_source(DEFAULT_TV_SETTINGS["sources"][0], index=len(normalized_sources))
normalize_tv_source(default_source, index=len(normalized_sources))
)
default_source_exists = any(

View File

@@ -614,6 +614,18 @@ async def get_current_vessels_snapshot(
return [item.to_dict() for item in result.scalars().all()]
async def get_current_vessels_by_mmsi(
db: AsyncSession, mmsis: list[int]
) -> list[dict[str, Any]]:
"""Read canonical render state for the vessels changed by a stream flush."""
if not mmsis:
return []
result = await db.execute(
select(VesselCurrentState).where(VesselCurrentState.mmsi.in_(mmsis))
)
return [_jsonable(item.to_dict()) for item in result.scalars().all()]
async def aggregate_vessel_observations(
db: AsyncSession,
observations: Iterable[AISRawObservation],

View File

@@ -0,0 +1,72 @@
from datetime import UTC, datetime
from unittest.mock import AsyncMock, MagicMock
import httpx
import pytest
from fastapi import FastAPI
from pydantic import ValidationError
from app.api.v1.auth import router
from app.core import security
from app.db.session import DEFAULT_LOGIN_USERS, get_db
from app.models.user import User
from app.schemas.user import UserRegister
async def identity_request(monkeypatch, user, token=None):
result = MagicMock()
result.scalar_one_or_none.return_value = user
db = AsyncMock()
db.execute.return_value = result
monkeypatch.setattr(security.redis_client, "sismember", lambda *_: False)
app = FastAPI()
app.include_router(router, prefix="/auth")
async def override_db():
yield db
app.dependency_overrides[get_db] = override_db
async with httpx.AsyncClient(
transport=httpx.ASGITransport(app=app), base_url="http://test"
) as client:
return await client.get(
"/auth/me",
headers={"Authorization": "Bearer " + (token or security.create_access_token({"sub": 1}))},
)
@pytest.mark.asyncio
@pytest.mark.parametrize("default", DEFAULT_LOGIN_USERS)
async def test_default_identity_preserves_profile_fields(monkeypatch, default):
created = datetime(2026, 9, 21, tzinfo=UTC)
user = User(
id=1, username=default["username"], email=default["email"],
role=default["role"], password_hash="not-exposed", is_active=True,
email_verified=True, gatekeeper_groups=["ops"], created_at=created,
)
response = await identity_request(monkeypatch, user)
assert response.status_code == 200
payload = response.json()
assert payload["email"] == default["email"]
assert payload["email_verified"] is True
assert datetime.fromisoformat(payload["created_at"].replace("Z", "+00:00")) == created
assert payload["gatekeeper_groups"] == ["ops"]
assert "password_hash" not in payload
@pytest.mark.asyncio
@pytest.mark.parametrize("user", [None, User(id=1, is_active=False)])
async def test_missing_or_inactive_identity_is_rejected(monkeypatch, user):
response = await identity_request(monkeypatch, user)
assert response.status_code == 401
@pytest.mark.asyncio
async def test_refresh_token_is_not_an_access_identity(monkeypatch):
response = await identity_request(monkeypatch, None, security.create_refresh_token({"sub": 1}))
assert response.status_code == 401
def test_registration_still_validates_email():
with pytest.raises(ValidationError):
UserRegister(username="newuser", email="newuser@planet.local", password="example-password")

View File

@@ -79,7 +79,8 @@ class TestTOP500Collector:
"""Test parsing empty response"""
collector = TOP500Collector()
result = collector.parse_response("<html><body><table></table></body></html>")
assert len(result) > 0
assert result == []
assert collector.fail_on_empty
def test_parse_response_single_item(self):
"""Test parsing single item response"""
@@ -97,9 +98,9 @@ class TestTOP500Collector:
</tr>
</table>
"""
result = collector.parse_response(response)
result = collector.parse_response(response, "https://top500.org/lists/top500/2025/11/")
assert len(result) == 1
assert result[0]["source_id"] == "top500_1"
assert result[0]["source_id"] == "top500_system_1"
assert result[0]["name"] == "Test Supercomputer"
assert result[0]["country"] == "USA"
assert result[0]["metadata"]["rank"] == 1
@@ -113,7 +114,7 @@ class TestTOP500Collector:
<tr><th>Rank</th><th>System</th><th>Cores</th><th>Rmax</th><th>Rpeak</th><th>Power</th></tr>
<tr>
<td>1</td>
<td>Valid\nVendor\nSite\nUSA</td>
<td><a href="/system/2/">Valid</a>\nVendor\nSite\nUSA</td>
<td>1000</td>
<td>10 PFLOP/s</td>
<td>12 PFLOP/s</td>
@@ -129,7 +130,7 @@ class TestTOP500Collector:
</tr>
</table>
"""
result = collector.parse_response(response)
result = collector.parse_response(response, "https://top500.org/lists/top500/2025/11/")
assert len(result) == 1
assert result[0]["name"] == "Valid"

View File

@@ -0,0 +1,222 @@
from copy import deepcopy
from types import SimpleNamespace
from unittest.mock import AsyncMock, MagicMock
import pytest
from app.api.v1.compute_intelligence import configuration_admin, get_series, router
from app.core.security import get_current_user
from app.db.session import get_db
from app.models.system_log import AuditLog
from app.schemas.compute_configuration import ComputeConfiguration
from app.services.compute_configuration import (
DEFAULT_CONFIGURATION,
default_envelope,
publish_configuration,
read_configuration,
)
from app.services.compute_history import compute_history
from fastapi import FastAPI, HTTPException, Response
from httpx import ASGITransport, AsyncClient
from pydantic import ValidationError
def default_config():
return DEFAULT_CONFIGURATION.model_dump(mode="json")
@pytest.mark.parametrize(
"mutation",
[
lambda value: value["observations"][0]["values"].update(CN=101),
lambda value: value["observations"][0]["values"].update(CN=-1),
lambda value: value["observations"][0]["values"].update(CN=float("nan")),
lambda value: value["observations"][0]["values"].update(CN=True),
lambda value: value["observations"][0]["values"].update(EUROPE=10),
lambda value: value["observations"][0].update(source_id="unknown"),
lambda value: value["observations"][0].update(year=2026),
lambda value: value["observations"].append(value["observations"][0]),
lambda value: value["sources"].append(value["sources"][0]),
lambda value: value["innovation"].append(value["innovation"][0]),
lambda value: value["sources"][0].update(url="javascript:alert(1)"),
lambda value: value["sources"][0].update(url="https://user:pass@example.com/report"),
],
)
def test_rejects_invalid_or_ambiguous_evidence(mutation):
payload = default_config()
mutation(payload)
with pytest.raises(ValidationError):
ComputeConfiguration.model_validate(payload)
def test_arbitrary_report_updates_values_sources_and_years_without_code_changes():
payload = default_config()
payload["sources"].append(
{
"id": "new-report",
"title": "New report",
"url": "https://example.com/report",
"publisher": "Research group",
"report_year": 2026,
"locator": "Figure 2",
}
)
payload["observations"].append(
{
"year": 2025,
"metric": "global_share",
"source_id": "new-report",
"values": {"CN": 25, "US": 50, "JP": 0},
}
)
config = ComputeConfiguration.model_validate(payload)
result = compute_history(["CN", "US", "JP", "EUROPE"], config=config)
row = next(row for row in result["compute"] if row["year"] == 2025)
assert row["values"] == {"CN": 25, "US": 50, "JP": 0, "EUROPE": None}
assert result["evidence"][row["evidence_id"]]["title"] == "New report"
assert DEFAULT_CONFIGURATION.model_dump(mode="json") == default_config()
def test_zero_bilateral_denominator_is_missing_and_empty_override_stays_empty():
payload = default_config()
payload["observations"][0]["values"] = {"CN": 0, "US": 0}
result = compute_history(
["CN", "US"], "bilateral_share", ComputeConfiguration.model_validate(payload)
)
assert result["compute"][0]["values"] == {"CN": None, "US": None}
payload.update(sources=[], observations=[], innovation=[])
result = compute_history(["CN"], config=ComputeConfiguration.model_validate(payload))
assert all(row["values"]["CN"] is None for row in result["compute"])
assert result["evidence"] == {}
def fake_db(record=None):
db = AsyncMock()
db.add = MagicMock()
result = MagicMock()
result.scalar_one_or_none.return_value = record
result.scalar_one.return_value = record
db.execute.return_value = result
return db
@pytest.mark.asyncio
async def test_fresh_install_uses_defaults_without_writing_database():
db = fake_db()
assert await read_configuration(db) == default_envelope()
db.add.assert_not_called()
db.commit.assert_not_awaited()
@pytest.mark.asyncio
async def test_series_reads_persisted_override_and_disables_http_cache():
envelope = default_envelope()
envelope["is_default"] = False
envelope["revision"] = "custom"
envelope["config"]["observations"][3]["values"]["CN"] = 30
response = Response()
data = await get_series(
response, "CN,US", "global_share", fake_db(SimpleNamespace(payload=envelope))
)
assert data["compute"][3]["values"]["CN"] == 30
assert data["version"] == "custom"
assert data["provenance"] == "admin_curated_reports"
assert response.headers["Cache-Control"] == "no-store"
@pytest.mark.asyncio
async def test_publish_and_audit_are_one_transaction_and_conflicts_do_not_overwrite():
record = SimpleNamespace(payload=default_envelope())
db = fake_db(record)
actor = SimpleNamespace(id=1, username="admin")
old = deepcopy(record.payload)
result = await publish_configuration(
db,
revision=old["revision"],
config=DEFAULT_CONFIGURATION,
actor=actor,
)
assert result["revision"] != old["revision"]
audit = db.add.call_args.args[0]
assert isinstance(audit, AuditLog)
assert audit.details == {"before": old, "after": result}
assert audit.actor_id == actor.id
db.commit.assert_awaited_once()
with pytest.raises(HTTPException) as failure:
await publish_configuration(
db, revision=old["revision"], config=DEFAULT_CONFIGURATION, actor=actor
)
assert failure.value.status_code == 409
assert record.payload == result
db.rollback.assert_awaited_once()
db.commit.assert_awaited_once()
@pytest.mark.asyncio
async def test_reset_changes_revision_and_restores_bundled_data():
record = SimpleNamespace(
payload={**default_envelope(), "revision": "custom", "is_default": False}
)
result = await publish_configuration(
fake_db(record),
revision="custom",
config=DEFAULT_CONFIGURATION,
actor=SimpleNamespace(id=1, username="admin"),
reset=True,
)
assert result["is_default"] is True
assert result["config"] == default_config()
assert result["revision"] != "custom"
@pytest.mark.asyncio
@pytest.mark.parametrize("role", ["user", "viewer", "operator"])
async def test_non_admin_cannot_configure(role):
with pytest.raises(HTTPException) as failure:
await configuration_admin(SimpleNamespace(role=role))
assert failure.value.status_code == 403
@pytest.mark.asyncio
async def test_http_configuration_auth_validation_and_public_series():
app = FastAPI()
app.include_router(router)
record = SimpleNamespace(payload=default_envelope())
db = fake_db(record)
app.dependency_overrides[get_db] = lambda: db
async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
assert (await client.get("/configuration")).status_code == 401
assert (await client.get("/series")).status_code == 200
app.dependency_overrides[get_current_user] = lambda: SimpleNamespace(
id=1,
username="admin",
role="admin",
)
loaded = (await client.get("/configuration")).json()
invalid = deepcopy(loaded["config"])
invalid["observations"][0]["values"]["CN"] = 200
response = await client.put(
"/configuration",
json={
"revision": loaded["revision"],
"config": invalid,
},
)
assert response.status_code == 422
db.commit.assert_not_awaited()
response = await client.put(
"/configuration",
json={
"revision": loaded["revision"],
"config": loaded["config"],
},
)
assert response.status_code == 200
assert response.json()["revision"] != loaded["revision"]
assert (
await client.post(
"/configuration/reset",
json={
"revision": loaded["revision"],
},
)
).status_code == 409

View File

@@ -0,0 +1,268 @@
import csv
import io
from datetime import UTC, datetime
from unittest.mock import AsyncMock, MagicMock
import pytest
from fastapi import HTTPException
from sqlalchemy.dialects import postgresql
from app.api.v1.compute_intelligence import sample_query, selected_countries
from app.services.collectors.epoch_ai import EpochAIGPUCollector
from app.services.collectors.top500 import TOP500Collector
from app.services.compute_history import compute_history
def epoch_csv(**overrides):
row = {
"Name": "Cluster Alpha",
"Country": "United States of America",
"Status": "Existing",
"First Operational Date": "2023-05-02",
"16-bit OP/s (log)": "18",
"Location": "3231 Riverport Rd, Memphis, TN 38109",
"latitude": "35",
"longitude": "-90",
**overrides,
}
output = io.StringIO()
writer = csv.DictWriter(output, fieldnames=list(row))
writer.writeheader()
writer.writerow(row)
return output.getvalue()
def test_epoch_csv_preserves_precision_date_and_missing_values():
collector = EpochAIGPUCollector()
row = collector.parse_response(epoch_csv())[0]
assert row["value"] == 1e18
assert row["unit"] == "FLOP/s"
assert row["metadata"]["precision"] == "FP16"
assert row["metadata"]["location_precision"] == "precise"
assert row["metadata"]["raw_country"] == "United States of America"
assert row["reference_date"] == "2023-05-02"
missing = collector.parse_response(
epoch_csv(**{"16-bit OP/s (log)": "", "First Operational Date": ""})
)[0]
assert missing["value"] is None
assert missing["reference_date"] is None
@pytest.mark.parametrize(
("location", "country", "expected"),
[
("United States of America", "United States of America", "country"),
("Japan", "Japan", "country"),
("Korea (Republic of)", "South Korea", "country"),
("Texas", "United States of America", "region"),
("Iowa, USA", "United States of America", "region"),
("Seoul", "South Korea", "city"),
("Paris, France", "France", "city"),
("Skolkovo Innovation Center\nMoscow Oblast\nRussia", "Russia", "site"),
("Cluster 2, Texas", "United States of America", "site"),
("Campus 7, Tokyo", "Japan", "site"),
("Building 240, Argonne", "United States of America", "site"),
("7", "United States of America", "unknown"),
("Jülich Supercomputing Centre\n52428 Jülich, Germany", "Germany", "precise"),
("35.380924,139.625008", "Japan", "precise"),
('33°24\'28.7"N 112°21\'54.3"W', "United States of America", "precise"),
("latitude: 35.0 longitude: 181.0", "United States of America", "unknown"),
('99°99\'00"N 200°00\'00"W', "United States of America", "unknown"),
("3231 Riverport Rd, Memphis, TN 38109", "United States of America", "precise"),
],
)
def test_epoch_location_precision_follows_location_field(location, country, expected):
row = EpochAIGPUCollector().parse_response(
epoch_csv(Location=location, Country=country)
)[0]
assert row["metadata"]["location_precision"] == expected
@pytest.mark.parametrize(
("location", "expected"),
[
({"lat": 35.0, "lng": 139.0}, "precise"),
({"latitude": 35.0, "longitude": 181.0}, "unknown"),
({"lat": 35.0}, "unknown"),
({"lng": 139.0}, "unknown"),
({"lat": "invalid", "lng": 139.0}, "unknown"),
],
)
def test_epoch_coordinate_object_requires_complete_valid_pair(location, expected):
from app.services.epoch_location import assess_epoch_location
assert assess_epoch_location(location).precision == expected
def test_epoch_anonymous_rows_are_not_geocoded():
row = EpochAIGPUCollector().parse_response(epoch_csv(Name="Anonymized 1", Country="China"))[0]
assert row["latitude"] is None and row["longitude"] is None
assert row["metadata"]["anonymous"] and row["metadata"]["rounded"]
@pytest.mark.parametrize(
"payload",
["<html>blocked</html>", "Name,Country\nA,US", epoch_csv(**{"16-bit OP/s (log)": "nan"})],
)
def test_epoch_invalid_payload_fails_closed(payload):
with pytest.raises(ValueError):
EpochAIGPUCollector().parse_response(payload)
def test_epoch_duplicate_identity_rejected():
payload = epoch_csv()
with pytest.raises(ValueError, match="identity collision"):
EpochAIGPUCollector().parse_response(payload + payload.splitlines()[1] + "\n")
def test_anonymous_source_rows_preserve_multiplicity_without_claiming_identity():
payload = epoch_csv(Name="Anonymized Chinese System", Country="China")
rows = EpochAIGPUCollector().parse_response(payload + payload.splitlines()[1] + "\n")
assert len(rows) == 2
assert rows[0]["source_id"] != rows[1]["source_id"]
assert all(row["metadata"]["identity_scope"] == "snapshot_record" for row in rows)
def test_top500_units_and_stable_identity():
collector = TOP500Collector()
html = '<table id="top500"><tr><th>Rank</th><th>System</th><th>Cores</th><th>Rmax [TFlop/s]</th><th>Rpeak [PFlop/s]</th><th>Power</th></tr><tr><td>{rank}</td><td><a href="/system/123/">Test</a><br>Lab<br>USA</td><td>10</td><td>1,500</td><td>2</td><td>1</td></tr></table>'
first = collector.parse_response(
html.format(rank=1), "https://top500.org/lists/top500/2025/11/"
)[0]
second = collector.parse_response(
html.format(rank=2), "https://top500.org/lists/top500/2026/06/"
)[0]
assert first["source_id"] == second["source_id"] == "top500_system_123"
assert float(first["value"]) == 1.5e15
assert first["metadata"]["rpeak_flops"] == 2e15
assert first["reference_date"] == "2025-11-01"
assert second["reference_date"] == "2026-06-01"
with pytest.raises(ValueError, match="date missing"):
collector.parse_response(html.format(rank=1))
@pytest.mark.asyncio
async def test_large_source_loss_does_not_create_snapshot():
db = AsyncMock()
db.scalar.return_value = 500
db.add = MagicMock()
with pytest.raises(ValueError, match="40%"):
await TOP500Collector()._create_snapshot(db, 1, [{}] * 2, datetime.now(UTC))
db.add.assert_not_called()
db.commit.assert_not_awaited()
def test_history_missing_years_and_denominators():
global_data = compute_history(["CN", "US"])
assert global_data["compute"][3]["values"] == {"CN": 31, "US": 41}
assert global_data["compute"][-1]["values"] == {"CN": None, "US": None}
bilateral = compute_history(["CN", "US"], "bilateral_share")
assert sum(bilateral["compute"][3]["values"].values()) == 100
assert bilateral["denominator"] == "US+CN"
absolute = compute_history(["US", "CN"], "absolute")
assert absolute["compute"][3]["values"] == {"US": None, "CN": 435}
assert global_data["innovation"][-2]["value"] == 17.95
assert global_data["innovation"][-1]["value"] is None
assert global_data["evidence"]["innovation-2026"]["published_at"] == "2026-07-17"
@pytest.mark.asyncio
async def test_snapshot_query_uses_observation_cutoff_not_location():
query = await sample_query(AsyncMock(), ["CN"], 2023)
sql = str(query.compile(dialect=postgresql.dialect(), compile_kwargs={"literal_binds": True}))
assert "2024-01-01" in sql
assert "row_number() OVER (PARTITION BY" in sql
assert "compute_contract" in sql and "中国" in sql
assert "latitude" not in sql and "longitude" not in sql
def test_country_input_is_bounded_and_deduplicated():
assert selected_countries("CN,CN,US") == ["CN", "US"]
assert selected_countries("CN,US,EUROPE,APAC,OTHER")[-1] == "OTHER"
with pytest.raises(HTTPException):
selected_countries("invalid")
with pytest.raises(HTTPException, match="overlapping"):
selected_countries("EUROPE,DE")
def test_catalog_supports_global_participants_without_inventing_history():
from app.services.compute_catalog import (
CATALOG,
COMPARISON_GROUPS,
COUNTRIES,
source_country_names,
source_country_participants,
)
assert {"CN", "US", "JP", "KR", "IN", "DE", "FR", "GB", "AU", "BR", "ZA"}.issubset(
{entry["code"] for entry in CATALOG}
)
assert not {"TW", "HK", "MO", "XK"}.intersection(entry["code"] for entry in CATALOG)
owners = source_country_names(["CN"])
assert {"中国", "中国(台湾)", "中国(香港)", "中国(澳门)"}.issubset(owners)
assert source_country_participants(["CN"])["中国(台湾)"] == "CN"
assert [entry["code"] for entry in CATALOG[:5]] == ["CN", "US", "EUROPE", "APAC", "OTHER"]
assert [entry["kind"] for entry in CATALOG[:5]] == [
"country",
"country",
"region",
"region",
"region",
]
group_members = [
member
for _code, _name, _name_en, members, _anchor in COMPARISON_GROUPS
for member in members
]
assert len(group_members) == len(set(group_members))
assert set(group_members) == set(COUNTRIES)
data = compute_history(["JP", "KR"])
assert [row["values"]["JP"] for row in data["compute"][:4]] == [6, 5, 4, 4]
assert all(row["values"]["KR"] is None for row in data["compute"])
assert all(row["value"] is None for row in data["innovation"])
assert len(data["catalog"]) > 30
def test_region_history_requires_complete_same_year_member_coverage():
data = compute_history(["CN", "US", "EUROPE", "APAC", "OTHER"])
row_2023 = next(row for row in data["compute"] if row["year"] == 2023)
assert row_2023["values"]["CN"] == 31
assert row_2023["values"]["US"] == 41
assert row_2023["values"]["EUROPE"] is None
assert row_2023["values"]["APAC"] is None
assert row_2023["values"]["OTHER"] is None
assert row_2023["coverage"]["EUROPE"]["status"] == "incomplete"
assert "DE" in row_2023["coverage"]["EUROPE"]["missing_members"]
assert data["default_participants"] == ["CN", "US", "EUROPE", "APAC", "OTHER"]
def test_anonymous_records_never_enter_the_location_repair_queue():
from app.api.v1.visualization import convert_compute_centers_to_geojson
from app.models.collected_data import CollectedData
record = CollectedData(
source="epoch_ai_gpu",
source_id="anonymous",
name="Anonymous",
extra_data={"compute_contract": "v1", "anonymous": True},
)
result = convert_compute_centers_to_geojson([record])
assert result["features"] == []
assert result["unresolved"] == []
assert result["withheld_count"] == 1
def test_absolute_report_year_mapping_and_bilateral_denominator_ignore_other_countries():
absolute = compute_history(["CN"], "absolute")
observed = {row["year"]: row for row in absolute["compute"]}
assert observed[2022]["values"]["CN"] == 302
assert observed[2022]["evidence_id"] == "caict-2023"
assert observed[2023]["evidence_id"] == "caict-2024"
bilateral = compute_history(["CN", "US", "JP"], "bilateral_share")
values = next(row["values"] for row in bilateral["compute"] if row["year"] == 2023)
assert values == {"CN": 43.06, "US": 56.94, "JP": None}
for row in absolute["compute"]:
if row["evidence_id"]:
report = absolute["evidence"][row["evidence_id"]]
assert report["url"].startswith("https://www.caict.ac.cn/")

View File

@@ -0,0 +1,18 @@
from app.services.compute_center_verified_sites import get_verified_compute_site
def test_registry_requires_exact_facility_identity_not_operator_name():
assert get_verified_compute_site("epoch_ai_gpu", "NVIDIA") is None
assert get_verified_compute_site("top500", "epoch_argonne_nl_sophia__formerly_theta_") is None
def test_official_campus_anchors_preserve_evidence_and_site_level_precision():
site = get_verified_compute_site("epoch_ai_gpu", "epoch_jamstec_zettascaler_2_2_gyoukou")
assert site["city"] == "Yokohama"
assert site["precision"] == "site"
assert site["coordinate_basis"] == "official_access_map_center"
assert site["sources"]
site["latitude"] = 0
assert get_verified_compute_site(
"epoch_ai_gpu", "epoch_jamstec_zettascaler_2_2_gyoukou"
)["latitude"] != 0

View File

@@ -1,4 +1,5 @@
import json
from pathlib import Path
import pytest
@@ -8,6 +9,10 @@ from app.services import earth_boundaries
def write_geojson(path, name="Test"):
names = [name]
if name == "earth_admin0_boundaries":
names = ["China", "India", "Japan", "Palestine", "Taiwan", "Hong Kong", "Macao", "Serbia", "Kosovo"]
codes = {"China": "CHN", "India": "IND", "Japan": "JPN", "Palestine": "PSE", "Taiwan": "TWN", "Hong Kong": "HKG", "Macao": "MAC", "Serbia": "SRB", "Kosovo": "KOS"}
path.write_text(
json.dumps(
{
@@ -15,12 +20,17 @@ def write_geojson(path, name="Test"):
"features": [
{
"type": "Feature",
"properties": {"name": name},
"properties": {
"NAME": f"source renamed {item}" if item in {"Taiwan", "Kosovo"} else item,
"ISO_A3": "-99" if item == "Kosovo" else codes.get(item, "TEST"),
"ADM0_A3": "XKS" if item == "Kosovo" else codes.get(item, "TEST"),
},
"geometry": {
"type": "Polygon",
"coordinates": [[[-1, -1], [1, -1], [1, 1], [-1, 1], [-1, -1]]],
},
}
for item in names
],
}
),
@@ -37,9 +47,14 @@ def patch_paths(monkeypatch, tmp_path):
config = repo / "config/earth-boundary-sources.local.json"
example = repo / "config/earth-boundary-sources.example.json"
policy = repo / "config/earth-boundary-pov-policy.china-v1.json"
geometry = repo / "resources/earth/boundary-pov-geometry.china-v1.json"
for path in (source_dir, boundary_dir, pmtiles.parent, legacy.parent, config.parent):
path.mkdir(parents=True, exist_ok=True)
policy.write_text('{"productionTileFormat":"pmtiles+mvt"}\n', encoding="utf-8")
source_policy = Path(__file__).resolve().parents[2] / "config/earth-boundary-pov-policy.china-v1.json"
policy.write_text(source_policy.read_text(encoding="utf-8"), encoding="utf-8")
geometry.parent.mkdir(parents=True, exist_ok=True)
source_geometry = Path(__file__).resolve().parents[2] / "resources/earth/boundary-pov-geometry.china-v1.json"
geometry.write_bytes(source_geometry.read_bytes())
example.write_text('{"collectorConfigs":{}}\n', encoding="utf-8")
monkeypatch.setattr(earth_boundaries, "REPO_ROOT", repo)
monkeypatch.setattr(earth_boundaries, "SOURCE_OUTPUT_DIR", source_dir)
@@ -49,10 +64,11 @@ def patch_paths(monkeypatch, tmp_path):
monkeypatch.setattr(earth_boundaries, "BOUNDARY_OUTPUT_DIR", boundary_dir)
monkeypatch.setattr(earth_boundaries, "BOUNDARY_MANIFEST_PATH", boundary_dir / "manifest.json")
monkeypatch.setattr(earth_boundaries, "PMTILES_ARTIFACT_PATH", pmtiles)
monkeypatch.setattr(earth_boundaries, "LEGACY_GEOJSON_PATH", legacy)
monkeypatch.setattr(earth_boundaries, "POLICY_FALLBACK_PATH", legacy)
monkeypatch.setattr(earth_boundaries, "LOCAL_CONFIG_PATH", config)
monkeypatch.setattr(earth_boundaries, "EXAMPLE_CONFIG_PATH", example)
monkeypatch.setattr(earth_boundaries, "POV_POLICY_PATH", policy)
monkeypatch.setattr(earth_boundaries, "POV_GEOMETRY_PATH", geometry)
return {
"repo": repo,
"config": config,
@@ -62,27 +78,73 @@ def patch_paths(monkeypatch, tmp_path):
}
def write_fallback(path, policy_path, geometry_path):
policy = earth_boundaries._read_json(policy_path)
geometry = earth_boundaries._read_json(geometry_path)
coverage = {
"profile": "china-pov-v1",
"sha256": earth_boundaries._stable_json_hash(policy),
"geometrySha256": earth_boundaries._stable_json_hash(geometry),
"reviewedGeometrySourceCommit": geometry["sourceCommit"],
"applied": True,
"appliedRuleIds": ["china-taiwan-penghu", "china-hong-kong", "china-macao", "kosovo"],
}
path.write_text(json.dumps({"type": "FeatureCollection", "features": [], "povPolicy": coverage}), encoding="utf-8")
def test_boundary_status_uses_legacy_provider_when_pmtiles_missing(monkeypatch, tmp_path):
paths = patch_paths(monkeypatch, tmp_path)
write_fallback(paths["legacy"], earth_boundaries.POV_POLICY_PATH, earth_boundaries.POV_GEOMETRY_PATH)
status = earth_boundaries.get_boundary_status()
assert status["provider"] == "policy-geojson-fallback"
assert status["fallback_available"] is True
assert status["high_precision_ready"] is False
def test_boundary_status_rejects_unverified_fallback(monkeypatch, tmp_path):
paths = patch_paths(monkeypatch, tmp_path)
write_geojson(paths["legacy"])
status = earth_boundaries.get_boundary_status()
assert status["provider"] == "legacy-geojson"
assert status["fallback_available"] is True
assert status["high_precision_ready"] is False
assert status["provider"] == "missing"
assert status["fallback_available"] is False
def test_boundary_status_prefers_high_precision_when_manifest_and_pmtiles_exist(monkeypatch, tmp_path):
paths = patch_paths(monkeypatch, tmp_path)
write_geojson(paths["legacy"])
write_fallback(paths["legacy"], earth_boundaries.POV_POLICY_PATH, earth_boundaries.POV_GEOMETRY_PATH)
paths["pmtiles"].write_bytes(b"pmtiles")
paths["manifest"].write_text('{"tileProvider":"pmtiles-mvt"}\n', encoding="utf-8")
(paths["manifest"].parent / "hover-index.geojson").write_text("{}", encoding="utf-8")
coverage = {
"profile": "china-pov-v1",
"sha256": earth_boundaries._stable_json_hash(earth_boundaries._read_json(earth_boundaries.POV_POLICY_PATH)),
"applied": True,
"appliedRuleIds": ["china-taiwan-penghu", "china-hong-kong", "china-macao", "kosovo"],
"reviewedGeometrySourceCommit": earth_boundaries._read_json(earth_boundaries.POV_GEOMETRY_PATH)["sourceCommit"],
"geometrySha256": earth_boundaries._stable_json_hash(earth_boundaries._read_json(earth_boundaries.POV_GEOMETRY_PATH)),
}
paths["manifest"].write_text(json.dumps({"tileProvider": "pmtiles-mvt", "hoverIndex": "hover-index.geojson", "povPolicy": coverage}), encoding="utf-8")
status = earth_boundaries.get_boundary_status()
assert status["provider"] == "pmtiles-mvt"
assert status["high_precision_ready"] is True
assert status["policy_publishable"] is False
def test_boundary_status_rejects_unverified_high_precision(monkeypatch, tmp_path):
paths = patch_paths(monkeypatch, tmp_path)
write_fallback(paths["legacy"], earth_boundaries.POV_POLICY_PATH, earth_boundaries.POV_GEOMETRY_PATH)
paths["pmtiles"].write_bytes(b"pmtiles")
paths["manifest"].write_text('{"tileProvider":"pmtiles-mvt","povPolicy":{"applied":false}}', encoding="utf-8")
status = earth_boundaries.get_boundary_status()
assert status["provider"] == "policy-geojson-fallback"
assert status["high_precision_ready"] is False
def test_save_boundary_config_writes_local_config(monkeypatch, tmp_path):
@@ -130,6 +192,22 @@ async def test_build_reports_missing_tools_after_source_artifacts(monkeypatch, t
assert status["high_precision_ready"] is True
assert (paths["repo"] / "data/earth-boundary-sources/manifest.json").exists()
assert paths["manifest"].exists()
manifest = json.loads(paths["manifest"].read_text(encoding="utf-8"))
assert manifest["povPolicy"]["applied"] is True
assert manifest["povPolicy"]["coverage"] == "partial"
assert manifest["povPolicy"]["publicationReady"] is False
assert "china-south-china-sea-islands" in manifest["povPolicy"]["unresolvedRuleIds"]
hover = json.loads((paths["manifest"].parent / manifest["hoverIndex"]).read_text(encoding="utf-8"))
by_rule = {
feature["properties"].get("PLANET_POLICY_RULE"): feature
for feature in hover["features"]
}
assert by_rule["china-hong-kong"]["properties"]["ISO_A3"] == "CHN"
assert by_rule["china-macao"]["properties"]["ISO_A3"] == "CHN"
assert all(
feature["properties"].get("ADM0_A3") not in {"TWN", "KOS", "XKS"}
for feature in hover["features"]
)
assert any(phase == "download" for _progress, phase, _message in progress_events)
@@ -175,3 +253,23 @@ def test_earth_boundary_collectors_are_not_registered_as_datasources():
assert removed.isdisjoint(DEFAULT_DATASOURCES)
for source in removed:
assert collector_registry.get(source) is None
def test_static_policy_fallback_contains_reviewed_jeju_polygon():
repo = Path(__file__).resolve().parents[2]
path = repo / "frontend/public/earth/data/countries-admin0.china-pov-v1.geojson"
payload = json.loads(path.read_text(encoding="utf-8"))
matches = [
feature for feature in payload["features"]
if feature["properties"].get("PLANET_GEOMETRY_SUPPLEMENT") == "jeju-island"
]
assert len(matches) == 1
jeju = matches[0]
assert jeju["properties"]["ISO_A3"] == "KOR"
assert jeju["properties"]["PLANET_SOURCE_NE_ID"] == 1159320985
assert payload["povPolicy"]["geometrySupplement"]["jejuComponent"] == 3
ring = jeju["geometry"]["coordinates"][0]
assert len(ring) == 61
assert min(point[0] for point in ring) < 126.53 < max(point[0] for point in ring)
assert min(point[1] for point in ring) < 33.36 < max(point[1] for point in ring)

View File

@@ -63,6 +63,21 @@ def test_build_earth_update_maps_derived_tables_to_layers():
assert vessel_update is not None
assert vessel_update["source"] == "vessel_position"
assert vessel_update["layers"] == ["vessels"]
assert vessel_update["refresh_strategy"] == "reload"
def test_vessel_stream_changes_do_not_request_full_layer_rebuilds():
from app.services.earth_db_change_listener import PendingEarthDbChange
for source in ("aisstream_vessels", "barentswatch_vessels"):
update = build_earth_update_from_db_payload({
"source": source, "table": "vessel_current_state", "operation": "UPDATE",
})
assert update["refresh_strategy"] == "delta"
pending = PendingEarthDbChange(source=source, layers=["vessels"], refresh_strategy="delta")
pending.add({"operation": "UPDATE"})
pending.add({"operation": "DELETE"})
assert pending.refresh_strategy == "reload"
def test_build_earth_update_maps_interactable_delete_to_delta():

View File

@@ -0,0 +1,41 @@
import json
from pathlib import Path
import pytest
from app.core.earth_geography_policy import display_country, display_iso_a3
def test_missing_deployment_policy_has_an_actionable_error(monkeypatch, tmp_path):
from app.core import earth_geography_policy as policy
policy._policy_country_by_code.cache_clear()
monkeypatch.setattr(policy, "_POLICY_PATH", tmp_path / "missing-policy.json")
try:
with pytest.raises(FileNotFoundError, match="EARTH_DISPLAY_POLICY_MISSING"):
policy.display_country("China")
finally:
policy._policy_country_by_code.cache_clear()
def test_display_country_groups_china_regions_and_kosovo() -> None:
for source in ("Taiwan", "TWN", "Hong Kong", "HKG", "Macao", "MAC", "中国(台湾)"):
assert display_country(source) == "中国"
assert display_iso_a3(source) == "CHN"
for source in ("Kosovo", "XK", "SRB"):
assert display_country(source) == "塞尔维亚"
assert display_iso_a3(source) == "SRB"
def test_display_country_keeps_other_country_and_rejects_unknown() -> None:
assert display_country("Japan") == "日本"
assert display_iso_a3("Japan") == "JPN"
assert display_country("unexpected non-country") is None
def test_country_projection_agrees_with_versioned_boundary_rules() -> None:
path = Path(__file__).resolve().parents[2] / "config/earth-boundary-pov-policy.china-v1.json"
policy = json.loads(path.read_text(encoding="utf-8"))
for rule in policy["rules"]:
for code in rule.get("sourceIsoA3", []):
assert display_iso_a3(code) == rule.get("targetIsoA3", rule.get("parentIsoA3"))

View File

@@ -0,0 +1,219 @@
from copy import deepcopy
import httpx
import pytest
from app.services import llm_model_catalog as discovery
from app.services import llm_provider_catalog as catalog
REAL_CLIENT = httpx.AsyncClient
def mock_http(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(
discovery.httpx, "AsyncClient", lambda **kw: REAL_CLIENT(transport=transport, **kw)
)
@pytest.mark.asyncio
@pytest.mark.parametrize(
"provider,path,auth",
[
("minimax", "/anthropic/v1/models", "x-api-key"),
("anthropic", "/v1/models", "x-api-key"),
("openai", "/v1/models", "authorization"),
("deepseek", "/v1/models", "authorization"),
("alibaba", "/api/v1/models", "authorization"),
("moonshotai", "/v1/models", "authorization"),
("openrouter", "/api/v1/models", "authorization"),
("opencode-go", "/zen/go/v1/models", "authorization"),
("ollama", "/api/tags", "authorization"),
],
)
async def test_official_catalog_requests(monkeypatch, provider, path, auth):
preset = catalog.get_fallback_llm_provider_preset(provider)
def handle(request):
assert request.url.path == path
assert request.url.host == httpx.URL(preset["base_url"]).host
assert request.headers[auth] == ("test-key" if auth == "x-api-key" else "Bearer test-key")
if provider == "alibaba":
assert request.url.params["capabilities"] == "TG"
return httpx.Response(
200, json={"output": {"total": 1, "models": [{"model": "latest"}]}}
)
if provider == "ollama":
return httpx.Response(200, json={"models": [{"name": "latest:7b"}]})
return httpx.Response(200, json={"data": [{"id": "latest"}]})
mock_http(monkeypatch, handle)
result = await catalog.refresh_llm_provider_preset(provider, api_key="test-key")
assert result["models"] == (["latest:7b"] if provider == "ollama" else ["latest"])
assert result["base_url"] == preset["base_url"]
assert "test-key" not in str(result)
@pytest.mark.parametrize(
"provider,base,api,path",
[
(
"minimax",
"https://api.minimax.io/anthropic/v1/",
"anthropic-messages",
"/anthropic/v1/models",
),
("anthropic", "https://api.anthropic.com", "anthropic-messages", "/v1/models"),
(
"alibaba",
"https://workspace.cn-beijing.maas.aliyuncs.com/compatible-mode/v1",
"openai-completions",
"/api/v1/models",
),
(
"alibaba",
"https://dashscope-intl.aliyuncs.com/compatible-mode/v1",
"openai-completions",
"/api/v1/models",
),
("alibaba", "https://custom.test/gateway/v1", "openai-completions", "/gateway/v1/models"),
("moonshotai", "https://api.moonshot.cn/v1", "openai-completions", "/v1/models"),
("ollama", "http://localhost:11434/api", "ollama-generate", "/api/tags"),
],
)
def test_urls_preserve_region_and_gateway(provider, base, api, path):
url = httpx.URL(discovery.model_catalog_url(provider, base, api))
assert url.host == httpx.URL(base).host
assert url.path == path
@pytest.mark.asyncio
async def test_anthropic_pagination_and_release_order(monkeypatch):
seen = []
def handle(request):
seen.append(request.url.params.get("after_id"))
if len(seen) == 1:
return httpx.Response(
200,
json={
"data": [{"id": "old", "created_at": "2025-01-01T00:00:00Z"}],
"has_more": True,
"last_id": "old",
},
)
return httpx.Response(
200,
json={"data": [{"id": "new", "created_at": "2026-06-01T00:00:00Z"}], "has_more": False},
)
mock_http(monkeypatch, handle)
result = await catalog.refresh_llm_provider_preset("anthropic", "test-key")
assert seen == [None, "old"]
assert result["models"] == ["new", "old"]
@pytest.mark.asyncio
async def test_dashscope_pagination(monkeypatch):
def handle(request):
page = int(request.url.params["page_no"])
return httpx.Response(
200,
json={
"output": {
"total": 2,
"models": [
{"model": f"page-{page}", "published_time": f"2026-06-0{page} 00:00:00"}
],
}
},
)
mock_http(monkeypatch, handle)
result = await catalog.refresh_llm_provider_preset("alibaba", "test-key")
assert result["models"] == ["page-2", "page-1"]
@pytest.mark.asyncio
@pytest.mark.parametrize(
"payload",
[
{},
{"data": []},
{"data": [None]},
{"data": [{}]},
{"data": [{"id": "same"}], "has_more": True, "last_id": "same"},
],
)
async def test_invalid_or_incomplete_catalog_fails(monkeypatch, payload):
mock_http(monkeypatch, lambda request: httpx.Response(200, json=payload))
with pytest.raises(discovery.LLMProviderCatalogError):
await catalog.refresh_llm_provider_preset("minimax", "test-key")
@pytest.mark.asyncio
async def test_all_models_retained_and_defaults_unchanged(monkeypatch):
before = deepcopy(catalog.FALLBACK_LLM_PROVIDER_PRESETS)
rows = [{"id": f"model-{i}", "created": i} for i in range(140)]
mock_http(monkeypatch, lambda request: httpx.Response(200, json={"data": rows}))
result = await catalog.refresh_llm_provider_preset("openai", "test-key")
assert len(result["models"]) == 140
assert result["models"][0] == "model-139"
assert catalog.FALLBACK_LLM_PROVIDER_PRESETS == before
@pytest.mark.asyncio
async def test_empty_ollama_catalog_is_valid(monkeypatch):
mock_http(monkeypatch, lambda request: httpx.Response(200, json={"models": []}))
result = await catalog.refresh_llm_provider_preset("ollama")
assert result["models"] == []
assert result["model"] == ""
@pytest.mark.asyncio
async def test_opencode_documented_protocols(monkeypatch):
models = [
"minimax-m3",
"qwen3.8-max",
"gpt-5.6-luna",
"grok-4.6",
"muse-spark-1.3-contributor",
"kimi-k3",
]
mock_http(
monkeypatch,
lambda request: httpx.Response(200, json={"data": [{"id": model} for model in models]}),
)
result = await catalog.refresh_llm_provider_preset("opencode-go")
assert list(result["model_provider_apis"].values()) == [
"anthropic-messages",
"anthropic-messages",
"openai-responses",
"openai-responses",
"openai-responses",
"openai-completions",
]
@pytest.mark.asyncio
async def test_retry_transient_error_only(monkeypatch):
calls = []
def handle(request):
calls.append(request)
return httpx.Response(503 if len(calls) == 1 else 200, json={"data": [{"id": "latest"}]})
mock_http(monkeypatch, handle)
await catalog.refresh_llm_provider_preset("minimax", "test-key")
assert len(calls) == 2
calls.clear()
def unauthorized(request):
calls.append(request)
return httpx.Response(401, text="secret-upstream-text")
mock_http(monkeypatch, unauthorized)
with pytest.raises(httpx.HTTPStatusError) as error:
await catalog.refresh_llm_provider_preset("minimax", "test-key")
assert len(calls) == 1
assert "secret-upstream-text" not in discovery.catalog_error_message(error.value)

View File

@@ -0,0 +1,165 @@
from copy import deepcopy
from types import SimpleNamespace
from unittest.mock import AsyncMock
import httpx
import pytest
from sqlalchemy import create_engine, select
from sqlalchemy.orm import Session
from app.api.v1 import settings as api
from app.models.system_setting import SystemSetting
from app.services import llm_model_catalog as discovery
from app.services.llm_provider_catalog import list_fallback_llm_provider_presets
REAL_CLIENT = httpx.AsyncClient
@pytest.fixture
def settings_db(monkeypatch, tmp_path):
# Real SQL reads, commits and reloads; do not mock the settings helpers.
engine = create_engine("sqlite://")
SystemSetting.__table__.create(engine)
session = Session(engine)
db = SimpleNamespace(
execute=AsyncMock(side_effect=session.execute),
add=session.add,
commit=AsyncMock(side_effect=session.commit),
refresh=AsyncMock(side_effect=session.refresh),
)
monkeypatch.setattr(api, "AI_PROVIDER_ENV_FILE", tmp_path / "missing.env")
monkeypatch.setattr(api, "_resolve_env_secret", lambda *names: ("", ""))
yield db
session.close()
engine.dispose()
def use_upstream(monkeypatch, handler):
transport = httpx.MockTransport(handler)
monkeypatch.setattr(
discovery.httpx, "AsyncClient", lambda **kw: REAL_CLIENT(transport=transport, **kw)
)
@pytest.mark.asyncio
async def test_fresh_database_lists_all_builtin_presets(settings_db):
result = await api.get_ai_provider_presets(current_user=None, db=settings_db)
assert len(result["data"]) == 9
assert all(row["models"] for row in result["data"])
assert "MiniMax-M3" in result["data"][0]["models"]
@pytest.mark.asyncio
async def test_refresh_commits_reloads_and_preserves_runtime_and_draft(settings_db, monkeypatch):
runtime = {
"ai_provider": {
"default_provider": "minimax",
"service_token": "private-service-token",
"providers": {
"minimax": {
"base_url": "https://saved.test/anthropic",
"model": "old",
"api_key": "saved-secret",
}
},
}
}
await api.save_setting_payload(settings_db, "external_integrations", deepcopy(runtime))
before = await api.get_setting_payload(settings_db, "external_integrations")
seen = []
def handle(request):
seen.append(request)
assert str(request.url) == "https://draft.test/anthropic/v1/models"
assert request.headers["x-api-key"] == "draft-secret"
return httpx.Response(200, json={"data": [{"id": "MiniMax-M3"}, {"id": "old"}]})
use_upstream(monkeypatch, handle)
draft = api.AIProviderIntegrationUpdate(
provider="minimax",
base_url="https://draft.test/anthropic",
model="unsaved-model",
api_key="draft-secret",
service_token="unsaved-service-token",
)
refreshed = await api.refresh_ai_provider_preset("minimax", None, settings_db, draft)
result = await api.get_ai_provider_presets(None, settings_db)
preset = next(row for row in result["data"] if row["provider"] == "minimax")
assert len(seen) == 1
assert preset["models"] == ["MiniMax-M3", "old"]
assert preset["refreshed_at"] == refreshed["data"]["refreshed_at"]
assert await api.get_setting_payload(settings_db, "external_integrations") == before
assert "secret" not in str(result)
assert "private-service-token" not in str(result)
assert draft.model == "unsaved-model"
stored = await settings_db.execute(
select(SystemSetting).where(SystemSetting.category == "llm_provider_preset:minimax")
)
assert stored.scalar_one().payload["models"] == ["MiniMax-M3", "old"]
@pytest.mark.asyncio
async def test_failed_refresh_preserves_stored_catalog_and_sanitizes_errors(
settings_db, monkeypatch
):
await api.save_setting_payload(
settings_db, "llm_provider_preset:openrouter", {"models": ["saved-model"]}
)
use_upstream(monkeypatch, lambda request: httpx.Response(401, text="private-upstream-secret"))
commits = settings_db.commit.await_count
with pytest.raises(api.HTTPException) as error:
await api.refresh_ai_provider_preset("openrouter", None, settings_db)
assert error.value.status_code == 502
assert "private-upstream-secret" not in error.value.detail
assert settings_db.commit.await_count == commits
assert (await api.get_setting_payload(settings_db, "llm_provider_preset:openrouter"))[
"models"
] == ["saved-model"]
@pytest.mark.asyncio
async def test_missing_key_is_actionable_and_does_not_request_upstream(settings_db, monkeypatch):
def fail_request(request):
pytest.fail("must not query a private catalog without credentials")
use_upstream(monkeypatch, fail_request)
with pytest.raises(api.HTTPException) as error:
await api.refresh_ai_provider_preset("minimax", None, settings_db)
assert error.value.status_code == 400
assert "API Key" in error.value.detail
@pytest.mark.asyncio
@pytest.mark.parametrize(
"preset", list_fallback_llm_provider_presets(), ids=lambda p: p["provider"]
)
async def test_404_never_passes_by_builtin_model_name(monkeypatch, preset):
use_upstream(monkeypatch, lambda request: httpx.Response(404, text="private-upstream-secret"))
result = await api._check_ai_provider_lightweight(
{**preset, "api_key": "invalid-key", "preset_models": preset["models"]}, 5
)
assert result["success"] is False
assert result["connected"] is False
assert "private-upstream-secret" not in result["message"]
@pytest.mark.asyncio
@pytest.mark.parametrize("payload", [{}, {"data": []}, {"data": [{"id": "different-model"}]}])
async def test_invalid_or_missing_models_never_pass(monkeypatch, payload):
use_upstream(monkeypatch, lambda request: httpx.Response(200, json=payload))
preset = list_fallback_llm_provider_presets()[0]
result = await api._check_ai_provider_lightweight({**preset, "api_key": "test-key"}, 5)
assert result["success"] is False
@pytest.mark.asyncio
async def test_new_model_does_not_need_a_builtin_whitelist(monkeypatch):
use_upstream(
monkeypatch, lambda request: httpx.Response(200, json={"data": [{"id": "future-model"}]})
)
preset = list_fallback_llm_provider_presets()[0]
result = await api._check_ai_provider_lightweight(
{**preset, "model": "future-model", "api_key": "test-key"}, 5
)
assert result["success"] is True
assert result["url"].endswith("/anthropic/v1/models")

View File

@@ -0,0 +1,141 @@
from pathlib import Path
import sys
from unittest.mock import patch
import httpx
import pytest
sys.path.insert(0, str(Path(__file__).resolve().parents[2]))
with patch(
"pydantic_settings.sources.providers.dotenv.DotEnvSettingsSource._read_env_files",
return_value={},
):
from aiprovider.provider_service import ProviderService
from aiprovider.schemas import SituationalAnalysisRequest
from app.api.v1.settings import _runtime_config_from_ai_payload
def test_custom_openai_gateway_keeps_its_configured_protocol():
config = _runtime_config_from_ai_payload(
{
"default_provider": "openai",
"providers": {
"openai": {
"base_url": "https://gateway.test/v1",
"provider_api": "openai-completions",
"model": "gpt-6-astra",
},
},
}
)["llm_config"]
service = ProviderService({**config, "api_key": "test-key"})
assert service._resolve_model_provider_api("gpt-6-astra") == "openai-completions"
@pytest.mark.asyncio
@pytest.mark.parametrize(
"model,path",
[("minimax-m3", "/messages"), ("qwen3.8-max", "/messages"), ("gpt-5.6-luna", "/responses")],
)
async def test_runtime_model_routes_use_current_protocols(monkeypatch, model, path):
llm = _runtime_config_from_ai_payload(
{
"default_provider": "opencode-go",
"providers": {
"opencode-go": {
"model": model,
"api_key": "test-key",
"model_provider_apis": {model: "openai-completions"},
}
},
}
)["llm_config"]
service = ProviderService(llm)
seen = []
async def post(**request):
seen.append(request)
assert request["path"] == path
assert request["request_body"]["model"] == model
if path == "/responses":
assert request["request_body"]["store"] is False
assert "max_tokens" not in request["request_body"]
return {
"output": [
{"type": "reasoning", "summary": [{"text": "reason"}]},
{"type": "message", "content": [{"type": "output_text", "text": "OK"}]},
]
}
return {"content": [{"type": "text", "text": "OK"}]}
monkeypatch.setattr(service, "_post", post)
result = await service.analyze(SituationalAnalysisRequest(title="test", objective="reply OK"))
assert len(seen) == 1
assert result.content == "OK"
@pytest.mark.asyncio
async def test_minimax_messages_url_accepts_both_documented_base_forms(monkeypatch):
seen = []
client_type = httpx.AsyncClient
def handle(request):
seen.append(str(request.url))
return httpx.Response(200, json={"content": [{"type": "text", "text": "OK"}]})
transport = httpx.MockTransport(handle)
monkeypatch.setattr(httpx, "AsyncClient", lambda **kw: client_type(transport=transport, **kw))
for base in ["https://api.minimaxi.com/anthropic", "https://api.minimaxi.com/anthropic/v1"]:
service = ProviderService(
{
"provider": "minimax",
"provider_api": "anthropic-messages",
"base_url": base,
"api_key": "test-key",
"model": "MiniMax-M3",
}
)
result = await service.analyze(
SituationalAnalysisRequest(title="test", objective="reply OK")
)
assert result.content == "OK"
assert seen == ["https://api.minimaxi.com/anthropic/v1/messages"] * 2
@pytest.mark.asyncio
async def test_m3_playground_thinking_and_opencode_session_headers(monkeypatch):
seen = []
client_type = httpx.AsyncClient
def handle(request):
import json
seen.append(request)
assert json.loads(request.content)["thinking"] == {"type": "adaptive"}
assert request.headers["user-agent"] == "Planet/1.0"
return httpx.Response(200, json={"content": [{"type": "text", "text": "OK"}]})
transport = httpx.MockTransport(handle)
monkeypatch.setattr(httpx, "AsyncClient", lambda **kw: client_type(transport=transport, **kw))
for session_id in ["conversation-one", "conversation-one", "conversation-two"]:
service = ProviderService(
{
"provider": "opencode-go",
"provider_api": "anthropic-messages",
"base_url": "https://opencode.ai/zen/go/v1",
"api_key": "test-key",
"model": "minimax-m3",
}
)
await service.analyze(
SituationalAnalysisRequest(
title="test",
objective="reply OK",
thinking={"type": "enabled"},
context={"session_id": session_id},
)
)
session_headers = [request.headers["x-opencode-session"] for request in seen]
assert session_headers[0] == session_headers[1]
assert session_headers[0] != session_headers[2]

View File

@@ -0,0 +1,160 @@
from datetime import UTC, datetime
from unittest.mock import AsyncMock
import pytest
from sqlalchemy import create_engine, event
from sqlalchemy.orm import Session
from app.db.session import Base
from app.models.collected_data import CollectedData
from app.models.data_snapshot import DataSnapshot
from app.models.system_setting import SystemSetting
from app.models.task import CollectionTask
from app.services.collectors.news_live_streams import NewsLiveStreamsCollector
from app.services.tv_catalog import get_tv_catalog_page
from app.services.tv_streams import normalize_tv_settings
class CatalogSession:
"""Execute the real catalog queries against an isolated SQLite database."""
def __init__(self, session):
self.session = session
async def execute(self, query):
return self.session.execute(query)
async def scalar(self, query):
return self.session.scalar(query)
async def scalars(self, query):
return self.session.scalars(query)
@pytest.fixture
def catalog_db():
engine = create_engine("sqlite:///:memory:")
@event.listens_for(engine, "connect")
def register_functions(connection, _record):
connection.create_function(
"concat_ws",
-1,
lambda sep, *args: sep.join(str(arg) for arg in args if arg is not None),
)
Base.metadata.create_all(
engine,
tables=[
CollectionTask.__table__,
DataSnapshot.__table__,
CollectedData.__table__,
SystemSetting.__table__,
],
)
with Session(engine) as session:
for index in range(135):
session.add(
CollectedData(
source="news_live_streams",
source_id=f"channel-{index:03}",
data_type="news_live_stream",
name=f"Channel {index:03}",
collected_at=datetime(2026, 9, 13, tzinfo=UTC),
is_current=True,
is_valid=1,
extra_data={
"stream_url": "https://example.invalid/live.m3u8",
"region": "Canada",
},
)
)
session.flush()
yield CatalogSession(session)
engine.dispose()
@pytest.mark.asyncio
async def test_pages_include_entire_catalog_without_overlap(catalog_db):
first = await get_tv_catalog_page(catalog_db, limit=50)
second = await get_tv_catalog_page(catalog_db, offset=first["next_offset"], limit=50)
third = await get_tv_catalog_page(catalog_db, offset=second["next_offset"], limit=50)
ids = [source["id"] for page in (first, second, third) for source in page["sources"]]
assert [len(page["sources"]) for page in (first, second, third)] == [50, 50, 45]
assert len(set(ids)) == first["source_count"] == 145
assert ids[-1] == "channel-134"
assert third["next_offset"] is None and not third["has_more"]
beyond = await get_tv_catalog_page(catalog_db, offset=200)
assert beyond["sources"] == [] and not beyond["has_more"]
@pytest.mark.asyncio
async def test_search_finds_later_pages_and_treats_wildcards_literally(catalog_db):
payload = await get_tv_catalog_page(catalog_db, q="CANADA 134")
assert [source["id"] for source in payload["sources"]] == ["channel-134"]
assert payload["total"] == 1 and payload["source_count"] == 145
assert (await get_tv_catalog_page(catalog_db, q="%"))["total"] == 0
@pytest.mark.asyncio
async def test_selection_survives_refresh_when_outside_first_page(catalog_db):
payload = await get_tv_catalog_page(catalog_db, selected_id="channel-134")
assert payload["selected_source"]["id"] == "channel-134"
assert "channel-134" not in [source["id"] for source in payload["sources"]]
assert payload["default_source_id"] == "aljazeera-mubasher"
default = (await get_tv_catalog_page(catalog_db, selected_id="removed"))["selected_source"]
assert default["id"] == "aljazeera-mubasher" and default["source_type"] == "hls"
@pytest.mark.asyncio
async def test_catalog_hides_inactive_records_and_deduplicates_ids(catalog_db):
for name, values in [
("Disabled", {"extra_data": {"is_enabled": False}}),
("Historical", {"is_current": False}),
("Invalid", {"is_valid": 0}),
("Deleted", {"deleted_at": datetime.now(UTC)}),
("Replacement", {"source_id": "channel-134"}),
]:
record = dict(
source="news_live_streams",
source_id=name,
name=name,
data_type="news_live_stream",
is_current=True,
is_valid=1,
)
catalog_db.session.add(CollectedData(**{**record, **values}))
catalog_db.session.flush()
payload = await get_tv_catalog_page(catalog_db, q="replacement")
assert payload["source_count"] == 145
assert [source["id"] for source in payload["sources"]] == ["channel-134"]
def test_missing_builtin_default_adds_aljazeera_without_losing_custom_source():
settings = normalize_tv_settings({"sources": [{"id": "custom", "name": "Custom"}]})
assert settings["default_source_id"] == "aljazeera-mubasher"
assert {source["id"] for source in settings["sources"]} == {"custom", "aljazeera-mubasher"}
@pytest.mark.asyncio
@pytest.mark.parametrize(
"config, expected", [({}, 135), ({"max_sources": 0}, 135), ({"max_sources": 7}, 7)]
)
async def test_collector_keeps_all_matching_channels_unless_explicitly_limited(
monkeypatch, config, expected
):
collector = NewsLiveStreamsCollector()
channels = [
{"id": f"channel-{i}", "name": f"Channel {i}", "categories": ["news"]} for i in range(135)
]
channels.append({"id": "sport", "name": "Sports", "categories": ["sports"]})
streams = [
{"channel": channel["id"], "url": "https://example.invalid/live.m3u8"}
for channel in channels
]
monkeypatch.setattr(
collector, "_gather_iptv_org_payloads", AsyncMock(return_value=(channels, streams, []))
)
records = await collector._fetch_iptv_org("https://example.invalid/channels.json", config)
assert len(records) == expected
assert all(record["source_id"] != "sport" for record in records)

View File

@@ -0,0 +1,37 @@
from app.services.tv_streams import (
CCTV4_HOMEPAGE,
CCTV4_STREAM,
UNSUPPORTED_CCTV4_STREAM,
normalize_tv_settings,
normalize_tv_source,
)
def test_default_keeps_aljazeera_and_does_not_offer_cdrm_as_plain_hls():
settings = normalize_tv_settings(None)
assert settings["default_source_id"] == "aljazeera-mubasher"
cctv = next(source for source in settings["sources"] if source["id"] == "cctv4")
assert cctv["source_type"] == "hls"
assert cctv["stream_url"] == CCTV4_STREAM
assert "美洲版" in cctv["name"]
assert cctv["use_proxy"] is False
assert cctv["homepage_url"] == CCTV4_HOMEPAGE
def test_saved_legacy_default_is_corrected_without_mutating_input():
saved = {"id": "cctv4", "source_type": "hls", "stream_url": UNSUPPORTED_CCTV4_STREAM}
source = normalize_tv_source(saved)
assert source["source_type"] == "hls"
assert source["stream_url"] == CCTV4_STREAM
assert source["use_proxy"] is False
assert saved["stream_url"] == UNSUPPORTED_CCTV4_STREAM
def test_custom_cctv_stream_and_fallback_preference_are_preserved():
custom = {"id": "cctv4", "source_type": "hls", "stream_url": "https://example.com/licensed.m3u8"}
settings = normalize_tv_settings({"auto_fallback": False, "sources": [custom]})
source = next(source for source in settings["sources"] if source["id"] == "cctv4")
assert source["stream_url"] == custom["stream_url"]
assert source["source_type"] == "hls"
assert source["use_proxy"] is True
assert settings["auto_fallback"] is False

View File

@@ -76,6 +76,7 @@ def test_convert_compute_centers_to_geojson_unifies_sources():
longitude=-90.05,
metadata={
"organization": "xAI",
"location": "3231 Riverport Rd, Memphis, TN 38109",
"gpu_type": "H100",
"gpu_count": 100000,
"value": "20000",
@@ -168,6 +169,241 @@ def test_compute_center_source_coordinates_win_over_stored_location():
compute_center_locations.set_compute_center_location_cache({})
def test_epoch_country_level_coordinates_are_unresolved():
record = _build_record(
record_id=33,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name="Meta 100k",
country="United States",
city="",
latitude=39.381266,
longitude=-97.922211,
metadata={
"location": "United States of America",
"raw_country": "United States of America",
},
)
payload = convert_compute_centers_to_geojson([record])
assert payload["features"] == []
assert len(payload["unresolved"]) == 1
assert payload["unresolved"][0]["source_id"] == "epoch_ai_gpu-33"
assert "source_coordinates:country" in payload["unresolved"][0]["attempted_queries"]
def test_epoch_city_coordinates_are_not_labeled_precise():
record = _build_record(
record_id=34,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name="Seoul Cluster",
country="South Korea",
city="",
latitude=37.5665,
longitude=126.978,
metadata={"location": "Seoul", "raw_country": "South Korea"},
)
result = compute_center_locations.resolve_compute_center_location_full(
record, record.extra_data
)
assert result.is_resolved
assert result.location.location_precision == "city"
assert result.location.needs_confirmation is True
assert result.location.is_estimated is True
assert result.location.city is None
assert result.location.matched_location_name == "Seoul"
assert "Epoch Location: Seoul" in result.location.location_source_note
def test_epoch_city_geojson_preserves_source_location_without_guessing_city():
record = _build_record(
record_id=37,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name="Petrobras Fênix",
country="Brazil",
city="",
latitude=-22.909534,
longitude=-43.209934,
metadata={
"location": "Rio de Janeiro, Brazil",
"raw_country": "Brazil",
},
)
payload = convert_compute_centers_to_geojson([record])
feature = payload["features"][0]
assert feature["properties"]["location_precision"] == "city"
assert feature["properties"]["matched_location_name"] == "Rio de Janeiro, Brazil"
assert "Epoch Location: Rio de Janeiro, Brazil" in feature["properties"][
"location_source_note"
]
def test_confirmed_stored_location_beats_epoch_precise_coordinates():
compute_center_locations.set_compute_center_location_cache({
"epoch_ai_gpu:epoch_ai_gpu-35": {
"source": "epoch_ai_gpu",
"source_id": "epoch_ai_gpu-35",
"name": "Confirmed Site",
"latitude": 35.0,
"longitude": 139.0,
"precision": "site",
"confidence": 1.0,
"location_source": "official_facility_registry",
"needs_confirmation": False,
}
})
try:
record = _build_record(
record_id=35,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name="Japan Cluster",
country="Japan",
city="Tokyo",
latitude=35.68882,
longitude=139.692526,
metadata={
"location": "1-1 Verified Street, Tokyo 100-0001, Japan",
"raw_country": "Japan",
},
)
result = compute_center_locations.resolve_compute_center_location_full(
record, record.extra_data
)
assert result.location.location_source == "stored_compute_center_location"
assert result.location.location_precision == "site"
assert (result.location.latitude, result.location.longitude) == (35.0, 139.0)
finally:
compute_center_locations.set_compute_center_location_cache({})
def test_legacy_seeded_country_coordinate_is_ignored_on_read():
compute_center_locations.set_compute_center_location_cache({
"epoch_ai_gpu:epoch_ai_gpu-36": {
"source": "epoch_ai_gpu",
"source_id": "epoch_ai_gpu-36",
"name": "Legacy Country Point",
"latitude": 36.386493,
"longitude": 138.59223,
"precision": "precise",
"confidence": 1.0,
"location_source": "source_coordinates",
"needs_confirmation": False,
}
})
try:
record = _build_record(
record_id=36,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name="Legacy Japan Cluster",
country="Japan",
city="",
latitude=36.386493,
longitude=138.59223,
metadata={"location": "Japan", "raw_country": "Japan"},
)
result = compute_center_locations.resolve_compute_center_location_full(
record, record.extra_data
)
assert result.location is None
finally:
compute_center_locations.set_compute_center_location_cache({})
def test_verified_sites_replace_epoch_country_center_coordinates():
specifications = [
(
"epoch_jamstec_zettascaler_2_0_gyoukou",
"Gyoukou 2.0",
"Japan",
36.386493,
138.59223,
[139.625008, 35.380924],
),
(
"epoch_jamstec_zettascaler_2_2_gyoukou",
"Gyoukou 2.2",
"Japan",
36.386493,
138.59223,
[139.625008, 35.380924],
),
(
"epoch_argonne_nl_sophia__formerly_theta_",
"Sophia",
"United States of America",
39.381266,
-97.922211,
[-87.984498648496, 41.718109383287235],
),
]
records = []
for index, (source_id, name, country, latitude, longitude, _expected) in enumerate(
specifications, start=40
):
record = _build_record(
record_id=index,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name=name,
country=country,
city="",
latitude=latitude,
longitude=longitude,
metadata={"location": country, "raw_country": country},
)
record.source_id = source_id
records.append(record)
payload = convert_compute_centers_to_geojson(records)
assert payload["unresolved"] == []
assert len(payload["features"]) == 3
for feature, specification in zip(payload["features"], specifications):
assert feature["geometry"]["coordinates"] == specification[-1]
assert feature["properties"]["location_precision"] == "site"
assert feature["properties"]["location_source"] == "official_facility_registry"
assert "https://" in feature["properties"]["location_source_note"]
def test_verified_site_beats_epoch_city_level_source_coordinates():
record = _build_record(
record_id=44,
source="epoch_ai_gpu",
data_type="gpu_cluster",
name="Gyoukou 2.2",
country="Japan",
city="Tokyo",
latitude=35.68882,
longitude=139.692526,
metadata={"location": "Tokyo, Japan", "raw_country": "Japan"},
)
record.source_id = "epoch_jamstec_zettascaler_2_2_gyoukou"
result = compute_center_locations.resolve_compute_center_location_full(
record, record.extra_data
)
assert result.location.location_source == "official_facility_registry"
assert result.location.location_precision == "site"
assert (result.location.latitude, result.location.longitude) == (
35.380924,
139.625008,
)
def test_compute_center_geojson_uses_stored_location_when_source_coords_missing():
compute_center_locations.set_compute_center_location_cache({
"epoch_ai_gpu:epoch_ai_gpu-32": {
@@ -637,7 +873,11 @@ async def test_compute_centers_geojson_endpoint_returns_stats():
city="Memphis",
latitude=35.15,
longitude=-90.05,
metadata={"value": "20000", "unit": "TFlop/s"},
metadata={
"location": "3231 Riverport Rd, Memphis, TN 38109",
"value": "20000",
"unit": "TFlop/s",
},
),
]
@@ -741,7 +981,11 @@ async def test_visualization_geo_summary_returns_counts(monkeypatch):
city="Memphis",
latitude=35.15,
longitude=-90.05,
metadata={"value": "20000", "unit": "TFlop/s"},
metadata={
"location": "3231 Riverport Rd, Memphis, TN 38109",
"value": "20000",
"unit": "TFlop/s",
},
),
]

View File

@@ -1,5 +1,9 @@
import pytest
import importlib
import json
import pytest
from fastapi import FastAPI
from fastapi.testclient import TestClient
from app.core.websocket.manager import ConnectionManager
from app.core.websocket.broadcaster import DataBroadcaster
@@ -113,6 +117,11 @@ async def test_vessel_broadcaster_keeps_latest_update_per_mmsi(monkeypatch):
broadcaster_module = importlib.import_module("app.core.websocket.broadcaster")
monkeypatch.setattr(broadcaster_module.manager, "broadcast_vessels", fake_broadcast_vessels)
broadcaster = DataBroadcaster()
async def load_current(keys):
assert keys == ["1"]
return [{"mmsi": 1, "lat": 60.0, "lon": 11.0, "source": "barentswatch_vessels"}]
monkeypatch.setattr(broadcaster, "_load_current_vessel_updates", load_current)
broadcaster.enqueue_vessel_update(
{
"source": "aisstream_vessels",
@@ -129,10 +138,67 @@ async def test_vessel_broadcaster_keeps_latest_update_per_mmsi(monkeypatch):
assert sent[0]["vessels"] == [
{
"mmsi": 1,
"lat": 59.1,
"lon": 10.1,
"source": "aisstream_vessels",
"lat": 60.0,
"lon": 11.0,
"source": "barentswatch_vessels",
"action": "upsert",
"created": None,
}
]
@pytest.mark.asyncio
async def test_global_vessel_subscription_delivers_every_item_in_bounded_frames():
manager = ConnectionManager()
socket = FakeWebSocket()
config = manager.subscribe_vessels(socket, {"scope": "global", "zoom": 4})
json.dumps(config)
vessels = [{"mmsi": index, "lat": 60, "lon": 10} for index in range(2501)]
await manager.broadcast_vessels({"vessels": vessels})
assert [len(frame["payload"]["vessels"]) for frame in socket.sent] == [1000, 1000, 501]
assert [item for frame in socket.sent for item in frame["payload"]["vessels"]] == vessels
manager.unsubscribe(socket, ["vessels"])
await manager.broadcast_vessels({"vessels": vessels})
assert len(socket.sent) == 3
@pytest.mark.asyncio
async def test_global_vessel_removal_does_not_require_coordinates():
manager = ConnectionManager()
socket = FakeWebSocket()
manager.subscribe_vessels(socket, {"scope": "global", "zoom": 4})
await manager.broadcast_vessels({"vessels": [{"mmsi": 123, "action": "remove"}]})
assert socket.sent[0]["payload"]["vessels"] == [{"mmsi": 123, "action": "remove"}]
def test_anonymous_earth_can_confirm_global_vessel_subscription(monkeypatch):
websocket_module = importlib.import_module("app.api.v1.websocket")
monkeypatch.setattr(websocket_module, "manager", ConnectionManager())
app = FastAPI()
app.include_router(websocket_module.router)
with TestClient(app) as client, client.websocket_connect("/ws") as socket:
assert socket.receive_json()["type"] == "connection_established"
socket.send_json({
"type": "subscribe",
"data": {"channels": ["earth_updates", "vessels"], "scope": "global", "zoom": 4},
})
response = socket.receive_json()
assert response["type"] == "subscription_confirmed"
assert response["data"]["vessels"]["scope"] == "global"
assert response["data"]["vessels"]["type"] == []
@pytest.mark.asyncio
async def test_vessel_flush_retries_without_overwriting_newer_queued_updates(monkeypatch):
broadcaster = DataBroadcaster()
broadcaster.enqueue_vessel_update({"vessels": [{"mmsi": 1, "lat": 59, "lon": 10}]})
async def fail_read(_keys):
broadcaster.enqueue_vessel_update({"vessels": [{"mmsi": 1, "lat": 61, "lon": 12}]})
raise RuntimeError("database unavailable")
monkeypatch.setattr(broadcaster, "_load_current_vessel_updates", fail_read)
with pytest.raises(RuntimeError, match="database unavailable"):
await broadcaster.flush_vessel_updates()
assert broadcaster._pending_vessel_updates["1"]["lat"] == 61

View File

@@ -25,6 +25,25 @@
{
"id": "china-taiwan-penghu",
"name": "Taiwan and Penghu",
"sourceIsoA3": ["TWN"],
"action": "union_to_country",
"targetIsoA3": "CHN",
"hoverIsoA3": "CHN",
"labelPolicy": "show_country_only"
},
{
"id": "china-hong-kong",
"name": "Hong Kong",
"sourceIsoA3": ["HKG"],
"action": "union_to_country",
"targetIsoA3": "CHN",
"hoverIsoA3": "CHN",
"labelPolicy": "show_country_only"
},
{
"id": "china-macao",
"name": "Macao",
"sourceIsoA3": ["MAC"],
"action": "union_to_country",
"targetIsoA3": "CHN",
"hoverIsoA3": "CHN",
@@ -57,6 +76,7 @@
{
"id": "kosovo",
"name": "Kosovo",
"sourceIsoA3": ["KOS", "XKS"],
"action": "render_as_disputed_with_parent",
"parentIsoA3": "SRB",
"hoverIsoA3": "SRB",

View File

@@ -1,5 +1,3 @@
version: '3.8'
services:
aiprovider:
image: ${AI_PROVIDER_IMAGE_NAME:-planet-aiprovider:latest}
@@ -32,7 +30,7 @@ services:
POSTGRES_PASSWORD: postgres
POSTGRES_DB: planet_db
ports:
- "5432:5432"
- "127.0.0.1:${PLANET_POSTGRES_PORT:-5432}:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
@@ -45,7 +43,7 @@ services:
image: redis:7-alpine
container_name: planet_redis
ports:
- "6379:6379"
- "127.0.0.1:${PLANET_REDIS_PORT:-6379}:6379"
volumes:
- redis_data:/data
healthcheck:

View File

@@ -8,6 +8,121 @@ This project follows the repository versioning rule:
- `improvement` -> `+0.0.1`(bugfix + 小功能混合)
- `bugfix` -> `+0.0.1`
## [0.75.2] — 2026-09-25
Released: 2026-09-25
### Highlights
- 修复 Windows 启动器固定使用旧版 PowerShell 的问题,优先在 Windows Terminal 新窗口中运行 PowerShell 7。
### Fixed
- 仓库与桌面 CMD 引导优先使用 `pwsh.exe`,缺少新版 PowerShell 或 Terminal 时保留可用环境回退。
- 已以管理员运行的入口同样进入首选终端,子窗口校验权限后直接衔接 WSL,避免重复提权和递归开窗。
- 补齐终端选择、含空格路径、提权分支回归和中英文说明,保持 WSL 重启参数及默认端口不变。
---
## [0.75.1] — 2026-09-25
Released: 2026-09-25
### Highlights
- Windows 启动器从当前项目位置识别 WSL 发行版与目录,首次生成本机桌面 CMD,后续沿用标准重启流程。
- 算力对比数据可由后台维护;小球恢复惯性拖动,大幅放大后保留球面跟手,修复移出球面后的反向旋转。
### Added / Fixed / Improved
- 桌面入口兼容两种 WSL UNC 路径和 Windows 本地磁盘映射,保留原有文件,支持只安装检查;默认端口仍为 3000 / 8000 / 8010。
- 算力报告来源、年度观测与创新分差使用数据库配置,提供数值校验、版本冲突保护、事务审计、恢复默认和地球自动刷新。
- 根据相机视锥在按下时固定拖动模式;小球使用旧版位移平滑与惯性,大球不再用球外最近点生成反向旋转。
- 更新开发 agent 的 Sol / Luna / Astra 分工规则,补齐中英文运维说明、启动故障编号与隔离回归测试。
---
## [0.75.0] — 2026-09-23
Released: 2026-09-23
### Highlights
- 新增算力与资源区域比较、可拖动的地理卡片和移动端原位展开;以算力占比为主,保留趋势、覆盖与来源信息。
- 四张 8K 底图可切换,原始底图设为默认并列于首位;优化高清首帧加载,修复最大缩放穿透和高倍拖动不跟手。
### Added / Fixed / Improved
- 统一物件卡片的先绘线后显示时序与靠边布局;地球明显转向后成组调整区域卡片,减少交叉并保留阅读时间。
- 合并算力区域目录及历史口径,校验设施坐标粒度与公开站点,避免以样本数冒充算力占比或将近似坐标规则排列。
- 地表贴图异步解码与按需替换,启动模块打包、数据并行加载;复用共享下拉菜单、HUD 样式与移动端交互。
- 国界贴合地表并贯穿固定地理显示规则;修复直播恢复、已选频道回退、本地账户身份响应与相关交互回归。
- 补齐后端镜像所需规则、审核几何和构建脚本,记录高精国界重建要求,更新中英文使用及运维文档。
---
## [0.74.7] — 2026-09-17
Released: 2026-09-17
### Highlights
- 启动进度统一显示在状态副行,成功清除、失败保留具体错误并退出,减少演示前的静默等待和重复报错。
- 镜像仓库网络故障支持一次临时备用源回退,校验具体镜像 manifest 并锁定 digest,不改写全局镜像源。
### Added / Fixed / Improved
- 为公开 Python、uv、PostgreSQL 和 Redis 镜像提供 DaoCloud 回退及可配置前缀;认证、证书、限流和 Dockerfile 依赖安装错误不误触发回退。
- 数据库启动前识别其他项目的端口占用,按后端实际配置同步本机映射,保留命名数据卷;修复停止后 TIME_WAIT 被误判为监听占用。
- Docker 命令增加有界等待、私有日志和进程组清理;失败副行保留实际原因,避免外层汇总及清理通知覆盖。
- 修复 Python 依赖清华源回退前过早冻结错误状态,移除 Compose 失败后的旧手动启动分支。
- 同步中英文排障手册,补齐镜像回退、非网络错误、digest 校验、端口复用及失败退出回归,并验证实际备用镜像构建与项目启动。
---
## [0.74.6] — 2026-09-16
Released: 2026-09-16
### Highlights
- AI Provider 构建前自动检测主机代理和直连路径,减少终端网络正常但 Docker 镜像拉取超时的问题。
- 运维错误原因与脚本诊断共用一份对照表,统一错误编号、原因和处理建议。
### Added / Fixed / Improved
- 已有 Compose v2 时不再因构建或启动失败回退 v1;Dockerfile 改用 BuildKit 内置解析器,减少一次外部镜像下载。
- 新增 --no-build 启动选项,明确复用本地 AI Provider 镜像,缺少镜像时报错且不改写构建指纹。
- 自动代理检测尊重 NO_PROXY,仅修改 Planet 管理的本地 Docker 配置;更新前备份和校验,必要时重启并恢复原有容器,失败时回滚。
- 中英文运维表覆盖网络、证书、权限、限流和启动故障;未知错误明确保留未归类状态,并要求新确认原因补表与回归用例。
- 修复详细日志模式丢失构建失败退出码的问题,补齐代理切换、Compose 选择、错误分类及文档一致性验证,并收录算力与资源态势实施计划。
---
## [0.74.5] — 2026-09-13
Released: 2026-09-13
### Highlights
- 恢复模型供应商预设的加载与持久化,模型刷新和轻量连接检查统一使用供应商官方目录接口。
- 修复 MiniMax M3 思考模式和 OpenCode Go 模型协议路由,补齐 Responses 适配。
### Added / Fixed / Improved
- 修复 `llm_provider_preset:*` 动态设置分类触发 KeyError 和列表 500,使用真实数据库读写回归验证保存与重载。
- 统一九家供应商的目录 URL、鉴权、响应解析与分页,保留服务地域和自定义地址;明确处理限流、空目录和失败,移除 404 白名单误判成功。
- 模型刷新读取表单草稿地址和凭证,保留默认模型、未保存修改及失败前目录;连接提示区分目录可用与实际生成成功。
- 增加 Responses 请求与响应解析、M3 adaptive 思考适配和 OpenCode 会话标识;已知模型选择同步协议,自定义 OpenAI 网关保留手动协议。
- 同步中英文接口说明、操作手册与快速开始,并覆盖模型目录失败、草稿保留和协议适配回归。
---
## [0.74.4] — 2026-09-13
Released: 2026-09-13
### Highlights
- 保留 Earth 全量对象与交互,优化海缆、登陆点和卫星绘制,并让 AIS / BarentsWatch 船只通过确认状态增量更新。
- 新闻直播支持完整频道目录搜索和无限滚动;模型目录刷新可保存结果,启动流程减少不必要的依赖安装与等待。
### Added / Fixed / Improved
- 海缆和登陆点合批绘制,卫星 SGP4 计算移入 Worker、呼吸动画移入 GPU,并缩小动态文本的翻译扫描范围。
- 船舶全局订阅按 MMSI 合并、拆包和原位更新,保留选择状态,并处理删除、重连及旧快照覆盖。
- 新闻直播取消默认 120 项采集截断,增加数据库分页、完整目录搜索和固定计数栏,默认源改为半岛电视台 HLS。
- 算力中心定位队列在当前 Earth 页面内独立于详情面板继续运行;模型刷新保存新目录并保留当前模型、凭证草稿与失败前目录。
- AI Provider 镜像使用独立依赖组,容器直接运行已安装环境;启动提前验证数据库、及时识别后端失败,并保留可复用容器。
---
## [0.74.3] — 2026-09-13
Released: 2026-09-13

View File

@@ -79,6 +79,16 @@ git diff --unified=0 HEAD -- <path>
## Harness Commands
Windows desktop launcher tests run separately on Windows with
`powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts\harness\test_windows_launcher.ps1`.
They use temporary desktop folders and mocked restart commands; no UAC prompt
or real service restart is triggered. The WSL bridge regression suite,
`scripts/harness/test_windows_restart.py`, is included in quick-check and uses
disposable command substitutes. For a real entry-point check without restarting,
run the repository `planet.cmd --install-only`, then the generated desktop
launcher with the same flag. Installation and mocked checks do not prove a live
elevated restart; report that separately when it has not been exercised.
| Tier | Command | What It Does |
| --- | --- | --- |
| Doctor | `scripts/harness/doctor.sh` | Checks required files, required tools, optional delivery tools, and forbidden frontend lockfiles. |
@@ -86,7 +96,7 @@ git diff --unified=0 HEAD -- <path>
| Backend Rules | `scripts/harness/backend-rules-check.sh` | Checks backend app Python for direct `print()`, `breakpoint()`, and `pdb.set_trace()` debug calls so service code uses structured logging. |
| Frontend Rules | `scripts/harness/frontend-rules-check.sh` | Checks Bun-only scripts, admin route manifest coherence, literal internal route links, admin search route targets, frontend debug output, native button safety, icon-button accessibility, no nested Cards, no AntD/Space layout primitives, ConnectionTestInput usage, admin/docs shell height-chain sizing, same-category style owner warnings, viewport-scaled font sizes, zero letter spacing, and high-signal UI rule warnings. |
| Docs Consistency | `scripts/harness/docs-consistency-check.sh` | Checks frontend Docs metadata against backend Gatekeeper metadata, public Docs registration, full technical-doc bilingual file pairs, public doc links, readable link titles, language-scoped technical links, README/project-context admin stack drift, supported credential collector contracts, manual console route coverage against the actual admin manifest, documented UI route drift, documented `?section=` deep-link validity against the actual admin section config in technical docs and active plan docs, and the harness rules-coverage notes. |
| Quick | `scripts/harness/quick-check.sh` | Runs doctor, whitespace diff check, shell syntax checks, security scan, backend/frontend/doc consistency checks, and CI backend smoke tests. |
| Quick | `scripts/harness/quick-check.sh` | Runs doctor, whitespace diff check, shell syntax checks, isolated Docker bootstrap/proxy and startup tests, error-catalog consistency tests, security scan, backend/frontend/doc consistency checks, and CI backend smoke tests. |
| Full | `scripts/harness/validate.sh` | Runs quick check, frontend Bun install/build, Playwright route smoke, optional Helm checks, and opt-in Docker image smoke builds. |
Docker image smoke builds are expensive and are off by default:

View File

@@ -22,6 +22,9 @@
当前重点入口:
- [开发 Agent 模型分配与 Token 控制计划](development-agent-model-allocation-plan.md)
- [Earth 地理资源、空间浮动卡片与性能实施计划](earth-spatial-presentation-and-geographic-assets-plan.md)
- [算力与资源态势:展示、采集与 AI 迭代实施计划](compute-resource-intelligence-plan.md)
- [控制台 i18n 接入计划](/home/ray/dev/linkong/planet/docs/plans/admin-console-i18n-plan.md)
- [Earth Mobile Drawer UI Plan](/home/ray/dev/linkong/planet/docs/plans/earth-mobile-drawer-ui-plan.md)
- [Earth Compute Center BGP Style Plan](/home/ray/dev/linkong/planet/docs/plans/earth-compute-center-bgp-style-plan.md)

View File

@@ -0,0 +1,602 @@
# 算力与资源态势:展示、采集与 AI 迭代实施计划
状态:首版空间卡片原型实现中(2026-09-21)。已接入算力专题入口、45 个参与方目录、年份/口径选择、区域数据卡与来源、成功快照设施查询,以及 Epoch/TOP500 的首批质量修复。旧固定双图布局已移除。历史年度数据使用受控版本导入;各国完整指标覆盖、卡片图表、全榜获取、跨源设施合并、持久化指标管理、变化归因与 A4 AI 解读尚未完成,不视为全部 A1—A3 已验收。后续章节保留完整设计目标,当前实现以技术文档为准。2026-09-14 完成的口径核查仍是内容依据。面向产品、开发、数据研究与运维人员,作为此专题后续实现的设计依据。
## 0. 当前执行方向与范围
2026-09-21 展示方向修订:UI、地球材质、政治展示规则、地理资源对齐及性能门槛以[Earth 地理资源与空间卡片计划](earth-spatial-presentation-and-geographic-assets-plan.md)为准。下文固定右侧国家面板、底部横贯双图等布局不再作为实施目标。采用从区域锚点引出连接线、卡片围绕地球自适应浮动的方向,先验证空间交互再完善卡片内容。数据来源、口径与证据要求继续有效。
平台长期定位是“信息展示 + 决策辅助”。当前实施重点是把全球算力对比作为第一个成熟专题做清楚:地图显示资源在哪里,图表说明规模和相对位置怎样变化,证据说明数字从哪里来,AI 帮助阅读变化。
采用“专题先交付,公共能力按复用需要沉淀”的顺序。第 7—9 节的完整 AI 迭代、模型实验与通用持久化是后续路线,不是首版展示的前置条件;第 6 节总分保持实验性,不能为了首屏有大数字而生成不完整排名。
| 当前要做 | 当前完成形态 | 后续才扩展 |
| --- | --- | --- |
| 算力数据可信 | 修复既有采集,国家/设施/精度/年份不混用 | 大规模自动来源发现、完整证据图谱 |
| 全球分布与国家对比 | 现有算力中心地球 + 可比参与方 + 明确样本覆盖 | 制造、矿产、电力和贸易的跨主题关系 |
| 历史展示 | 两张核对后的图、时间选择、已知事件轨道 | 复杂预测、不确定性传播与可训练参数 |
| 证据与解释 | 来源、参考年、关键字段;基于数据快照的简短 AI 说明 | 自动补证、模型候选、回测、影子运行与提升 |
首版验收路径:打开算力专题 → 看全球公开设施分布 → 选中国和美国 → 同时看算力与创新历史 → 点一个变化或设施 → 查看出处及解释。此路径稳定后,才扩大评价维度。
## 1. 要交付什么
在智能星球现有算力中心图层上增加“算力与资源态势”模式,并在控制台提供对应的数据管理和模型实验工作区。核心问题是:各方拥有什么资源、资源如何形成可用能力、相对位置如何变化、变化有何证据,以及未来什么约束可能改变格局。
首批比较中国和美国;数据结构从第一天支持多方。随后纳入欧洲、日本、韩国、印度及其他数据合格的参与方。地区汇总与成员国不得重复计入全球分母。所有国家和地区字段使用现有 `normalize_country()` 和规范字典,不以网页原文或 AI 判断覆盖项目标签。
交付三个相互连接的产品能力:
| 能力 | 用户获得的结果 | 首版边界 |
| --- | --- | --- |
| 历史观测 | 算力绝对规模、全球份额、创新指数与分差、设施时间线 | 使用可验证观测;缺失年份可见,不插值冒充事实 |
| 变化解释 | 哪些设施、资源、数据修订造成指标变化,每项贡献和证据 | 数学贡献可复算;原因解释区分事实、假设与未解问题 |
| 模型改进 | AI 发现缺口、补证、提出改进,经过验证产生新版本 | 固定工作流先行;不能由大模型直接决定国家分数 |
第一版先交付可信的事实与回放,不以总分完整作为上线前提。完整创新指数分项、有效算力参数和项目交付数据不足时,继续展示已完成的观测能力,综合评分显示“数据不足”,不生成示意国家排名。
不在第一版承诺全球所有设施的完整清单、精确的国家真实算力、每日变化的国家创新能力、可验证的国家综合实力真值,或自动预测某年必然追平。
## 2. 当前项目基础与必须补齐的边界
以下结论来自当前文件检查,未对生产数据库、凭证可用性或线上采集结果做完整审计。
| 已有入口 | 已确认能力 | 本专题需要补齐 |
| --- | --- | --- |
| [采集基类](../../backend/app/services/collectors/base.py)、[数据作业](../../backend/app/services/data_jobs.py) | 采集、转换、任务状态、快照、取消与重试基础 | 文档型数据源、质量门槛、领域规范化、失败保留正式快照 |
| [Epoch 采集器](../../backend/app/services/collectors/epoch_ai.py) | GPU 集群采集入口 | 改读公开 CSV;去掉解析失败返回示例记录;单位、精度、状态和时间字段校正 |
| [TOP500 采集器](../../backend/app/services/collectors/top500.py) | 榜单与详情解析 | 去掉示例回退、固定参考日期、排名作为身份;按当期表头解析单位 |
| [采集记录](../../backend/app/models/collected_data.py)、[快照](../../backend/app/models/data_snapshot.py) | `entity_key`、快照关联、前一记录和变化摘要 | 原始证据版本、发表时间、有效时间、指标注册和可重放的观测集合 |
| [算力位置服务](../../backend/app/services/compute_center_locations.py)、[GeoJSON](../../backend/app/api/v1/visualization.py) | 设施定位、位置置信信息、未定位记录 | 国家统计与地图可定位集合分离;匿名设施不能猜坐标 |
| [算力中心渲染](../../frontend/public/earth/js/compute-centers.js)、[Earth 壳](../../frontend/src/pages/Earth/Earth.tsx) | Three.js 交互层;Earth 由 iframe 加载独立页面 | 历史时点、国家比较、双图、关系层和解释面板 |
| [图层适配器](../../backend/app/services/earth_layer_adapters.py)、[数据库监听](../../backend/app/services/earth_db_change_listener.py) | 通过 PostgreSQL outbox 驱动缓存与 `earth_updates` | 正式指标快照发布事件;草稿与实验结果不改变默认地球 |
| [AI Client](../../backend/app/services/ai_client.py)、[提示词注册](../../backend/app/ai_tasks/prompts.py) | 全局 provider 配置、模型调用、任务提示词及运维覆盖 | 算力专题的固定工作流、结构化输出验证、预算与审计 |
| [搜索工具](../../backend/app/services/ai_tools/web_search.py)、[网页取证](../../backend/app/services/ai_tools/web_fetch.py) | 搜索和基础 HTML 正文提取 | PDF/表格提取、引用位置、下载时大小限制、URL 与重定向的访问限制 |
| [证据辅助函数](../../backend/app/services/ai_tools/evidence_store.py) | 摘要、去重和内容哈希规范化 | 该文件不是持久化证据库,需补建不可变原文存储和证据记录 |
沿用[数据作业与 Outbox 架构](../technical/zh/data-job-earth-sync-architecture.md)及[轻量 Agent 编排计划](agents-light-orchestrator-websearch-plan.md):PostgreSQL 负责可靠账本,Redis 负责缓存;不另起 Kafka、Celery 或另一个独立调度真相源。`aiprovider` 保持协议适配,业务取证、估计、工具权限和模型发布全部由后端负责。
当前 `JobType` 只有采集、清理和 Earth 刷新。新增分析、实验工作流需要显式扩展任务类型、worker 分发和取消策略,不能把现有采集队列描述成已经支持通用 Agent。
## 3. 页面与地球怎样展示
### 3.1 默认工作区
保留 `/earth` 主入口,在现有算力中心工具中增加专题模式,避免增加一个与地球无关联的新大屏。进入后使用以下概念布局;这是规划线框,不是现有页面截图。
```text
┌ 时间/参考期 · 中国/美国/其他参与方 · 指标口径 · 观测/情景 ┐
│ │
│ 3D 地球:设施、资源、已证实关系 国家对比面板 │
│ 绝对量/份额 │
│ 变化贡献 │
│ 证据与缺口 │
├────────────── 历史时间轴 / 事件轨道 ────────────────────┤
│ 算力规模与份额图 综合创新指数与分差图 │
└───────────────────────────────────────────────────────┘
```
桌面默认同时显示两张历史图,用相同国家配色,但各自保留单位和实际观测年份。图表高度优先保证刻度、来源标识与数据点可读,不能通过压缩字体维持布局。较窄屏幕把图表移入可切换的详情区,国家和设施详情使用抽屉;不叠加多个遮挡地图的浮层。
沿用项目一屏高度链和主题。桌面在 1366×768、1920×1080,移动端在约 390×844 验证,另测 125%/150% 浏览器缩放。页面根与中间 flex 容器分别正确设置高度和 `min-height: 0`,仅详情、长表和证据正文内部滚动。
### 3.2 视图职责
| 视图 | 主内容 | 点击或切换后的行为 |
| --- | --- | --- |
| 国家总览 | 可比算力绝对量、份额、子分、实际参考期;总分合格才出现 | 选中参与方,定位地球并联动两图;不重设全局评分尺度 |
| 算力趋势 | 绝对规模、全球份额、双边份额三种独立视图 | 同一来源口径内切换;不把百分比轴和 FLOP/s 轴混合 |
| 创新趋势 | 各方指数;只有分差时只显示中美分差 | 切换年度,查看分项与报告版本,不能反推缺失的单国得分 |
| 变化账单 | 新投产、扩建、退役、效率变化、资料补录、方法修订 | 点选贡献定位资产;逐步展示输入、计算和证据 |
| 设施详情 | 设备、精度、投产状态、功率、运营方、地点精度、历史版本 | 展开相关公告、设备来源及不确定字段 |
| 资源关系 | 已证实的芯片供给、所有权、电力接入或网络连接 | 选一类关系再显示,不能让所有关系默认铺满地球 |
| 情景工作区 | 并网推迟、交付变化、效率假设及结果区间 | 独立 scenario ID;退出恢复正式快照 |
| 控制台研究区 | 数据缺口、待核验证据、采集任务、模型实验和版本 | 使用已有采集与 AI 设置入口;Playground 仍用于调试 |
### 3.3 两张图进入产品的明确规则
总算力图以核对后的报告观测为起点:2020 年美国/中国为 36%/31%,2021 年 34%/33%,2022 年 34%/33%,2023 年 41%/31%。原始来源和报告版本见第 15 节。2024—2025 年暂不填值;2026 年 42%/33% 只保存为待核实主张,不能接到正式历史曲线上。
创新图使用报道明确披露的 2023、2024、2025 年分差 22.02、19.96、17.95 分。报告发表于 2026 年,必须区分报告年份与参考年份。完整分项、年度得分和方法修订仍需采集。图中是综合创新指数,不是纯算力或纯产出。
原始主张、报告估计、已验证记录和未来情景用文字标签区分。年度点之间的视觉连线只辅助阅读,不进入计算;不能把连线插值写成月度实测。正式指标没有新观测时保留参考期并提示新鲜度,不让刷新网页造成“国家实力实时跳动”。
### 3.3a 当前数据合同(2026-09-22)
默认主指标是全球计算设备总规模,拆分基础计算、智能计算和超级计算,观察期为 2020—2023。CAICT 报告核查目前使用官方 PDF 索引文本;本地 PDF 下载未成功,因此不宣称已肉眼核验 PDF 页面。绝对量合同记录 2022 年 302(证据版本 `caict2023`)和 2023 年 435(证据版本 `caict2024`);2024 年报告正式书名为《先进计算暨算力发展指数蓝皮书(2024年)》。官方 URL 已替换为四个镜像来源。
日本全球份额观测为 2020/2021/2022/2023 = 6%/5%/4%/4%。产品分组 `APAC` 与 `OTHER` 不能直接套用报告中的 Japan 或“其他”;`EUROPE` 成员集合尚未与报告口径对齐,仍返回缺失。双边份额分母固定为 US + China,新加入日本不改变分母。主要指标缺失时可显示最近有报告的年份(当前可到 2023),而不是留空为 2026;用户明确选择 2026 时仍显示缺失,不回填最近值。不可比性尚未确认,不展示未经确认的同比变化;覆盖数据保留在辅助区并带 count。
数据 API 审计字段为 `reviewed_at=2026-09-22`,版本与该日期一致。独立 Epoch 份额模式尚未实现,用户对该问法的选择待后续确认;当前合同不提前新增该模式或声称完成 dataset 计算。
默认“历史回顾”允许查看最新证据重述后的过去;“当时可知”只使用在截止时点已经发表且可证明可获得的资料。每张图同时展示参考期与知识截止时间。已选 2025 年而指标最新只到 2023 年时,必须显示“最新观测:2023”,或在严格同年模式显示缺失。
### 3.4 地球编码与交互
- 节点面积按同口径能力编码;跨度过大时可切换对数档位并明示图例。国家颜色只用于身份,不能把颜色本身当作领先或落后。
- 已投产用实心,建设或情景用空心/虚线;置信不足使用独立标记,不能把低置信透明度误读为小算力。
- 来源只到国家或区域时,保留在国家/区域统计和未定位列表;不画在首都,不把行政区域中心当作设施坐标。
- 国家总量与可见设施合计可以不同,分别显示覆盖范围。不得只统计有坐标的节点,也不得把宏观缺口按现有节点比例摊分。
- Epoch 当前公开版本对中国部分集群匿名化并取整;匿名记录只能按允许的粒度使用。新增官方公告可作为独立证据,不能依据匿名数值猜出设施身份或制造地点。
- 供应、所有权和网络关系采用不同图例,显示关系来源与有效期;商业供应关系不等同于实际运输路线。
- 全国可汇总吞吐与最大单集群能力分别展示;地理分散设备不能合并成一个训练集群。
- 时间回放使用稳定实体 ID 更新已有节点。隐藏图层、变更时点或情景时同步清理 hover、locked、tooltip 和 selection;请求过期结果不得覆盖新时点。
- 使用现有 Three.js Interactable、图标和实例化能力。新增壳层必须遵循现有深度顺序,在远距约 50% 缩放检查闪烁;不另建悬浮球面掩盖深度问题。
## 4. 数据需要补充采集什么
### 4.1 P0:决定首版可信度的数据
“已核对入口”只表示资料或文档可查,不表示接口凭证、完整历史和生产连通性已经验证。以下频率是 Planet 建议检查频率,不是来源承诺的更新频率。
| ID / 优先级 | 数据与来源 | 必需字段/粒度 | 采集方式与频率 | 当前缺口与采用条件 |
| --- | --- | --- | --- | --- |
| D01 / P0 | 信通院历年算力报告 | 国家×参考年×口径;绝对规模、全球总量/份额、单位、精度、估算方法、版本 | 已取得报告先受控导入;月度检查新版;PDF 表格提取后核验 | 已核对 2020—2023 份额及部分绝对量;补 2024 以后和其他国家;跨版本方法改变时断开序列 |
| D02 / P0 | Epoch GPU Clusters | 集群/版本;设备数、型号、16/8/32-bit 性能、所在地、状态、投产/退役时间、功率、扩建关系 | 官方公开 CSV,每日条件抓取,按内容哈希去重 | 改造现有 HTML 解析;处理匿名、取整、缺坐标和覆盖偏差;不能声称完整全国清单 |
| D03 / P0 | TOP500 / 可选 Green500 | 稳定系统 ID×榜单期;Rmax/Rpeak、FP64、功率、机构、名次、列表日期 | 每周检查新榜单;按发布期导入历史与详情 | 修复 rank 作为 source ID、固定日期和单位问题;榜单消失不等于退役;不当成 AI 总算力 |
| D04 / P0 | 全球 AI 创新指数原报告及发布材料 | 国家×参考年×指标;单国总分、分项、权重、标准化、成员范围、方法版本 | 文档导入+月度更新检查 | 当前只具备部分新闻转述分差;优先获得分项和年度可比性证据;不能用差值合成各国历史得分 |
| D05 / P0 | 已有位置维表与公开设施公告 | 实体别名、运营方、国家、地点、精度、采用依据、有效期 | 复用位置管线,新增/变化时触发 | 全国汇总独立于定位成败;不把同址不同分期误合并,也不把扩建当第二个全量集群 |
首版最低闭环:D01 和 D04 的已验证历史可先支撑双图,D02/D03/D05 支撑设施地图;两者不互相伪装为同一覆盖范围。若 D04 分项不可取得,创新总分仍作为独立观测,O 分与综合 S 不发布。
首版可以先受控导入已核实的年度数据,不必先完成整套 PDF 自动解析和报告发现。D04 原报告分项继续作为补数任务,不阻塞已经核对的创新分差图;原始摘录、出处、年份和未核实范围必须随数据保存。
### 4.2 P1/P2:有效能力、增长与资源关系
| ID / 优先级 | 补充数据与候选来源 | 粒度与核心字段 | 方式/频率 | 对模型的作用及限制 |
| --- | --- | --- | --- | --- |
| D06 / P1 | Epoch 芯片销售、所有权、硬件数据;厂商规格 | 芯片型号×时间;规格、销售/交付、持有主体、数量、性能换算、估计区间 | 官方数据文件与规格文档;周查/季度快照 | 校验硬件供给;设计者、所有者、实际所在地与使用权分开;不能把出货和已部署重复加总 |
| D07 / P1 | MLPerf Training / Inference、可复现厂商测试 | 配置×任务×软件版本;模型、质量门槛、batch、上下文、时延、吞吐、能耗、可用状态 | 官方结果导入;月查新版本 | 校准特定任务的性能系数;不同 benchmark 版本/质量/负载不直连;实验室成绩不等同生产利用率 |
| D08 / P1 | IEA、电力统计机构、EIA、并网与运营方资料 | 国家/地区/项目×时点;发电、用电、接入 MW、PUE、并网日期、可用负荷 | 国家统计月/季;报告年;项目公告日查 | 电力作为设施约束;国家发电量、用电 TWh 不能直接转换机房可用 MW;中国与美国地方数据可得性不同 |
| D09 / P1 | 项目运营方、许可部门、官方采购和工程进展 | 项目×阶段;公告、开工、设备交付、并网、验收、部分投产、撤销、金额与范围 | 已注册来源每日检查,重要项目每周核验 | 建立投产与延期标签;最终失败项目也保留,避免只收集成功项目;单纯新闻未更新不判定失败 |
| D10 / P1 | 公司投资者关系、SEC EDGAR、境内及其他市场法定披露 | 法人/分部×财季;CapEx、已付款/指引、租赁、合同、地域/业务归属 | 官方 API/报告;日查新披露,按季归档 | 用于资金与项目约束;总 CapEx 不是 AI 投资;合并报表、融资租赁、设备与机房成本须防重复 |
| D11 / P1 | 创新报告分项、公开模型评测、OpenAlex;HF 作为辅助 | 机构/国家×年/月;研发成果、质量归一化产出、可比模型能力、应用指标 | 原报告年;评测周;研究元数据月 | 重建不含重复资源项的 O;合作成果使用明确分摊;不从作者姓名推断国籍;下载量不代表实际使用或产业产出 |
| D12 / P2 | 晶圆制造、先进封装、HBM、关键设备的正式披露 | 工厂/供应方×工艺/产品×季度;已投产能力、良率范围、交付、客户与依赖 | 公司/监管原件;月查、季度快照 | 建立瓶颈和替代关系;晶圆数不能无依据换成 GPU 数;总行业份额不能归给某集群 |
| D13 / P2 | USGS、各国统计、UN Comtrade | 商品×国家×年/月;产量、储量、加工量、贸易流、HS 版本和单位 | 年度报告、月/年贸易数据 | 扩展矿产与资源依赖;储量、开采和加工分开;宽泛服务器税号不能反推高端芯片数量 |
| D14 / P2 | 既有海缆、PeeringDB、BGP 与已证实连接资料 | 设施/网络/关系×有效期;容量声明、连接、依赖、故障 | 复用现有产品,按其真实更新节奏 | 提供连通性和韧性背景;BGP 可见性不是带宽,海缆容量不是集群内部互联性能 |
EIA、SEC、Epoch、MLCommons、IEA、USGS 已核对资料入口;UN Comtrade 门户可见,但本次未验证可调用 API,接入要单独完成认证、额度、字段和历史测试。OpenAlex 需在接入时按最新访问规则配置,不假定无限制匿名访问。所有付费、登录和受限数据只列为可选增强,不作为首版隐藏前提。
### 4.3 首批采购与补数优先顺序
1. 先完成现有 Epoch/TOP500 的真实性与时间口径修复,避免新增来源扩大错误。
2. 导入信通院历史、创新分差和原始出处;追索创新原报告分项及最新版总算力数据。
3. 建立中美重点设施及项目阶段台账,并记录未知量。重点设施用于高影响解释,不冒充全国抽样无偏估计。
4. 用官方规格与 MLPerf 建立少量明确工作负载的换算表;其余硬件维持理论值或范围。
5. 补项目级并网、交付、资金与退役记录,积累可校准的真实标签。
6. 最后扩展制造、HBM、矿产与多方关系;每个新主题必须有对应指标和展示消费者,不只收集而不使用。
## 5. 采集到正式数据的流水线
```mermaid
flowchart LR
A[注册的数据源] --> B[原文与内容哈希]
B --> C[确定性解析或 AI 提取候选]
C --> D[单位/时间/实体/引用校验]
D --> E[版本化有效观测]
D --> F[冲突与待核实队列]
E --> G[确定性指标计算]
G --> H[正式结果快照]
H --> I[Outbox 与缓存更新]
I --> J[地球/双图/变化账单]
```
### 5.1 数据源合同
每个数据源保存来源机构、域名与下载入口、文档类型、凭证配置引用、检查频率、预期粒度、单位与精度、更新时间语义、空结果语义、历史范围、使用许可、维护负责人和解析器版本。授权文件或数据保存在运行数据存储,不提交到仓库;公开 UI 只展示允许公开的摘录或链接。
API/CSV 优先;HTML/PDF 采用固定解析;必须用 AI 时先产生候选,核验后保存确定性映射。扫描件 OCR 必须保留页号、表格区域和原图,数值不能只靠模型复述。下载器在流式读取时限制实际字节量,不能下载完整文件后才截断;按域名限速,遵守来源规则。
### 5.2 事实键与时间
观测键至少包括实体、指标、单位、精度、工作负载、地理范围、归属口径、参考期及来源版本。历史至少区分:
| 时间 | 含义 |
| --- | --- |
| `reference_start/end` 或 `effective_at` | 数据描述的年度、季度或设施事件发生时间 |
| `published_at` | 资料何时首次公开;不知道就保持未知 |
| `retrieved_at` | Planet 何时取得该版本 |
| `recorded_at` / `superseded_at` | 系统何时采用、替换该观测 |
抓取日期不代替投产日期;报告年份不代替参考年份;机构估计、实测、公告、模型估计和人工情景分别标识。只有年份时保留年份精度,不能伪造 1 月 1 日投产;回放按时间范围或“当年”事件显示。
### 5.3 合并与质量门槛
- 稳定身份优先使用官方系统 ID、项目 ID、法人 ID 和可信来源链接。TOP500 排名只作为观测属性。自动实体合并必须可撤销,保存别名和合并理由。
- 同一集群的全量扩建公告、一期/二期、重复转载与另一个数据集的同一设备只计一次。芯片交付和机房投产分别保留,不能合并为两个已投产资产。
- 国家级统计、企业所有权统计、设施地理统计是不同视图。国家公司在海外租用算力不得同时作为两国境内部署能力;全球成员集合固定且版本化。
- 转换单位前先核验浮点精度、稀疏/稠密、峰值/实测和时间量纲。FLOP 与 FLOP/s、TWh 与 MW 均不得互换。
- 解析为空、字段突然缺失、总量异常下降或源结构改变,进入失败/隔离状态,保留上次正式结果。缺榜、缺行或匿名化不直接触发实体退役。
- 多条转载归到同一 `source_family`;媒体数量不构成多份独立证据。冲突先检查口径,再保存多个主张及采用规则,不能简单取平均。
- 缺坐标不影响已知国家统计;未知国家不强行分配。数据覆盖率可描述已登记实体或必填字段的完整率;无法知道总体时不声称“覆盖全球百分之多少”。
- 一位有效数字等取整值保留取整范围;这是观测精度范围,不冒充统计置信区间。
首批历史导入先双人或规则加人工抽查核验高影响记录,保存一套回归样本。后续同版本确定性解析通过固定门槛可自动采用;新来源、新语义或无法解释的大幅变化进入候选队列。
## 6. 数学模型如何工作
### 6.1 分清事实、估计和评价
模型分三层:观测层保存来源说了什么;估计层回答可用能力、交付时间和不确定性;评价层按明确价值取向组合指标。国家综合实力没有一个可直接取得的标签真值,因此不能声称“AI 找到了客观最优的国家评分权重”。
保留 v2 的模型草案:
\[
S_{c,t}=0.40H_{c,t}+0.30O_{c,t}+0.20G_{c,t}+0.10R_{c,t}
\]
H 是资源能力,O 是不重复的创新与应用表现,G 是扩张潜力,R 是韧性;四项均为 0—100 的版本化子分。S 称为“综合态势分”,包含未来潜力,不能称作已投产算力、胜率或国家实力百分比。权重是默认设计参数,需通过敏感性分析和评价目的评审后冻结。
### 6.2 资源能力 H 与两类算力模式
总算力模式使用 D01 的同口径绝对规模;AI 训练模式使用明确工作负载的有效能力;推理模式另选固定模型、质量、上下文与时延要求。不同模式不共用没有物理依据的换算,也不比较彼此总分。
\[
C^{eff}_{c,t,w}=\sum_{j\in deployed(c,t)} C^{peak}_{j,t,p}\,u_{j,t,w}
\]
`p` 是固定精度,`w` 是参考工作负载;`u` 由可比测试、已知运行约束或经校准的估计支持。没有依据时只发布理论能力,或者明示区间;不能统一假设某国芯片只有另一个国家的固定折扣。
内存、互联、软件和电力会影响同一个有效利用系数,不得分别随意乘一串折扣重复惩罚。若已知场地供电上限,可用场地总功率除以 PUE 约束 IT 负荷,再按同配置效率估计可运行容量;已在利用系数中计入的限电不再扣一次。总吞吐与可用于单次大训练的最大连续集群另列。
国家宏观估计和公开集群样本并列校验,不直接把二者相加。若 H 只能覆盖已观测设施,分数标题和参与方排名必须明确限定为这个样本,不外推为全国真实能力。
### 6.3 创新与应用 O
完整创新指数 I 单独显示。取得 D04 分项后,建立指标归属表,逐一标注资源、扩张、韧性或创新;只把不重叠的研发质量、模型/科研成果与应用表现用于 O。不是简单执行“创新总分减去算力分”,因为不同分项的权重和标准化可能不同。
备用指标可来自 D11,但新增指标意味着新定义和新版本,不能偷偷替换原报告。论文数量须去重、处理合作分摊及引用年龄;模型结果要冻结评测版本;应用指标明确平台样本。专利、下载、融资和模型排行榜均不是彼此的等价替代。
\[
D^I_t=I_{US,t}-I_{CN,t},\qquad v^I_t=(D^I_{t-1}-D^I_t)/\Delta years
\]
2023—2025 年披露分差缩小 4.07 分,约为起点分差的 18.5%;不能解释为中国能力提高 18.5%,也不能据此推断算法效率提升。只有差值不能恢复各国绝对得分。跨年方法不一致时分段显示,不能外推追平时间。
### 6.4 扩张潜力 G
\[
G^{raw}_{c,t,h}=\sum_{j\in pipeline(c,t)}P(T_j\leq t+h\mid X_{j,\leq t})\,\Delta C^{eff}_j
\]
默认 `h=12 个月`。X 只含当时可知的项目状态、设备交付、并网、建设和资金证据。分期投产拆分未交付增量;已投产部分进入 H 后从剩余 G 扣除。公告金额不直接转成已投产能力。
初期采用明确的保守/基准/乐观情景,没有历史标签就不提供伪精确概率。标签积累后可比较阶段条件概率、校准的逻辑回归或生存分析;处理尚未到期项目的右删失、取消与部分交付,不能把所有未完成记录标成失败。
### 6.5 韧性 R
\[
R_{c,t}=100\sum_s q_s\,\min(1,C^{eff}_{c,t,s}/C^{eff}_{c,t,base})
\]
s 是对各方一致定义的冲击情景,例如特定供应类别中断或并网延迟;q 为公开、固定的情景权重,除非有概率依据,否则不称为发生概率。基准能力为零或未知时不计算此比值。
关系图用于识别依赖、替代和共同故障源。没有替代来源证据就标为未知;政策公告通常改变未来供应或可用性假设,不自动删除境内现有芯片存量。
### 6.6 标准化与发布条件
数量型正向指标可采用固定基期映射:
\[
N(x;a,b)=100\,clip\left(\frac{\ln(1+x/a)}{\ln(1+b/a)},0,1\right),\quad a,b>0
\]
a、b 与 x 单位一致,来自有覆盖说明的固定参考面板,存入模型版本。成本型、强度型和比例型指标各自定义映射。不得按每日领先国家重定标,也不能因加入一个国家导致全部历史分数被静默重写。
关键输入缺失时 `S=null`,保留可用子分,不填零或自动重分配权重。O、G 或 R 没有可比输入时综合分不上线;原始图表和证据浏览仍可上线。样本少、匿名取整或来源偏差反映在不确定性和适用范围中,不直接扣国家实力分。
不确定性计算分开保存观测精度、参数估计和情景假设。只有有依据的输入分布才运行 Monte Carlo;共享芯片规格、同一公告、同一供应链的误差必须相关采样。无法量化的覆盖偏差直接披露,不能靠窄置信区间掩盖。排名用“无法区分”处理不稳健结果,不展示过多小数制造精确感。
### 6.7 变化与归因
\[
g_{c,t}=C_{c,t}/C_{c,t-1}-1,\quad p_{c,t}=C_{c,t}/\sum_kC_{k,t},\quad b_{CN,t}=C_{CN,t}/(C_{CN,t}+C_{US,t})
\]
全球分母必须完整定义;双边份额不得标为全球份额。按报告披露值,2022—2023 年中国总算力从 302 增至 435 EFlops,约增长 44%,份额从 33% 降至 31%。这是“绝对增长、相对份额下降”,不是存量消失。算力与创新现有历史窗口不同,不能拼成同一时期的因果结论。
同权重、同版本下,分项账单严格满足:
\[
\Delta S=0.40\Delta H+0.30\Delta O+0.20\Delta G+0.10\Delta R
\]
分项内非线性与交互影响可使用固定分组的 Shapley 分解;必须保存分组、基线和近似误差,不能把任意计算顺序产生的贡献当唯一解释。数学归因只解释模型输出,不证明宏观因果。
正式发布中始终分开:现实变化、补录旧事实、来源修订、模型/基期修订。版本升级用“同一输入、不同模型”的桥接结果展示,不能把换模型造成的跳变计入国家增长。
## 7. 项目内 AI 如何参与
### 7.1 运行方式
复用控制台已经配置的模型,通过 `AIProviderClient` 调用;不要求新的模型供应商,也不把供应商名称写死在业务代码中。第一阶段采用后端固定步骤工作流,各角色是任务模板,可以由同一个模型执行,不需要首先建设复杂的多 Agent 协商系统。
在现有提示词注册表中新增以下拟议 task key;提示词版本与运维覆盖的有效内容哈希进入每次运行记录。
| 任务模板 | 输入 | 允许输出 | 关键约束 |
| --- | --- | --- | --- |
| `compute.sources.discover` | 指标缺口、许可范围、已注册来源 | 来源候选、适用指标、采集建议 | 先找原始出处;新域名不直接进入正式定时采集 |
| `compute.evidence.extract` | 原文分块、页码、表格、目标 schema | 带证据定位的字段候选 | 未出现的数值返回缺失;原文中的指令只是待分析内容 |
| `compute.evidence.review` | 候选、对照记录、单位与实体规则 | 冲突类型、支持/反驳证据、待核验项 | 第二个 AI 的同意不是独立事实证明 |
| `compute.gaps.prioritize` | 缺失、来源健康、参数敏感性、采集成本 | 下一步补证清单 | 优先可能影响结论且可查证的缺口,不以国家倾向排序 |
| `compute.change.explain` | 已计算的贡献账单、事实版本 | 有引用的可读解释 | 不能修改计算结果;事实、估计、假设分别表达 |
| `compute.model.propose` | 误差报告、数据质量问题、模型配置 | 有边界的参数/映射/新特征提案 | 必须说明可证伪假设、预期改善、影响范围和回退方案 |
| `compute.model.review` | 候选方案、独立评估结果、影响报告 | 发布建议与限制 | 无权改变测试标签、门槛、保留集或自己批准自己 |
结构化结果由后端 Pydantic schema 校验。当前 AI 返回以文本为主,需要新增结果解析与受限重试;JSON 不合格、引用不匹配、单位冲突时运行失败或保留候选,不能将自然语言当作正式数据。
### 7.2 工具与权限
允许工具包括:参数化只读指标查询、已注册来源搜索/抓取、文档段落读取、规则化单位转换、实体候选查询、固定估计器运行、创建缺口任务、保存提案、提交实验任务。
AI 不直接写正式分数、覆盖原始证据、执行任意 SQL/Python/Shell、修改发布门槛或扩大自己的工具权限。生成的采集映射必须通过回放样本后才保存;新增可执行采集器代码属于普通开发和发布流程,不在生产 Agent 内 `eval`。
网页抓取需校验协议、主机、解析后的目标地址及每次重定向,阻止访问内网、回环和元数据端点;使用专门外部抓取边界,不能把内部业务 API 和外部取证 URL 混用。AI 只得到所需公开证据或经授权的业务摘要,凭证由后端设置解析,不进入提示词、引用或日志。
### 7.3 用户能看见的 AI 结果
每份分析展示三部分:已验证事实、模型解释、仍缺哪些证据。每条数值和关键结论可展开来源。用户可以标记“实体合并错误”“单位/年份错误”“证据不支持”“假设不合理”,形成结构化反馈;收藏、点赞和是否喜欢国家排名不能成为事实标签。
“AI 发现数据不足”可以是成功结果。模型不可用时,确定性采集、计算、双图和地球继续工作,解释面板保留上次结果并标注版本与时间。
## 8. 数学模型如何自我迭代
### 8.1 自动迭代分成三类
| 层次 | 可以怎样改进 | 自动程度 |
| --- | --- | --- |
| 证据和数据 | 补充来源、修正字段、发现重复和过期、改进解析映射 | 已核准来源与已验证规则可自动运行;新语义及高影响冲突先进入候选 |
| 可验证的估计参数 | 项目延期分布、特定硬件/任务效率、记录错误率 | 通过冻结评估、影子运行和预先配置的边界后,可以自动发布小范围参数更新 |
| 评价定义 | H/O/G/R 权重、指标含义、基期、国家范围和价值取向 | 默认生成提案,由模型负责人确认版本;不是机器从不存在的“国家实力真值”中自动学习 |
此处是产品建议的默认发布策略,不是在本次计划工作中申请执行权限。未来可配置自动发布的参数白名单,但不能用“开启自动迭代”笼统授权所有定义变化。
### 8.2 迭代闭环
```mermaid
flowchart TD
A[新事实/人工纠错/成熟预测结果] --> B[确定性误差与质量评估]
B --> C[AI 提出可证伪改进]
C --> D[生成候选配置与锁定实验]
D --> E[按时间回测及国家分组评估]
E --> F{是否通过预设门槛}
F -->|否| G[记录失败原因并保留现行版本]
F -->|是| H[影子运行:结果不改变正式排名]
H --> I[按发布策略评审或受限自动提升]
I --> J[发布不可变版本与变动桥接]
J --> K[监测退化并可原子回滚]
K --> B
```
每次提案只改变一种主要因素,或者明确作为一个联合实验;不能同时换来源、标签、权重和评估方法后宣称单个参数有效。失败提案也保存,避免只展示成功实验。
提案记录至少包括:问题、证据 ID、基线版本、拟议配置差异、适用实体/任务、可检验目标、锁定数据截止时间、主要评估指标、停止条件和回滚版本。可以提出“将并网阶段纳入交付预测”,不能只提出“让某国得分更合理”。
### 8.3 真实反馈标签从哪里来
| 估计对象 | 可观察标签 | 评估方式 |
| --- | --- | --- |
| 文档字段提取 | 人工核对原文的数值、单位、年份、实体和引用位置 | 字段精确率/召回率、关键错误、引用支持率,按语言/文档类型分组 |
| 实体合并 | 官方 ID、项目连续记录或人工确认的同一/不同实体 | 错误合并率和遗漏合并率;高影响错误单列 |
| 项目按期投产 | 后续正式验收/投产/取消记录,保留观测截止 | Brier score、可靠性图;右删失与失访不能计作失败 |
| 新增容量 | 未来实际确认的设备与容量,具有同口径 | 绝对误差、对数误差;同时报告项目等权与容量加权结果 |
| 任务性能系数 | 未参与拟合的可复现实测配置 | 相同任务质量下的预测误差与区间覆盖 |
| 区间预测 | 后续实际值是否落入范围及区间宽度 | 覆盖率与区间评分一起评价,避免靠无限放宽范围“提高准确率” |
不使用 AI 自己生成的摘要作事实标签,不用现行总分拟合下一版总分,不把另一个复合指数直接当国家实力真值。完整创新指数可做外部一致性讨论,但若其包含输入指标或与 O 重叠,就不能作为独立验证集。
### 8.4 回测与数据泄漏控制
预测训练、验证和保留集按真实可获得时间滚动切分,同一项目的后续阶段、同一公告转载和同一扩建链必须归组,避免分散进训练和测试。每个特征都要求 `published_at <= forecast_cutoff`;仅今天抓到而无法证明当时可得的资料,不参加“当时可知”回测。
新模型先与简单基线比较:维持上次状态、沿用公告日期、同阶段历史交付中位数。不得只与一个刻意弱的旧模型比较。参数、标准化锚点和来源质量估计只从训练窗口取得。
大模型的训练记忆可能知道后来结果,所以历史回测中的预测器必须是只读取冻结特征的确定性估计器;不能让 LLM 凭记忆补旧事实。提案生成与真实保留集隔离,实际前瞻影子运行仍是必要证据。
按国家、硬件代际、项目规模、来源语言分别评估,并按项目/运营方/时间块进行重采样,不能把高度相关的重复记录当独立样本。小样本报告区间与不足,不为满足仪表盘而宣称显著改进。
### 8.5 建议发布门槛
以下是实施阶段的初始门槛草案,需在 M0 用试点数据冻结。它们是最低操作条件,不保证统计有效性,AI 无权自行降低。
| 门槛 | 初始建议 |
| --- | --- |
| 硬正确性 | 单位、时间穿越、重复计数、采集失败保留正式值等必测案例全部通过 |
| 提取评估 | 至少 200 条人工核验字段、覆盖中英文及主要文档类型;报告精确率/召回率及区间;关键数值/单位错误不得进入自动采用集 |
| 预测参数自动提升资格 | 至少 50 个有可验证结果的独立项目、覆盖至少 3 个滚动窗口;每个主要参与方至少 10 个,不足则继续影子运行或人工评估 |
| 改善幅度 | 预先指定主要损失相对现行版和简单基线至少改善 5%;按依赖结构重采样的损失差区间须支持改善 |
| 分组保护 | 主要参与方损失退化不超过预先约定范围,初始建议 2%;不能以平均改善掩盖一个国家明显恶化 |
| 区间质量 | 不仅检查覆盖,还检查宽度与评分;对无法量化的来源覆盖偏差单列说明 |
| 影子观察 | 至少连续 4 周稳定运行,并达到所需成熟标签数;12 个月预测不因观察满 4 周就算验证完成 |
| 试验节制 | 初始每月最多 3 个主要候选,锁定真实保留集;多次搜索造成的选择偏差要通过新的前瞻样本再次检查 |
标签暂时不足时,系统可以自动补数、解释、积累预测与结果,但不发布“自我学习后的更准模型”。这不会阻塞第一版事实产品上线。
### 8.6 一个完整迭代例子
以下是流程示例,不是已发生的项目事实或评估成绩。
系统发现一批已订购芯片的项目未如期投产;确定性评估记录预测误差。AI 查询证据后提出:现有交付模型只使用芯片到货时间,遗漏电力接入阶段。它提交一个新增“并网是否完成”特征的候选。
后端按项目原始发表时间构造训练数据,用固定估计器拟合;锁定保留集检验总体和中美分组误差。即便论文或第二个 AI 都认为这个想法合理,只要结果未通过门槛就不发布。通过后先影子运行;正式发布时展示受影响的 G 分、预计容量和模型版本变化,已投产 H 不因推测而下降。
若后续发现并网字段提取错误导致退化,回滚活动模型指针,保存新证据和失败实验。不能删除失败记录或改写过去已发布的预测。
## 9. 后端数据与实现结构
首版采用最小领域实现:沿用 `CollectedData`、`DataSnapshot` 和位置维表,新增经过 schema 校验的国家指标记录与聚合服务。数值口径、参考期、发表时间、来源链接、原文哈希和证据定位可先作为受约束的元数据保存;必要的原件存放在许可允许的运行数据目录。只有现有表确实无法表达查询或历史约束时才增加专用表。
先冻结面向专题的 API 契约,地球与图表只消费该契约。将来拆出指标观测、证据、实验等表时,通过服务层迁移,不让前端依赖临时 JSON 或物理表结构。首版不以第 9.1 节全部表建完作为开工条件。
### 9.1 拟议持久化对象
下表为待新增的逻辑对象,可按最终数据库设计合并有限的小表;不要与现有采集、位置和任务状态形成并行真相源。
| 对象 | 保存内容 | 关联和约束 |
| --- | --- | --- |
| `evidence_documents` | 原文存储引用、原始/提取哈希、来源、发表/抓取时间、许可、提取版本 | 内容不可变;全文不在高频列表接口返回 |
| `metric_definitions` | 粒度、单位、精度、工作负载、归属、方向、适用范围和版本 | 一个稳定 metric ID;模型版本引用具体定义版本 |
| `metric_observations` | 数值/范围、参考期、知识时间、状态、来源版本、取整和缺失原因 | 链接原采集记录;追加式修订,不复制整份原始 JSON |
| `observation_evidence` | 观测与证据的支持/反驳关系、页号、表格/段落、短摘录 | 多对多;来源家族避免转载重复投票 |
| `compute_entities` / `resource_relations` | 稳定实体、别名、分期关系、所有权/供应/连接、有效期 | 复用现有位置维表;不另存一套会漂移的正式坐标 |
| `model_versions` | 指标集合、权重、锚点、参数、代码版本、情景与发布状态 | 不可变版本;活动指针独立且原子切换 |
| `model_runs` / `model_results` | 输入清单哈希、版本、截止时点、随机种子、环境、国家结果、分项和不确定性 | 结果可重放;实验、情景与正式运行分开 |
| `model_proposals` / `model_evaluations` | 配置差异、假设、冻结评估方案、损失、分组结果、决定与回退目标 | 发布必须引用通过的评估,保存失败实验 |
| `agent_runs` | task key、提示词有效哈希、模型、工具调用、引用、预算、结构化输出、终态 | 链接现有任务账本,Agent 自身不拥有另一套调度状态 |
重放保留输入观测 ID/版本清单或不可变清单文件,不能只保存一段 SQL 然后查询已被更改的数据。记录计算实现版本、依赖环境和随机种子;LLM 输出保存为证据,不要求重新调用模型才能复算数值。
历史删除与保留策略需要显式区分:普通缓存可清理;被正式模型版本引用的观测及许可允许保存的证据受保留保护。来源撤回或许可变更时标记可访问状态并保留允许保留的哈希和元数据,不强行公开原文。
### 9.2 待新增模块与现有边界
| 拟议模块 | 职责 |
| --- | --- |
| `backend/app/services/compute_intelligence/observations.py` | 规范化、有效观测选择和数据缺口 |
| `.../metrics.py`、`.../scoring.py` | 纯数值计算、单位/指标注册、标准化和总分 |
| `.../forecasting.py`、`.../evaluation.py` | 固定估计器、时间切分、基线、分组指标与实验 |
| `.../workflows.py`、`.../proposals.py` | 固定 AI 工作流、结构化候选、预算和发布规则 |
| `.../publication.py` | 正式快照、变动桥接、活动版本和 outbox |
| `backend/app/api/v1/compute_intelligence.py` | 参数化只读查询和后台任务入口 |
| `frontend/public/earth/js/compute-intelligence.js` | 专题 UI 状态、图表与现有图层联动,消费后端计算结果 |
| 控制台专题页与现有采集/AI 设置页签 | 证据核验、缺口、模型实验、版本回看 |
具体文件在实现时保持单一职责,优先复用已有服务,不为目录结构创建空文件。新来源继续注册到现有采集器体系及 datasource 配置;新提示词放入现有 `default_prompts.json`,不散落在 renderer 或 `aiprovider`。
### 9.3 拟议 API
下列接口尚不存在,名称在 M0 与当前路由约定核对后冻结。
| 方法与路径(统一前缀 `/api/v1/compute-intelligence`) | 返回内容 |
| --- | --- |
| `GET /overview` | 所选参与方、口径、参考期与知识截止下的结果、适用范围及覆盖说明 |
| `GET /series` | 指定指标的有界历史序列,含缺失、版本、实际观测时间与来源引用 |
| `GET /entities/{id}` | 设施、分期、资源关系和历史观测 |
| `GET /changes` | 数据或模型贡献账单,数据库侧分页 |
| `GET /evidence/{id}` | 根据权限和许可返回证据摘要、定位与允许展示的内容 |
| `GET /models`、`GET /runs/{id}` | 模型版本、输入清单、计算结果与评估摘要 |
| `POST /recompute`、`POST /agents/run` | 建立任务并立即返回任务 ID,允许取消和查看进度 |
| `POST /proposals/{id}/evaluate` | 创建冻结数据与方案的评估任务 |
| `POST /models/{id}/promote`、`POST /models/{id}/rollback` | 按角色及发布策略原子切换活动版本,记录理由和影响 |
查询参数固定包含 `metric_profile`、`entities`、`reference_at`、`knowledge_cutoff`、`model_version` 和可选 `scenario_id`。前端不接收任意表达式或 SQL,不自行补算另一套总分。
### 9.4 发布与性能
后台完整计算并通过质量门槛后,在事务中发布结果快照和活动指针,通过既有 outbox 唤醒刷新。新 adapter 显式声明国家面板/历史图的刷新范围;设施变化才触发相应 `computeCenters` 更新。草稿证据、失败实验和影子运行不改变默认地球。
缓存键包含口径、实体集合、参考期、知识截止、数据清单哈希、模型与情景版本。WebSocket 只通知结果版本与必要增量,客户端拒绝较旧响应;不能每次国家 hover 都重算模型或执行 LLM。
聚合、过滤、排序和分页在数据库完成;预计算国家年度/月度可用序列,地图按分辨率聚合节点。时间滑动去抖并复用已取快照,播放使用相邻状态差量。建议试点验收目标为缓存查询 P95 小于 1 秒、切换已缓存时点小于 500 毫秒,声明基准机器、数据规模和网络条件;未实测前不作为已达到性能。
## 10. 刷新、成本和失败处理
| 工作 | 建议节奏 | 触发条件与成本控制 |
| --- | --- | --- |
| CSV/API 检查 | 日或来源允许的更慢频率 | ETag/Last-Modified/哈希去重,无内容变化不调用 AI |
| 年报与指数发现 | 月度,发布季可提高 | 找到新版本再下载和提取,不把月检写成月度指标 |
| 重点项目证据 | 日查、周复核 | 去重后仅高影响变化进入提取队列 |
| 指标重算 | 有效观测或模型版本改变时 | 只重算受影响实体/时点/指标依赖 |
| AI 解释 | 有意义的结果变化或用户请求 | 相同输入哈希复用,不因页面打开重复生成 |
| 数据缺口排序 | 每周 | 敏感性、覆盖风险、可获得性和预计采集成本共同决定 |
| 参数候选实验 | 月度或成熟标签达到门槛 | 限制候选数量;无足够标签不强制产出升级 |
试点默认每工作流最多 3 次搜索、8 个页面抓取、4 次 LLM 调用、1 次结构化修复重试,最多同时运行 2 个 AI 工作流;均为可配置预算,不是第三方接口能力承诺。按全局模型配置记录 token/费用,设置日预算和单任务截止时间,超额进入延后或停止状态,不无限自递归。
失败处理:采集失败保留上次正式值和新鲜度;AI 不可用仍提供确定性结果;引用失效保留许可允许的原件和哈希;长期资料缺口显示缺失并降低结论覆盖范围;新源冲突进入待核验;模型退化恢复前一活动版本。不能用重试或回滚删除已提交有效证据。
## 11. 分期实施与依赖
工作量是规划估算,按一名熟悉项目的全栈开发者、持续可用的数据核验支持和现有环境可用计算;不是交付承诺。开发时间不包括等待付费授权、原始报告或 12 个月预测结果成熟的时间。
### 11.1 当前优先交付:展示专题
| 顺序 | 修改方向 | 验收内容 | 估计开发工作日 |
| --- | --- | --- | ---: |
| A1 修复数据与最小合同 | Epoch CSV、TOP500 日期/身份/单位、空结果保护;保存历史指标和来源 | 能得到真实当前快照与核验后的年度序列,不要求宏观指标全到最新年 | 2—3 |
| A2 国家对比与历史接口 | 国家汇总、公开样本与宏观统计分离、双图数据、证据摘要 | 绝对量/份额/创新分差有各自定义;缺失与时点可见 | 2—3 |
| A3 地球与图表展示 | 专题模式、国家面板、双图、时间联动、设施/证据详情 | 完成第 0 节完整浏览路径,并通过桌面/移动/缩放验证 | 3—5 |
| A4 可选 AI 阅读辅助 | 复用现有 client 和任务提示词,针对冻结快照生成带引用说明 | 事实与推断分开;可关闭;失败不影响地球与图表 | 1—2 |
A1—A3 是当前必须交付的闭环,约 7—11 个开发工作日;A4 可在闭环稳定后增加。实际数据源不通、现有采集回归问题或主题布局复杂度会改变估算。实现时首先验证 A1,而不是先开发通用 Agent 框架或完整综合评分。
### 11.2 后续平台能力路线
下表是完整能力建设的工作包估算,与 A 路线存在复用和重叠,不应重复相加。A 路线完成后先盘点已具备的 M0—M2 能力,只实施剩余部分。
| 阶段 | 工作包 | 交付物与通过条件 | 估计开发工作日 |
| --- | --- | --- | ---: |
| M0 合同与试点 | 固定口径、成员、来源许可、历史时间语义、角色权限、质量/评估门槛 | 指标注册表、数据源接入清单、黄金核验样本;所有未知明确登记 | 3—5 |
| M1 可信数据 | 修复 Epoch/TOP500;证据与指标观测;导入两组历史;稳定身份与失败保护 | 可查询、可追溯、可重放的历史 API;无示例数据进入正式结果 | 7—10 |
| M2 可见产品 | 地球专题、双图、国家对比、变化账单、证据抽屉和控制台缺口页 | 中美历史回放闭环;移动/缩放和旧图层回归通过;综合分可保持未就绪 | 7—10 |
| M3 资源与潜力 | 芯片、实测、项目、电力/资金台账,场景与分项估计 | H 与可用 G/R 子项具备来源;O 仅在分项齐备后发布;总分门槛满足才开放 | 8—12 |
| M4 AI 工作流 | 固定提取/核验/补证/解释流程、反馈和预算;扩展可靠任务类型 | AI 只能产生有引用候选与解释,失败不会污染正式数据 | 7—10 |
| M5 模型迭代 | 真实标签、时间回测、候选评估、影子运行、发布桥接与回滚 | 能证明某个可观察估计目标改善;不要求每轮必有新模型 | 8—12 |
从零完整建设上述广义平台能力约 40—59 个开发工作日,包含比 A 路线更完整的证据和实验治理;它不是当前展示专题的工期。影子期至少四周,但不成熟的预测标签会延长自动提升资格等待期。
依赖顺序:M0 → M1 → M2;M3 的项目台账应尽早开始积累标签;M4 依赖 M1 的可信证据结构;M5 依赖 M3 的标签和 M4 的审计工作流。多方、矿产和复杂供应链在 M2 之后按数据质量逐个扩展,不等全部主题齐备再上线。
## 12. 验收与测试矩阵
| 范围 | 必测案例 | 通过标准 |
| --- | --- | --- |
| 来源真实性 | HTML 改版、空 CSV、404、部分下载、示例字符串 | 正式数据不被示例或空结果替换;错误原因可见 |
| 单位与精度 | T/P/E 换算,FP64/FP16/FP8,稀疏/稠密,MW/TWh,FLOP/FLOP/s | 不同口径拒绝合并;可比转换有可复算结果 |
| 时间与版本 | 固定抓取日误作投产日、2026 报告回填 2023、未知发布日期 | 严格截止查询无未来信息;历史回顾明确重述 |
| 实体与去重 | 排名换位、扩建、名称变化、匿名化、重复转载、跨源同集群 | 身份稳定;不重复计数;匿名或缺榜不自动退役 |
| 统计与地理 | 缺坐标、未知国家、欧盟与成员国、跨境所有权 | 国家统计不依赖坐标;分母不重叠;归属视图不混用 |
| 两张历史图 | 已核验年份值、缺失年份、2026 待核实主张 | 只展示有证据的观测;不能伪造平直历史或默认预测 |
| 综合评分 | 缺 O/G/R、权重和不为 1、标准化越界、国家集合变化 | 非法配置拒绝;缺失总分为 null;固定版本不静默漂移 |
| 变化账单 | 新投产、补录、源修订、换模型 | 分项贡献之和满足计算容差;类型和版本差异可解释 |
| AI 输出 | 格式错误、虚构引用、原文含指令、模型超时、重复任务 | 输出隔离;工具边界有效;任务幂等并能取消 |
| 模型评估 | 标签穿越、同项目跨集、过拟合小样本、分组退化 | 不通过门槛不发布;保留失败实验 |
| 发布与回滚 | 两个并发发布、计算中断、旧 WS/HTTP 响应 | 原子版本;半成品不可见;客户端不回滚到旧时点 |
| 地球渲染 | 远距/近距、时间播放、层隐藏、节点锁定、场景退出 | 无闪烁和悬空 tooltip;实例复用,不每帧重新建纹理 |
| UI | 桌面、移动、125%/150% 缩放、键盘、无数据/低置信/加载状态 | 一屏工作区无意外全局滚动;关键标签不靠颜色区分 |
实现阶段按变更执行后端精确测试、`scripts/harness/quick-check.sh`、Bun 前端构建和渲染 smoke;地球改变不能只凭构建通过。保持现有公共页面、鉴权、管理路由、安全导航及其他地球图层的回归证据。性能阈值用记录过的基准规模验证,不能把开发空数据测试称为生产验收。
## 13. 文档、发布与完成定义
每阶段 PR 和部署使用已有项目流程,功能旗标分别控制专题 UI、估计分项、AI 工作流与自动参数提升。先开放读取与事实浏览,再启用模型实验;关闭新旗标应回到原有算力图层而不丢历史数据。
发生真实用户工作流变化时更新中英文手册与快速入门;采集、AI、作业与位置变化更新相应后端技术文档;控制台、Earth、图层顺序和样式变化更新对应 context 与规范。新增公共技术文档需要同步中英文和 Docs 注册;本文件是内部中文计划,不注册为已实现的公共手册。
当前展示专题完成定义:用户完成第 0 节浏览路径,可以从国家指标变化回到出处和所用计算版本;重新计算得到同一数值;知道哪些资料尚缺。两张图应和国家、设施、时间、证据联动,不能只是贴在地球旁的静态图片。
后续决策辅助与模型迭代完成定义:情景的假设与结果可复算;AI 的一次成功改进有独立验证目标、评估记录和可回滚版本。只有一段 AI 分析或未经校验的总分,不代表完成决策辅助能力。
## 14. 风险与预先决定的退路
| 风险 | 产品和实施处理 |
| --- | --- |
| 无法取得创新原报告分项 | 保留完整指数独立观测;O 和总分不发布,不伪造拆分 |
| 中国或其他地区公开设施更少 | 显示宏观统计与设施样本两个覆盖视图,扩充本地语言原始来源,不把保密或匿名化当能力下降 |
| 硬件真实利用率不可知 | 理论量先行,明确任务和区间;不把实验室 benchmark 当生产实测 |
| 来源改版、匿名化或许可证变化 | 版本化下载、质量隔离、许可范围内保存证据,必要时停更单一来源 |
| 资金、电力、芯片相互重复计分 | 固定指标归属与约束链;不把投入、存量、产出和事件各加一次 |
| 训练标签不足或结果多年后才成熟 | 先提供情景与证据改进,延后参数自动提升资格 |
| 模型迎合既定国家排名 | 预先定义目标和门槛,评价权重单独评审,不以用户喜好或另一个 AI 打分训练 |
| 已有采集/定位状态被新服务覆盖 | 复用当前真相源,使用明确字段所有者、发布快照和回归测试 |
## 15. 来源与相关文档
以下链接支持数据口径、来源可得性和架构选择;除明确标注的历史值外,本计划的字段、频率、权重、工期和发布门槛均是拟议设计。
- [信通院 2021 年版算力白皮书(镜像)](https://pdf.dfcfw.com/pdf/H3_AP202109271518811781_1.pdf):2020 年总算力份额。
- [信通院 2022 年版白皮书(镜像)](https://13115299.s21i.faiusr.com/61/1/ABUIABA9GAAgvq2fmwYozLrrpwU.pdf):正文第 13—14 页,2021 年份额与口径。
- [信通院 2023 年版白皮书(镜像)](https://www.ahchanye.com/wp-content/uploads/2023/09/2023092815342684.pdf):正文第 13 页的 2022 年份额,以及 302 EFlops 总规模。
- [信通院 2024 年版蓝皮书(镜像)](https://pdf.dfcfw.com/pdf/H3_AP202502051642799257_1.pdf):正文第 10 页的 2023 年份额及后续规模说明。
- [科技日报关于创新指数报告的报道(新浪转载)](https://finance.sina.com.cn/tech/roll/2026-07-17/doc-iniickaz8224252.shtml):分差与综合指标维度;未替代原报告分项。
- [2026 年 42%/33% 候选出处](https://www.mornai.cn/news/gpu/ai-computing-power-never-sleeps/):待核实商业文章,不作为正式统计。
- [Epoch GPU 集群公开数据](https://epoch.ai/data/gpu-clusters)、[字段与下载说明](https://epoch.ai/data/gpu-clusters-documentation):CSV、地理字段、覆盖与公开版本限制。
- [Epoch 芯片所有权](https://epoch.ai/data/ai-chip-owners):所有权不等于使用权或所在地,交付与投产有时间差。
- [MLPerf Training](https://mlcommons.org/benchmarks/training/)、[MLPerf Inference Datacenter](https://mlcommons.org/benchmarks/inference-datacenter/):固定任务、质量及测试配置。
- [IEA Energy and AI](https://www.iea.org/reports/energy-and-ai)、[EIA 开放数据](https://www.eia.gov/opendata/):能源资料入口,不能直接替代设施并网证据。
- [SEC EDGAR API 说明](https://www.sec.gov/search-filings/edgar-application-programming-interfaces):公司提交与财务数据访问;需注意财季与标签口径。
- [USGS 矿产概要](https://www.usgs.gov/centers/national-minerals-information-center/mineral-commodity-summaries)、[UN Comtrade 门户](https://comtradeplus.un.org/):矿产与贸易候选数据源。
- [OpenAlex 帮助与数据说明](https://help.openalex.org/):研究元数据候选来源;接入时核验访问与归属规则。
- [业务架构与数据流转](../technical/zh/platform-data-flows.md)、[后端采集器](../technical/zh/backend-collectors.md)、[AI Provider 边界](../technical/zh/agents-aiprovider.md)。
- [智能星球前端上下文](../technical/zh/earth-frontend-context.md)、[渲染图层顺序](../technical/zh/earth-render-layer-order.md)、[图层视觉规范](../technical/zh/earth-layer-style-reference.md)。
- [轻量 Agent 编排](agents-light-orchestrator-websearch-plan.md)、[态势感知基础计划](agents-situational-awareness-foundation-plan.md):复用总体方向,不重复建设 provider 内业务编排。

View File

@@ -0,0 +1,154 @@
# Planet 开发 Agent 模型分配与 Token 控制计划
日期:2026-09-21。状态:已用于本轮地理与空间卡片开发;未改变全局或项目默认模型配置。适用于已授权的开发任务,不代替用户对任务范围的决定。
目标:在固定政治展示规则、现有 UI 逻辑和性能门槛下,降低完成一个合格任务的总成本与返工次数。不是让所有工作都使用低价模型,也不是让最强模型从头包办。
当前重点任务见 [Earth 地理资源与空间卡片计划](earth-spatial-presentation-and-geographic-assets-plan.md)。该计划的功能与验收要求优先;本文件只决定如何组织开发。
## 1. 模型角色
模型定位参考 OpenAI Docs;下表的具体分工是针对 Planet 的工程建议,需要结合实际一次通过率和返工记录调整,不是模型能力保证。
| 模型 | 精确模型 ID | 默认推理强度 | 分配的工作 | 不作为默认用途 |
| --- | --- | --- | --- | --- |
| Astra | `gpt-6-astra` | `high` | 跨系统难题、空间展示方案、复杂坐标/深度/性能诊断、关键设计与独立审查 | 日常文件搜索、机械编辑、反复运行已确定命令 |
| Sol | `gpt-5.6-sol` | `medium`;复杂实现用 `high` | 执行主控、核心代码、共享组件整合、Three.js 状态与生命周期、最终集成 | 逐条整理大批重复记录、重复复述整个项目背景 |
| Terra | `gpt-5.6-terra` | `medium` | 接口已明确的常规实现、局部 UI 修复、数据处理脚本、测试编写、浏览器回归 | 独自决定空间交互方向、坐标模型或跨模块架构 |
| Luna | `gpt-5.6-luna` | `low`;明确的小修改可用 `medium` | 定向检索、清单比对、日志归类、说明文档初稿、固定格式整理 | 核心渲染、数据迁移、几何归属修改、最终审美或性能放行 |
日常推荐组合是 Sol 主控加一个 Terra worker;Luna 和 Astra 按需进入,不要求每个阶段凑齐四种模型。已经完成的资源准备不因采用这份方案重新做一遍。
当前会话的主模型不会因为写入计划自动变成 Sol。真正执行时,主会话模型由实际任务设置决定,子 Agent 在派发时明确指定模型和推理强度,并核对返回的运行配置;不能把角色名称当作已经切换模型的证据。
## 2. 本项目任务分配表
| 工作包 | 主执行 | 独立验证/审查 | 交付和升级条件 |
| --- | --- | --- | --- |
| 现有窗口、连线、巡航、滚动条入口盘点 | Terra | Sol 检查引用与遗漏 | 文件/符号清单、现有行为证据;Luna 可辅助定向搜索,无需读取全仓库 |
| 贴图归档、尺寸、哈希、配准元数据检查 | 直接运行脚本;脚本需新增时 Terra | Sol 抽查实际输出 | S0 已完成,后续只做有变化的核对,不委托模型重复计算哈希 |
| 固定政治规则的技术合同与几何覆盖边界 | Sol | Astra 审查规则是否贯穿各数据路径 | 权属口径已由项目确定,任何模型均不得重新决定;标识不明确先核对,不能猜测几何 |
| 贴图、海岸线、拾取错位定位 | Sol `high` | Astra 在根因不清或涉及多个坐标层时介入 | 用明确经纬点复现,区分错误数据、回退、投影、UV 与半像素偏移 |
| 规则已明确的映射、来源清单、局部转换脚本 | Terra | Sol;关键几何结果由 Astra 抽查 | 只落实批准合同;发现新的几何语义不得自行扩大规则 |
| 多区域空间卡片的第一版运动设计 | Astra `high` | Sol 检查能否复用当前实现 | 明确锚点、投影、避让、换侧、背面及阅读稳定性;产出小合同,不产出整套 UI 大改 |
| 2 张卡片的连接、弹性移动、生命周期原型 | Sol `high` | Terra 实测;Astra 看关键动效证据 | 优先实现最小完整循环;使用参考图和实际运动证据,不用文字自评代替 |
| 扩至 4/6 卡、增删区域和事件整合 | Sol | Terra 跑交互/竞态测试 | 复用原型合同;`main.js` 等共享入口只由指定 owner 写入 |
| 共享窗口行为、标准字体、移动抽屉及深色滚动条 | Terra | Sol 看渲染与旧面板回归 | 修改公共滚动所有者,验证图层等所有相关页;涉及共享状态重构时升级给 Sol |
| 地球柔光、材质层次、节点与背景关系 | Sol | Astra 对照 UE/PPT/海报审查 | 第一轮只改少量确定参数并给实际截图;性能退化交 Sol 诊断,难题交 Astra |
| 卡片内指标、普通图表、加载/缺失/错误状态 | Terra | Sol | 数据合同先冻结;不复活固定全屏大面板,不编造缺失年份 |
| 国家/区域接口和前后端联动 | Terra;跨模块变更由 Sol | Sol | 不硬编码中美为唯一对象;范围筛选、比较对象和组合去重必须一致 |
| 性能基准工具和回归测试 | Terra | Sol 检查方法;Astra 审查疑难退化 | 固定设备、数据与相机操作;日志含帧时间和内存证据,不能只报平均 FPS |
| 单位、日期、缓存、旧请求覆盖等复杂数据故障 | Sol | Astra 仅处理跨系统未解问题 | 先给最小复现和失败测试,避免多个模型同时试不同修复 |
| 中英文手册、资源清单、计划状态同步 | Luna 初稿;技术性较强时 Terra | Sol 对照实际 diff | 已实现与待实施分开;Luna 不能自行宣告功能完成 |
| 最终集成与验收 | Sol | Astra 对关键风险做一次集中审查 | 相关测试、渲染、政治规则及性能证据齐全;普通修订不逐项再次走 Astra |
## 3. 顺序与并行安排
### 第一轮:证据与边界
Sol 读取现有计划、当前工作区差异和必要规则,整理短任务清单。Terra 盘点实际运行的数据提供者、组件和性能基线;独立且批量的文档/引用整理才交 Luna。先读已有检查结果,避免重新下载贴图、重新研究 World Monitor 或重复跑已经通过的检查。
Sol 将确定的组件接口、规则约束、实际截图与未解问题整理成一个短包,Astra 只审查高返工风险的空间方案和坐标问题。Astra 返回决定、理由和验证点,不再从零扫描整个仓库。
### 第二轮:最小空间原型与独立 UI 修复
Sol 实现 2 张卡片的锚点、连线和有阻尼移动;Terra 可以同时处理公共滚动条或写回归测试,前提是文件所有权不冲突。先完成最小可演示运动,再扩展内容和卡片数量。若两者都要改 `controls.js`、`hud.css` 等共享文件,则按顺序交接,不并行写入。
Astra 对原型的实际截图/录屏做一次聚焦审查,检查是否符合“从地球四周延伸”的方向。空间原型尚未确认时,不展开完整指标卡片和全量国家 UI,以免重演方向错误后大规模返工。
### 第三轮:内容、全球扩展与性能
合同稳定后,Terra 接入指标与状态,Sol 负责与渲染/生命周期的整合。性能测试由脚本执行,Terra 整理差异;可复现的退化先交 Sol。只有不能局部解释的 GPU/CPU、坐标、异步问题才交 Astra。
### 第四轮:一次集成验收
Sol 整合并运行所需检查;Luna 根据最终 diff 更新文档初稿。Astra 集中审查关键几何、政治规则完整性、卡片运动与性能结论。已通过范围不反复重测,除非新修改影响该范围。
## 4. 并发上限与文件所有权
当前环境总并发槽位为 4,含主 Agent;执行时仍以实际可用额度为准。默认使用主 Agent 加 1 个 worker。只有存在两项以上真正独立的工作,且主 Agent 同时有有用工作可做时,才扩大到主 Agent 加 2 个 worker;第 4 个槽位只给临时审查或独立批处理。
每项任务指定可写文件范围。`main.js`、`controls.js`、共享 HUD/InfoCard 与跨层状态接口同一时间只有一个写入 owner。其他 Agent 只读检查或提交补丁建议,由 owner 集成。不得并行启动多个浏览器重测相同场景,也不得共享有冲突的登录/本地存储测试状态。
浏览器验证优先使用当前可用的 Browser 控制能力,包括统一 CUA 入口。不能仅因技能目录中缺少单独的 Browser 条目就判定工具不存在;应继续检查已暴露工具并多次重试连接。只有多次确认 Browser 不可用后,才考虑独立 Playwright,并说明具体阻塞。
Agent 完成有边界的任务后停止。只有同一任务需要补充验证时才续用;不能让 worker 在空闲时自动寻找新功能或自行扩大范围。创建用户侧新任务不作为内部拆工方式。
## 5. 节省 Token 的具体做法
1. 派发窄任务,不默认继承整个历史对话。使用简短任务卡加源文件路径;必要历史按最小范围提供。短上下文不能省略政治规则、性能要求和相关视觉证据。
2. 主控保存一份简洁任务账本:当前合同、文件 owner、已完成证据、失败原因、下一步。各 Agent 引用同一事实,不各写一份长计划。
3. 先 `rg` 定位,再读函数和局部 diff;不重复读取巨大 JSON、整仓日志或全部历史文档。大数据留在文件里,由脚本给出数量、哈希、差异和异常样例。
4. 图片、截图和录屏用实际文件查看。不能为了省 token 把视觉任务变成只读文件名或文字概括。
5. 相同来源只研究一次,保存来源、日期和关键结论;只有数据变化、结论不足或任务扩大时重查。
6. 子任务首次任务卡建议控制在约 500–1500 token,结果摘要约 300–800 token;这是上下文组织目标,不是截断必要证据的硬上限。
7. 返回“改了什么、哪些文件、验证证据、剩余阻塞”即可。完整日志和截图放到文件,用路径引用;失败时返回相关片段,不整段倒入主控上下文。
8. 哈希、解压、下载、构建、测试和统计交给确定性工具。不要为了运行一条已知命令另开一个 Agent。
9. 不重复安排多个模型完整实现同一功能。关键风险做一次独立审查,普通修改由测试与 owner 检查。
10. 默认不开 `max` / `ultra` 推理。提高模型或推理强度只针对具体难题,并记录升级理由。
## 6. 升级与停止条件
| 情况 | 处理 |
| --- | --- |
| Luna 遇到不明确需求、需要理解跨文件状态或开始猜测行为 | 交 Terra;若涉及核心状态/几何,直接交 Sol |
| Terra 的任务超出明确合同,或同一实质失败在一次有针对性的修复后仍未解决 | 给 Sol 最小复现、尝试记录和失败证据,不继续盲试 |
| Sol 遇到跨坐标层、性能来源不明或与现有交互语义冲突的问题 | 给 Astra 聚焦诊断任务;可以由 Astra 完成必要关键补丁,不限于口头审查 |
| 已知一开始就是复杂的空间布局或高风险几何设计 | 直接用 Sol/Astra,不为了经过低模型梯级而增加一次无效尝试 |
| 网络、凭证、工具缺失、原始资料不存在 | 不通过升模型解决环境问题;检查环境或记录缺口,继续独立工作 |
| 失败来自缺失用户决定或无法识别的地名/视觉目标 | 提出具体问题,不让多个模型猜不同版本 |
默认允许 worker 一次有证据的修复;是否继续由主控根据新信息决定。超时或 token 消耗增加但没有新增证据时停止扩展,整理已知结果后重新分配,不能无限自递归。
## 7. 审查门槛
Astra 重点进入三个位置:最初高风险空间/几何合同、首次运动原型、最终关键风险验收;中间疑难问题按需增加。它不需要审阅每一处 CSS、每一段翻译或每个测试日志。
Sol 对各 worker 的交付负责。不能以“另一模型说通过”代替运行结果。任何模型都无权改变项目政治口径、降低性能门槛、把未知数据填成确定值,或自行部署和发版。
涉及政治几何和性能结论,使用与实现过程分离的检查证据;普通文档格式或固定映射使用针对性检查即可。视觉方向仍由已确认的用户要求和参考图决定,Astra 的审美判断不替代用户目标。
## 8. 单任务派发模板
```text
任务:一个可独立验收的目标
模型与推理:明确 model ID + reasoning effort
必要约束:相关 rules 模块、政治口径、性能、共享组件、视觉参考
输入:最少文件/符号、当前合同、已有验证结果和图片路径
可写范围:明确文件;共享文件由指定 owner 修改
验收:必须出现的行为、必要测试/渲染/性能证据
停止/升级:哪些条件不能继续猜测或扩展
返回:结果摘要、改动文件、检查结果、证据路径、遗留问题
```
派发有意使用较小上下文时,必须把不可省略的约束写进任务卡。执行任务的 Agent 自行按规则加载相应模块,不因主控已读过就假定子 Agent 也知道。
## 9. 如何验证分配确实更高效
每个工作包记录模型、推理强度、输入/输出 token(环境可提供时)、耗时、工具耗时、修复轮数、一次验收是否通过和最终证据。若读不到准确 token,就标记未知,不用消息长度伪造 token 统计。
比较每个已验收任务的总消耗,包含主控、worker、审查和返工,而不是仅比较某次调用。先完成 3–5 个有代表性的工作包,再调整默认路由;这个数量是试点取样建议,不代表统计显著性或保证节省比例。
API 单价、订阅额度和任务级成本不是同一个指标。本计划不预先承诺固定节省百分比;模型可用性和实际费用以执行时配置为准。低成本模型导致反复修复时应提前升级;高能力模型能明显减少整体返工时可以直接采用。
## 10. 官方依据与执行边界
OpenAI Docs 将 Astra 定位为最具能力、适合困难端到端任务的模型,Sol 面向复杂专业工作,Terra 平衡能力与成本,Luna 面向成本敏感任务;它们并未规定 Planet 各文件应由哪个模型处理。本计划按任务风险给出项目级分工。
- [OpenAI 模型说明](https://developers.openai.com/api/docs/models)
- [Codex 模型选择](https://learn.chatgpt.com/docs/models)
- [子 Agent 配置与模型选择](https://learn.chatgpt.com/docs/agent-configuration/subagents)
只在用户已授权相应开发与委派、且运行环境支持时执行这套拆工。仅制定本计划不会切换当前会话模型、修改默认配置或自动启动 Agent。执行时若某个精确模型不可用,先说明并选择可用替代,不把替代模型冒充原分配。
## 11. 本轮实际执行账本
| 工作包 | 实际分配 | 结果与升级 |
| --- | --- | --- |
| 地理政策、源证据、几何构建与回退 | Sol `high`,`geography_policy` | 版本化规则与校验已实现;如实保留未核验几何,未用模型猜测填补 |
| 空间连接线、卡片生命周期与主入口 | Sol `high`,`spatial_callouts` | 完成原型;六卡避让多次渲染仍有冲突后,交主控接手约束求解,worker 停止同文件写入 |
| 同机性能基线、对照工具、公共滚动条 | Terra `medium`,`performance_baseline` | 保存桌面/模拟手机各 5 次基线;共享滚动容器修复和对照工具已完成 |
| 复杂避让、全球目录/API、视觉参数、集成验证 | 主会话 | 主控整合并审查;性能脚本的场景同步多次失败后也由主控接手修复。独占布局文件,浏览器/构建按顺序执行,避免测试相互干扰 |
| Luna 辅助 | 未启用 | 没有值得额外传递上下文的独立机械任务,不为凑齐模型而派工 |
本轮精确 token 与费用不可得,记为未知;不报告未经测量的节省比例。空间避让存在返工,不能把功能实现或子任务数等同于效率提升。性能问题应在首次稳定复现后采集 CPU/布局证据,再修改热点;本轮最终定位到共享 SVG 长度读取造成的同步布局,单纯调整避让参数不足以解释该成本。性能、运动观感及地理缺项以功能计划的验收状态为准。

View File

@@ -0,0 +1,237 @@
# Earth 地理资源与空间卡片展示实施计划
日期:2026-09-21。状态:本地可运行原型。贴图已替换并归档旧图;固定规则、部分核验几何、全球参与方目录、空间卡片、共享移动滚动条与轻量材质调整已实现。完整政治几何、逐段海岸对齐、视觉认可及性能门槛尚未全部验收,不代表可发布。
本计划以已确认的展示方向为准,替代算力专题旧方案中的固定右侧栏、横贯底部双图和隐藏其他面板的布局。空间卡片与算力目录已有本地可运行原型,但仍需完成最终 QA;算力数据采集、指标证据和长期模型设计继续参见[算力与资源态势计划](compute-resource-intelligence-plan.md)。
## 1. 目标与工作边界
地球是空间信息的主体。打开区域对比后,多个区域分别引出连接线,牵引周围的浮动信息卡。地球转动时,地理锚点跟随球体,卡片保持正向可读,通过平滑、有阻尼的运动调整位置并互相避让,形成信息从地球四周延伸出来的感觉。
地球质感参考 UE 3D 截图在易拉宝与 PPT 中的表现:球面有层次、边缘与明暗交界柔和,背景退后,数据与球体构成同一空间。保持当前操作流畅度是硬验收条件。
已授权实施范围包括资源、空间卡片、共享 UI 与性能验证。继续使用现有 Three.js 引擎和资源加载机制。资源版本与实际转换参数以[地球资源说明](../../resources/earth/README.md)和清单为准。
### 当前实现与未完成项
本轮已实现卡片运动、配置控制、内容承载及中性地球材质调整。配置窗口与卡片显示开关已分离,关闭配置不再停用卡片。卡片默认常显基础信息:36px 主数值、年份/指标/口径说明、设施样本与 Epoch 来源、趋势首末年份及单位、证据入口和区域成员数;长列表仅在需要时展开完整图表。桌面卡片约宽 268px,高度随内容自适应,国家约 214px、区域约 223px;展开卡片支持用户调整至最高 480px,位置/尺寸可保存并恢复自动布局。手机保留同等摘要信息。高清 overlay 使用白色 MeshBasicMaterial、opacity 1、toneMapped:false;r128/r160 按 active API 使用对应 sRGB 读写路径。没有新增 GPU draw pass 或高分辨率纹理,但新增并预加载 45 个本地 SVG 旗帜资源。
旧版底边换侧的连续跳动已定位并由路径规划与稳定换侧策略处理。新实现由 `spatial-callout-controller.js` 管理,布局采用事件驱动并保留有效旧矩形;投影连线持续跟随地球,另一侧候选需持续 1800ms 才换侧,12 秒阅读间隔用于抑制不必要的替换。悬停、焦点和详情阅读会锁定卡片,用户拖动复用 `hud-drag.js`,现有 controls 也消费该共享模块。多卡布局通过实际空闲矩形安排最终位置,优先避开已有功能窗;局部 UI QA 已完成,全球性能与最终发布回归仍按后续门槛验证。下方性能数字和旧尺寸的回归记录属于此前原型,不能直接作为当前大卡片的性能验收结论。
- 底图默认“原始底图”(菜单首项),显示菜单可切换并保存;Natural Earth I 保留,NASA 21K 仅作源归档。运行时统一 8K,按需只保留一张 GPU 纹理;成功后释放旧图,失败保留旧图,过期响应不能覆盖当前图源,不新增渲染层。星空逐字节恢复 commit `83a10a6c`,不再校准密度和颜色。高清 overlay 使用白色 MeshBasicMaterial;r128/r160 按 active API 选择对应 sRGB 读写。没有新增长驻高分辨率纹理、后处理或 GPU draw pass;算力卡另新增并预加载 45 个本地 SVG 旗帜资源。Natural Earth 原图偏浅,不把不存在的细节写成恢复结果。
- 地理政策由版本化清单贯穿规范编码、后台聚合、高精几何及低精回退。中国、台湾、港澳、科索沃等已有回归;已核验几何及济州岛补全见资源说明。高精技术就绪与政策可发布分开:`high_precision_ready=true`,`policy_publishable=false`。东沙、中沙、曾母暗沙点注记及经核验海上主张线仍有缺项。
- 规范目录覆盖 45 个国家编码,界面分组为中国、美国、欧洲、不含中国的亚太和其他;分组不当作统计合计。当前空间原型最多同时显示 6 张,后端最多接受 12 个参与方。没有历史观测的地区返回 `null` 历史与覆盖信息;设施样本数永远不作为计算出的国家总量。
- 桌面卡片复用共享 HUD 和连接线,保留稳定 DOM 身份;选择器和来源窗使用共享拖动。背面卡片保留内容并隐去连接线。旋转通过原动画循环更新,投影连线持续更新;布局按事件分帧处理并保留有效旧矩形。`main.js` 的 `isDragging`、`activeTouchPoints`、`hasActiveGlobeInertia` 接入运动模式:手动/惯性快速模式以 72ms/12px 为响应预算,在当前帧完成受限布局,停止后一次 settle;slow auto 保留 1.8 秒换侧和 12 秒阅读保持,用户拖动放置优先。
- 手机复用公共抽屉与深色滚动所有者,使用适合手机阅读的字号;目前是抽屉内紧凑卡片列表,尚未实现手机环绕空间卡片。桌面原型没有强制隐藏新闻、媒体或统计窗口。
- 2/4/6 卡片、背面投影、共享窗口避让、饱和纵列与共享连线动画已有 11 项单元测试。浏览器已检查 1366×768、1920×1080、390×844 及 125%/150% 对应的有效 CSS 视口;单元测试不替代运动观感验收。
### 本轮空间卡片 UI 定向验收(2026-09-21)
本计划保留此前的历史验证记录;它们不代表当前地图源切换或全球 Earth 验收已经完成。当前实现说明只记录所需资源生命周期与 UI 约束,Browser/全球性能结论仍按后续门槛单独复核。
以上是本轮卡片 UI 与局部资产的定向证据。全球地理政策、逐段海岸对齐、完整 Earth 渲染性能、真实设备性能及发布级回归仍按本计划后续门槛单独验收。
## 2. 优先级与不可退让的要求
| 优先级 | 要求 | 执行原则 |
| --- | --- | --- |
| P0 | 固定政治展示规则 | 外部数据、更新、回退及 AI 均不能覆盖项目规则 |
| P0 | 性能不退化 | 在同设备、同数据、同场景下验证;可复现的退化不能用视觉提升抵消 |
| P1 | 地理对应一致 | 贴图定位、海岸线、区域归属、拾取和卡片锚点一致 |
| P1 | 地球关联的空间卡片 | 使用现有连接线与展示机制扩展,不恢复固定大屏布局 |
| P1 | 共享 UI 与移动端一致 | 共用字体、标题栏、按钮、滚动容器及窗口行为 |
| P2 | 卡片内图表和内容完善 | 先验收空间关系与运动,再充实图表和解释 |
## 3. 固定政治展示规则
项目展示采用固定中国视角:中国相关领土及争议区域按项目既定中国视角处理,台湾按中国处理;科索沃按塞尔维亚处理。该口径先于数据源选择,不由数据源默认分类、模型输出或换源结果决定。
实施时建立单一、版本化的规则清单与几何修正包。名称映射只是其中一部分,必须覆盖国家编码、Polygon/MultiPolygon 归属、边界绘制、区域填色、鼠标拾取、高亮、统计及卡片归属。处理后的结果才可以成为运行资源;未经规则校验的新来源不得替换正式资源,旧资源回退也必须通过同一规则。
规则清单应包含:稳定规则 ID、目标区域的明确标识、目标归属、显示名称、几何来源、合并/裁切操作、版本及回归样例。不能只依赖模糊地名字符串替换。语音中未能确认的具体岛礁名称必须在修正清单中明确定位,不能猜测名字后直接移动多边形。
海岸线、陆地行政区域和海上主张线是不同几何层。项目口径在各层一致执行;海上主张线不能作为陆地面直接填充,也不能代替具体岛礁的几何和归属规则。
必须具备的失败测试:外部源恢复原始分类、别名变化、同区域不同编码、几何修改后拾取仍用旧数据、低精度回退覆盖规则。任何一项失败,都不发布该资源版本。
## 4. 已查明的现状
### 4.1 贴图与国界
- 原运行贴图为 `frontend/public/earth/assets/8k_earth_daymap.jpg`,8192×4096;代码直接加载 JPG,未读取该文件配套的地理定位元数据。
- `country-boundaries.js` 使用独立 GeoJSON/PMTiles 几何绘线与拾取;高精度是一个选择状态,存在退回低精度的路径,按钮名称不等于实际正在使用的文件。
- 本机此前检查到 `natural-earth-v1`、`geojson-high-precision` 构建清单,以及独立 base、hover、claim 文件;该构建不能直接等同于完整应用了项目政治几何规则。实施前需要重新读取实际运行版本。
- 已提供的两张问题截图中,一张出现海岸折线与影像陆地错位;另一张有可见岛屿但没有对应线框。截图只能证明现象,尚不能断言根因一定是 UV、投影、低精度回退或缺岛。
- 精度检查必须同时检查“看得到轮廓”和“能选中对应区域”,二者不能各用一套不一致的几何。
### 4.2 World Monitor 对照
本机 World Monitor `463aec0f` 使用独立地表图片与 `countries.geojson`;经纬度归属由点落入多边形的计算决定,二维底图和自带的交互区域也分开加载。
本次文件统计:World Monitor 的中国、日本、韩国分别为 200、60、14 个坐标点;Planet 的低精度分别为 240、65、19;本机高精度文件分别为 14,118、6,952、3,060。点数不等于测绘准确度,但表明不能靠直接换用 World Monitor 的简化国家文件解决近景海岸问题。它的原始归属处理也不能代替本项目规则。
### 4.3 已有共享机制
| 入口 | 当前作用 | 计划复用方式 |
| --- | --- | --- |
| `frontend/public/earth/js/callout-connector.js` | 连接线、元素锚点与路径 | 扩展为多个稳定区域锚点与卡片配对 |
| `frontend/public/earth/js/presentation-controller.js` | 展示生命周期与连接器更新 | 在单项展示的基础上设计受控的多项展示,不破坏现有模式 |
| `frontend/public/earth/js/cruise-sequencer.js` | 巡航时序 | 保留当前巡航行为,区域对比明确自己的生命周期 |
| `frontend/public/earth/js/info-card.js` | 公用详情与移动详情 | 抽出/复用内容渲染能力,不复制成另一套卡片组件 |
| `frontend/public/earth/js/hud-panels.js`、`controls.js` | 面板收起、显示、拖动与约束 | 工具窗继续遵循这些规则;明确 createHUDPanel 本身不包办所有拖动/缩放 |
| `frontend/public/earth/css/hud.css`、`info-panel.css` | 公用外观、字号、滚动与移动抽屉 | 去掉专题私设的大字号、独立控件和白色滚动条 |
## 5. 地理资源:先采用配套贴图的简单方案
### 5.1 选择与准备
默认运行底图为“原始底图”(菜单首项),显示菜单可切换并保存底图源;Natural Earth I 保留可用。NASA 21K 仅保留为源归档,运行包统一使用 8192×4096 的单张 GPU 贴图。官方源版本与许可信息仍以资源清单为准,不能把归档源误写成运行资源。
采用不带政治标注的物理地表图。按需只保留一张运行 GPU 贴图;新源成功加载后释放旧图,失败保留旧图,过期响应不得覆盖当前图源。源包保留 TIFF、世界文件和说明,运行资源统一为 8192×4096 的 Web 图。转换只做尺寸与格式适配,不用生成式图像重绘海岸、岛屿或国界,不随意裁切、旋转或拉伸经纬范围;底图切换不新增渲染层。
下载的原始大文件保存在本地 `downloads/earth/`,不纳入前端发布包。可追溯清单、配准信息与旧贴图归档放在 `resources/earth/`;运行图放在现有 `frontend/public/earth/assets/`。旧图必须先按字节完整归档并核对 SHA256,之后才能替换运行路径。
### 5.2 对齐验证
1. 读取 TIFF/TFW 的像元大小、原点、行列方向、地理范围和 CRS;同时记录 JPEG 派生后的像元中心定位,避免半像素偏移。
2. 明确运行中加载的贴图、边界文件、资源版本与是否发生回退。
3. 使用同一投影和球面经纬转换叠加成套矢量,检查赤道、零经线、日期变更线和高纬区域。
4. 在截图对应海岸、近海岛屿、台湾周边和其他不同经度的典型海岸逐点验证,区分整体偏移、局部概化和缺失几何。
5. 政治修正后的同一份区域几何用于标注、高亮和拾取。所有需要烘焙的线也从这份数据导出,不能在图片和矢量上分别手工描线。
6. 小岛、岛礁或海岸不满足要求时,先补充对应区域数据并重新配准;仅当这条简化路线确实不够用时,再考虑局部细节或分级资源。
成套资源声明配准不等于本项目已经逐像素验收。不能承诺卫星影像每一处潮线都与概化海岸线完全一致;验收需保存具体偏差和缺失清单,不能靠强行拉伸边界迎合图片。
### 5.3 常态渲染与交互
优先预合成常态显示所需的边界,以减少额外线段绘制。需要独立国界开关时,评估复用现有覆盖层或同定位的透明边界层;不默认再常驻一张 8K 纹理。矢量数据继续承担区域判定和选中高亮,不在每帧重新解释整份 GeoJSON。
本阶段不以替换 Three.js、引入新 GIS 平台或建设全球在线瓦片服务作为前置条件。
## 6. 整体视觉方向
参照资产:`Planet-Handoff-20260916` 中的完整易拉宝效果稿和卡片展示 PPT,尤其 PPT 内来自 UE 3D 的原始星球画面。海报中的线点含概念叠加,不能拿来充当精确地理数据。
- 地球保持正圆与正确地理位置,视觉上成为主体;不把纹理精细度等同于画面品质。
- 球面采用柔和、连续的明暗层次,边缘有薄而自然的大气过渡;地理细节仍然可读。
- 降低背景星点对比与密度造成的干扰,让背景退后;变化以参考图与实际渲染对照决定。
- 数据节点与连接线有轻重层级,避免强霓虹、油亮塑料感、浓重太空雾和粗硬轮廓。
- 优先在现有材质、光照与透明度范围内调整;新增后处理、辉光或高分辨率透明层须先证明成本。
- 用同一位置、距离、光照条件对比前后效果,近景与远景都检查边缘、深度冲突和细节。
## 7. 区域卡片:从地球四周延伸出来
### 7.1 行为模型
一个区域对应稳定的地理锚点、连接线和卡片 ID。锚点随球体旋转;卡片处在屏幕空间,保持正向、稳定大小和内容身份。转动时变化的是屏幕位置与线的路径,不重建卡片、不交换身份、不清空正在阅读的内容。
卡片根据区域投影分布到球体周围,保留疏密和空间关系,不固定平均分成八个方向,不横排占满屏幕。使用有限的避让和有阻尼的缓动,最终能够安定;不持续抖动或为了追踪锚点逐帧跳位。
### 7.2 布局与生命周期
- 沿用现有连线的起点/终点接口和动画语言,先验证 2 张、4 张、6 张卡片;6 张是原型测试档位,不是已确定的产品上限。
- 保持地域锚点与区域绑定;算法可以移动卡片,但不能移动真实地理坐标来换取好看的布局。
- 优先避让其他卡片和已有工具窗,不自动隐藏新闻、直播、统计、图层面板。
- 对卡片换侧设置稳定区间,避免锚点越过某条中心线就立即左右互换。
- 背面区域不画成穿过地球的前景锚点。建议保留卡片身份,用边缘提示表达背面方位;完整隐藏、收束或边缘提示的最终表现用短动画原型比较后决定。
- 用户正在操作的卡片短暂锁定其屏幕位置;是否允许长期固定/拖动偏移,先以现有交互能力做原型验证,不增加未经确认的复杂模式。
- 增删区域、切换年份、退出对比、隐藏相关图层和请求失败都要有明确的清理与恢复逻辑;旧响应不能复活已关闭卡片。
- 区域名称锚点用于对比展示;不能把区域中心点画成设施位置。
### 7.3 卡片内容与参与方
先验证空间展示,再完善图表。首轮卡片可以只包含名称、一个有来源的关键指标、参考期及来源入口;固定侧栏和横贯底部双图的旧设计不再继续扩展,普通浮窗加分页的方案也不是最终展示方向。
参与方必须支持全球扩展,不能在接口和 UI 中永久硬编码成中美两项。日本、韩国等可以作为独立对象;亚太、欧洲可用于选择菜单的分组或区域筛选。具体默认组合与区域汇总口径还需在完整 UI 原型中确定,不把早期讨论的菜单建议当成已确认设计。组合区域与成员国不能重复计入同一分母。
缺少同口径历史数据的参与方仍可展示公开设施覆盖及明确的缺失状态,不伪造总量、得分或排名。全国总算力、公开 AI 集群样本、TOP500 FP64 和创新指数继续分开。2026 年未核验值不得加入正式历史。A4 AI 解读不阻塞空间展示验收。
## 8. 共享窗口与移动端
设置、来源、图层等操作窗口继续复用已有标题栏、按钮、拖动和按能力启用的缩放逻辑。不得用单独字号和固定网格重新构建一套视觉体系。数据更新和语言切换应保留窗口位置、尺寸、选中对象和滚动进度。
移动端复用原底部抽屉与详情展示。区域卡片数量受可用空间控制,保证地球与主要交互可见;可收束为可切换的关联卡片,但具体排列需要实机原型确认。不能把桌面全部卡片原尺寸堆到手机上。
白色滚动条问题从 `.earth-mobile-drawer-content` 及实际滚动所有者核对,统一复用 Earth 的深色滚动条样式与滚动容器。覆盖图层、搜索、态势、算力、新闻和设置页;底部导航保持固定,避免嵌套重复滚动,不仅对算力页补一条局部 CSS。
## 9. 性能基线与硬验收
### 9.1 基线
资源替换前的画面可通过归档旧贴图复现。保存设备、浏览器、GPU、视口、DPR、图层开关、数据量、相机路径与资源版本。在相同条件下重复测试,至少记录 5 次稳定运行,并区分冷启动、热启动和稳定旋转。
观察:首个可操作画面时间、首个有效地球画面时间、帧时间 p50/p95/p99、长帧/长任务、交互延迟、JS 堆与纹理内存估算、draw calls、纹理/对象数、网络字节及请求次数。只比较平均 FPS 不足以证明没有卡顿。
### 9.2 实现预算
- 运行贴图先维持 8192×4096。相同格式下纹理像素量不增加;RGBA8 粗估基础约 128 MiB,完整 mipmap 约 171 MiB,不能把 JPEG 文件变小当成显存同比下降。
- 保持已有 mesh、图标实例与纹理缓存;旋转不重新解码图片或构造国家几何。
- 地理投影和轻量位置更新与渲染帧同步;碰撞布局只在锚点明显变化、卡片变化或窗口尺寸变化时重算,不让每张卡片各起一套动画循环。
- 卡片 DOM 保持稳定,图表仅在数据/尺寸变化时更新;避免每帧测量布局后又写布局造成同步回流。
- 网络、缓存、监听器、纹理及卡片数量均有边界,关闭/重开多轮不得持续增长。
- 默认不增加新的高分辨率常驻覆盖层、昂贵后处理或全部区域的最高精度常驻数据。
### 9.3 判定
同条件下不能出现可复现的流畅度、加载、交互或资源占用退化。测量噪声必须记录并重复验证,不能事先允许固定比例退化来降低要求。超出预算先调整效果、卡片密度或实现,再复测。桌面与移动分别验收;没有真实手机数据时只能说明模拟检查通过,不能宣布手机性能达标。
### 9.4 内容卡扩容前的原型验证记录(2026-09-21)
同机 Headless Edge、桌面 1440×900 / 手机尺寸模拟 390×844、DPR 1、自动旋转,每种尺寸 5 次;预热 1 秒、帧时间采样 2 秒。该短采样用于定位,不能代替真实设备长时交互验收。原始基线标识 `2026-09-20T18-30-56-184Z`,最新对照标识 `2026-09-20T21-07-40-495Z`(UTC);完整 JSON 留在本机临时验证目录 `planet-earth-performance-baseline`。
| 项目 | 原基线 | 本轮结果 | 判断 |
| --- | --- | --- | --- |
| 桌面无卡 p95 帧时间,5 次中位数 | 7.09 ms | 7.10 ms | 接近;不能据此宣称完整性能通过 |
| 手机尺寸模拟无卡 p95 | 7.00 ms | 7.20 ms | 未做真实手机测量 |
| 桌面 2 / 4 / 6 卡 p95,5 次中位数 | 无同场景旧实现对照 | 7.10 / 7.10 / 14.00 ms | 六卡比本轮无卡场景有明确成本,仍需优化 |
| 六卡采样窗口长任务 | — | 5 次均为 0 | 没有 50 ms 长任务不等于没有掉帧 |
| 桌面 DOMContentLoaded 中位数 | 1.262 s | 1.627 s | 已移除 Google Fonts 的约 10 秒阻塞;仍不宣称加载性能与基线完全持平 |
| 桌面基础 JS 堆中位数 | 约 50.2 MiB | 约 53.2 MiB | 不是显存;未替代纹理/绘制统计 |
每轮另做 5 次专题开关并强制 GC 检查,关闭后的保留堆约 25 MB,首末差约 0.28–0.33 MB;这不构成长期无泄漏证明。浏览器单独核对关闭后的卡片悬停/焦点监听器为 0,来源窗与拖出容器的选择器均能关闭。
最终 Bun 构建、前端规则检查、文档一致性检查与 `git diff --check` 通过。既有前端 smoke(使用其 API mocks)通过 321 项路由/交互检查,包括公共页面、鉴权、管理导航、移动布局和管理页缩放;Earth 空间卡片另用真实本地接口检查,不能混同两种证据。
布局求解已分帧执行、减少重复候选评分、优先前景区域,在合格布局找到后停止重复试排;工具栏按可见按钮而非透明容器避让。CPU 采样发现共享 SVG 的 `getTotalLength()` 同步布局热点,已改为按已知折线点计算长度,保留原绘线动画。11 项布局/投影/连线测试通过。1366×768 浏览器中 4/6 卡、旋转后、稳定后及工具栏展开时未发现与卡片、选择器或现有 HUD 的矩形交叠;1920×1080、1093×614、911×512 同样完成六卡边界检查,后两项是浏览器缩放的等效 CSS 视口,不冒充真实设备缩放测试。46 项相关后端测试通过,高精与回退的台湾、藏南、阿克赛钦、科索沃、济州岛浏览器拾取均符合规则。官方几何缺项、完整海岸对齐、六卡性能与真实手机仍未通过最终验收。
## 10. 分阶段实施与交付门槛
开发模型分工、文件所有权、并行限制和上下文控制按[开发 Agent 模型分配计划](development-agent-model-allocation-plan.md)执行:Sol 主控与核心实现,Terra 承担明确子任务,Luna 处理有边界的辅助工作,Astra 负责高风险设计及疑难审查。模型分工不改变本计划的政治规则、视觉方向和性能验收门槛。
| 阶段 | 工作 | 必须交付的证据 |
| --- | --- | --- |
| S0 资源准备 | 新贴图、配准信息、旧图归档、来源清单 | 新旧尺寸/哈希、解码与球面渲染检查;本轮范围 |
| S1 规则与配准 | 固定政治清单、实际提供者诊断、配套边界对齐 | 规则测试、截图问题点对照、回退测试;不能只报高精度开关已打开 |
| S2 空间运动原型 | 复用连接器,多区域锚点、卡片稳定身份与避让 | 转动/缩放/换侧/背面/退出短录屏,2/4/6 卡档位及性能差值 |
| S3 视觉与组件 | 柔光质感、背景层次、共享字体与移动滚动 | UE/PPT/海报参照对照、多个视角、桌面/手机及缩放截图 |
| S4 内容联动 | 多参与方、真实指标与来源、时间/设施联动 | 可操作路径、缺失/错误状态、来源与口径验证 |
| S5 回归与发布准备 | 性能复测、旧图层/巡航/工具窗回归、同步文档 | 完整验收记录;需要发布时另走仓库发布流程 |
S1 未通过,不能把新图片称为边界问题已解决。S2 的空间交互未确认,不能直接投入完整图表和全部国家卡片。性能或政治规则任何一项未通过,后续阶段不能据此宣布完成。
## 11. 验收清单
- 政治:标签、几何、填色、拾取、统计、卡片与回退源口径一致;科索沃等明确规则有专门用例。
- 地理:截图所指海岸和小岛、台湾周边、日期变更线、高纬度;记录偏差而非只肉眼说“差不多”。
- 视觉:球体不变形,边缘柔和但地理清楚,卡片从区域延伸且不压满地球;远距无闪烁和黑块。
- 交互:转动、缩放、选中、背面、卡片增删、触摸、键盘、语言切换、退出、窗口大小变化及旧响应竞态。
- 共存:新闻、媒体、统计、图层及原详情卡保持可用;不通过强制隐藏制造布局空间。
- 移动:公共抽屉无白色原生滚动条,底部导航固定,只有预期内容区滚动,文字和操作不被截断。
- 响应式:1366×768、1920×1080、约 390×844,以及 125%/150% 缩放;模拟与实机结果分别记录。
- 回归:公共页面、鉴权、管理路由、导航/搜索/标签页、既有 BGP/新闻/动捕巡航与旧图层。
- 性能:与归档基线同条件比较,保存帧时间、内存、网络和交互证据;循环打开关闭验证资源释放。
- 文档:更新实际变更的中英文手册、Earth context、图层规则和资源清单;计划与已实现行为不能混写。
## 12. 来源、恢复与完成定义
- [Natural Earth 栅格与矢量配准说明](https://www.naturalearthdata.com/downloads/10m-raster-data/)
- [Natural Earth I 资源页面](https://www.naturalearthdata.com/downloads/10m-raster-data/10m-natural-earth-1/)
- [Natural Earth 使用条款](https://www.naturalearthdata.com/about/terms-of-use/)
- [现有 Presentation 解耦计划](earth-presentation-decoupled-architecture-plan.md)
- [原高精度边界计划及历史说明](earth-high-precision-boundary-tiles-plan.md)
- [地球前端上下文](../technical/zh/earth-frontend-context.md)
- [渲染层次规范](../technical/zh/earth-render-layer-order.md)
旧贴图恢复路径和 SHA256 见资源清单。先核对归档哈希,再将归档副本复制回原运行资源路径并重建预览;不以回退图片为由撤销其他用户改动。新资源和旧归档均不得无记录覆盖。
最终完成意味着:展示规则固定且经验证,地图与区域选择对应,多个卡片确实由地球锚点牵引并稳定运动,操作组件和手机滚动保持一致,实际性能不低于基线。单张效果图、构建通过或“高精度”名称都不能单独证明完成。

View File

@@ -2,6 +2,8 @@
## 当前状态
最新实现继续保留分桶 `Points` 与全局数据范围,并已接通 `vessels` 全局实时订阅:后端读取确认状态后按 MMSI 拆包推送,前端原位修改缓冲;快照用于首次加载、重连和删除校准。旧快照不能覆盖较新的更新或删除,旋转与缩放仍不触发视口请求。下方 Sprite 问题分析和阶段方案保留为历史背景,当前约束以[地球前端上下文](../technical/zh/earth-frontend-context.md)为准。
该计划的前端核心部分已经在 `0.44.1` 落地,但最终实现不是原文设想的 `InstancedBufferGeometry` quad,而是更稳的分桶 `THREE.Points` 方案:
- 普通船只按 moving / anchored 和 `VESSEL_COURSE_BINS` 航向分桶,使用 `PointsMaterial` 批量绘制。

View File

@@ -51,6 +51,7 @@ That split makes MiniMax, Claude-compatible gateways, and self-hosted OpenAI-com
Supported request adapters:
- `openai-completions`
- `openai-responses`
- `anthropic-messages`
- `ollama-generate`
@@ -95,12 +96,37 @@ The AI settings page uses:
- `POST /api/v1/settings/integrations/ai-provider/connect`
- `GET /api/v1/settings/integrations/ai-provider/secrets`
- `GET /api/v1/settings/integrations/ai-provider/presets`
- `POST /api/v1/settings/integrations/ai-provider/presets/{provider}/refresh`
- `GET /api/v1/settings/ai-prompts`
- `PUT /api/v1/settings/ai-prompts/{task_key}`
- `POST /api/v1/settings/ai-prompts/{task_key}/reset`
These endpoints require an authenticated user. The `secrets` endpoint is only used when the settings page reveals a key or token; hiding the field restores the masked preview.
Model refresh and lightweight checks share `backend/app/services/llm_model_catalog.py` and query provider endpoints directly, without models.dev. Refresh uses the draft base URL, protocol and credentials, or saved configuration when no draft is supplied. Regional hosts and custom gateway paths are preserved. Missing required credentials return 400; upstream failures return safe errors and retain the previous catalog.
Successful catalogs and `refreshed_at` are stored under `llm_provider_preset:<provider>` in `system_settings`. Dynamic categories use the provider preset as their defaults and are validated before committing. Listing prefers saved catalogs and otherwise shows bundled suggestions. Refresh never modifies the active model, protocol, URL or credentials in `external_integrations`. The frontend retains the draft and previously loaded models when listing fails, with an explicit error message.
### Model Catalog Endpoints
These endpoints were checked against official documentation on 2026-09-13. The table lists discovery URLs; the form still takes the generation base URL. Credentials must belong to the configured service region.
| Provider | Model list GET endpoint | Authentication and parsing |
| --- | --- | --- |
| [MiniMax](https://platform.minimax.io/docs/api-reference/models/anthropic/list-models) | `https://api.minimaxi.com/anthropic/v1/models`; international: `api.minimax.io` | `x-api-key`; `data[].id`; `has_more/last_id` pagination |
| [OpenAI](https://developers.openai.com/api/reference/resources/models/methods/list) | `https://api.openai.com/v1/models` | Bearer; `data[].id` |
| [Anthropic](https://platform.claude.com/docs/en/api/models/list) | `https://api.anthropic.com/v1/models` | `x-api-key`, `anthropic-version`; paginate with `after_id` |
| [DeepSeek](https://api-docs.deepseek.com/api/list-models) | `https://api.deepseek.com/v1/models` | Bearer; `data[].id` |
| [Alibaba Model Studio](https://help.aliyun.com/zh/model-studio/list-models) | `/api/v1/models` on the configured regional host | Bearer; `output.models[].model`; `page_no/page_size/output.total`; filter `capabilities=TG` |
| [Moonshot / Kimi](https://platform.kimi.ai/docs/api/list-models) | `https://api.moonshot.ai/v1/models`; China: `api.moonshot.cn` | Bearer; `data[].id` |
| [OpenRouter](https://openrouter.ai/docs/api/api-reference/models/list-all-models-and-their-properties) | `https://openrouter.ai/api/v1/models` | Public catalog; Bearer when supplied; `data[].id` |
| [OpenCode Go](https://opencode.ai/docs/go/#models) | `https://opencode.ai/zen/go/v1/models` | Public catalog; Bearer when supplied; `data[].id` |
| [Ollama](https://docs.ollama.com/api/tags) | `/api/tags` on the configured server | Local servers need no key; `models[].model/name`; empty means no installed models |
New Model Studio endpoints for Beijing, Tokyo, Frankfurt and Virginia require the actual workspace host, such as `<WorkspaceId>.cn-beijing.maas.aliyuncs.com`. Singapore uses `dashscope-intl.aliyuncs.com`; Hong Kong uses `cn-hongkong.dashscope.aliyuncs.com`. Discovery changes only the path, never guesses a workspace or switches credential regions. If a legacy Beijing host no longer accepts the account, update the base URL from the provider console.
Pagination and retries have a 30-second overall deadline. Network failures and 502/503/504 allow up to two attempts; authentication failures are not retried. Models are sorted by supplied creation/release dates, preserving upstream order on ties, without arbitrary list truncation. Public catalog membership does not establish account-specific generation permission.
Admin keeps the AI page aligned with the legacy information architecture:
- `Model Providers`
@@ -283,22 +309,11 @@ Tool keys follow the same rule. WebSearch and OCR must match the current tool an
### Lightweight Connectivity Testing
The Admin plug button performs a lightweight connectivity check and does not save configuration. Common API-platform practice is two-tiered:
- Check a provider catalog or low-cost endpoint to validate base URL, authentication, and model reachability.
- Send full model requests only when the user explicitly runs Playground or a business task.
Connectivity results should be explicit:
- `ok`: authentication, route, and model catalog are usable.
- `warning`: service is reachable, but the current model is missing from the catalog or capability metadata is incomplete.
- `error`: authentication, network, protocol, or model lookup failed.
Toast titles must match the result; failures must not be titled as a successful connection.
The Admin plug button checks proxy configuration and queries the current provider catalog. It returns `success=true` only when discovery succeeds and contains the selected model. A 404, authentication error, invalid response or missing model never becomes success based on bundled suggestions. The message distinguishes discovery from generation; Playground or a business task verifies actual generation. Testing never saves the draft or switches the default provider.
### OpenCode Go Routing Model
Subscription channels such as OpenCode Go should not be handled by hard-coded frontend model sets. Prefer provider catalog or backend capability discovery that records per-model capabilities such as `chat_completions`, `anthropic_messages`, `models_endpoint`, and whether a subscription key is required. The frontend should display capabilities; the backend should map provider, base URL, model, and adapter into the real request.
OpenCode Go protocol mappings are owned by the backend catalog: MiniMax M3/M2.7/M2.5 and the verified Qwen3.6/3.7/3.8 models use Anthropic Messages; GPT-5.6 Luna, Grok 4.6 and Muse Spark Contributor use Responses; other supported models use Chat Completions. The official `/models` response currently supplies IDs only, so new models still require checking the documented endpoint table. Selecting a model updates its protocol in the draft, and backend mappings supersede known stale mappings. Responses uses `input`, `max_output_tokens` and `store=false`, and parses text and reasoning summaries. OpenCode calls carry an application User-Agent and a stable Playground conversation identifier. Generic `thinking=enabled` is translated to MiniMax M3’s `adaptive` format.
#### Settings Page Behavior

View File

@@ -99,7 +99,7 @@ AIS vessel collectors use a different persistence path from regular `CollectedDa
Earth boundaries are no longer data collectors. They are Earth static rendering assets: the Earth Assets settings panel owns source configuration, and `/api/v1/earth/boundaries/*` builds `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`. When no high-precision PMTiles artifact is available locally, the frontend uses the bundled low-precision GeoJSON fallback and does not write boundary records to `CollectedData`.
TOP500 and Epoch AI compute sources do not always provide usable coordinates. The unified Earth compute-center endpoint uses only valid source-provided coordinates or `compute_center_locations` dimension-table coordinates during the main map startup path; records without coordinates are returned as `unresolved` instead of being rendered from a local registry, country centroid, or guessed city. When users manually collect candidates, the backend queries ROR and Nominatim/OpenStreetMap from source fields; accepted candidates are saved into `compute_center_locations` and rendered from that table on the next layer refresh.
TOP500 and Epoch AI compute sources do not always provide usable coordinates. Epoch's official CSV `Location` may contain only a country or be empty even when a row has latitude/longitude; it is never treated as a precise site without the required source semantics. The unified Earth compute-center endpoint uses valid source-provided coordinates or `compute_center_locations` dimension-table coordinates during the main map startup path; records without renderable coordinates remain in the complete `unresolved` list instead of being rendered from a local registry, country centroid, or guessed headquarters. `precise`/`site` keep source coordinates; city-level and confirmation-required points may receive only a deterministic display offset keyed by `source` + `source_id`, with the original coordinate untouched. When users manually collect candidates, the backend queries ROR and Nominatim/OpenStreetMap from source fields; accepted candidates are saved into `compute_center_locations` and rendered from that table on the next layer refresh.
Admin collection management follows the business hierarchy instead of flattening every endpoint into one table:
@@ -111,6 +111,12 @@ Snapshot lists should not show every snapshot of the same collector as separate
Credential guides are maintained by `backend/app/services/credential_guides.py`. The console uses read / generate / reset actions to load or create Markdown instructions. The frontend should render the guide Markdown for operators, not expose generation prompts or raw metadata.
### News Live Stream Catalog
The IPTV-org adapter in `news_live_streams` retains its news-category filters. Its default `max_sources` is `0`, meaning all matching channels; an explicit positive value still limits collection. Existing configurations retaining the old `120` limit must change it to `0` and collect again to populate the complete catalog.
`GET /api/v1/tv/streams` uses database pagination in `tv_catalog.py`: `offset` defaults to `0`, `limit` defaults to `50` with a maximum of `100`, and whitespace-separated `q` terms match channel names, providers, regions, and languages. Built-in and configured sources come first. Collected sources are deduplicated by channel ID, exclude configured overrides, and are filtered, counted, sorted, and paginated in SQL. The response's `total` counts matches, while `source_count` counts the complete available catalog; use `has_more` and `next_offset` for subsequent pages. `selected_id` can also return the selected channel outside the current page without consuming its quota. `tv_streams.py` remains the owner of default and fallback sources.
## IV. Data Format (stored in CollectedData table)
```python
@@ -352,7 +358,11 @@ GET /api/v1/visualization/vessels/{mmsi}/conflicts
`/api/v1/vessels/snapshot` requires `bbox` and `zoom`, and caps `limit` at `5000`. The Earth frontend uses a global bbox for current state and does not refetch on camera viewport changes. The endpoint reads `vessel_current_state` and reports `diagnostics.source = "vessel_current_state"`. The old `/api/v1/visualization/geo/vessels` route has been removed.
High-frequency AIS updates must not become per-delta full-layer rebuilds. If Earth uses the `/ws` `vessels` channel, it should send low-frequency reload/dirty hints and let the frontend merge snapshot refreshes. Tracks and conflicts still read historical facts through the single-vessel APIs.
AISStream and BarentsWatch share the `/ws` `vessels` delta channel. Earth subscribes with `scope: "global"` on its existing WebSocket, without changing subscriptions as the camera moves. The backend coalesces notifications by MMSI for one second, then reads confirmed `vessel_current_state` rows. Frames contain at most 1000 items; further frames carry the remaining vessels rather than truncating them. Raw source messages must not overwrite confirmed client positions. Deleting current-state rows also produces MMSI-based remove notifications.
The frontend coalesces short bursts per MMSI and uses `Interactable.updateItems()` to update position and color buffers in place. Heading-bucket changes touch only affected buckets, growing capacity when needed. Existing marker identities, selections, and materials survive. Initial entry, reconnects, deletion hints, and the minute reconciliation still use snapshots without first clearing the layer. A bounded snapshot must not treat truncated vessels as deleted, and older responses must not overwrite newer stream updates or removals received while the request was in flight. Snapshots include query-start `generated_at`; it is compared with stream frame time to prevent stale cached snapshots from rolling back new vessels or removals.
Ordinary vessel writes use the `delta` strategy on `earth_updates`; while the dedicated channel is connected, they no longer request whole-layer reloads. Deletion or disconnected reconciliation uses `reload` while reusing existing objects. Hiding the layer unsubscribes and clears queued changes. Tracks, conflicts, and audit data continue through the single-vessel historical APIs.
### Layer APIs And Global Stats
@@ -450,3 +460,15 @@ POST /api/v1/datasources/trigger-batch
The request body may pass `source_ids` for selected rows. Without `source_ids`, the backend filters by `product`, `module`, `is_active`, `run_status`, `collected`, `credential_status`, and `q`. The endpoint skips disabled sources, sources already running without `force`, and sources still inside their frequency window, then returns `triggered`, `skipped`, and `failed` groups.
**Core file**: `backend/app/api/v1/datasources.py`
## Compute source contracts and topic APIs
Epoch uses its official public CSV, retaining theoretical FP16 performance, original dates, status, anonymization/rounding flags, and upgrade relationships. Anonymous rows use content fingerprints plus equal-row ordinals to preserve snapshot multiplicity, without claiming persistent facility identity or assigning coordinates. Unknown countries remain flagged and are excluded from selected-country counts. TOP500 uses system-detail IDs, reads the edition from the list URL, and converts header/cell units into FLOP/s with an explicit FP64 LINPACK basis. Public pages may expose only part of an edition; this does not establish full TOP500 coverage.
Neither collector returns sample data on failure. Empty fetches or transformed results, schema errors, and losses exceeding 40% of the previous snapshot fail while retaining prior data. Incomplete writes are rolled back. TOP500 location migration copies only a unique exact name and canonical-country match to the new system ID, retaining the historical key; rank is never a migration key.
`compute_history_defaults.json` ships the original report data. `compute_configuration.py` reads a complete override from the `compute_comparison` category in `system_settings`; absent records use bundled defaults without migrations or manual initialization. `compute_history.py` performs same-year aggregation. Reference periods and publication dates remain distinct, and unknown publication days remain unspecified. `/api/v1/data-products/compute/series` reads configuration on each request with `Cache-Control: no-store`, offering global share, fixed US + China bilateral share and FP32 absolute observations. Innovation gaps remain independent. An empty override stays empty instead of silently restoring observations.
Under the same prefix, `GET/PUT /configuration` and `POST /configuration/reset` require admin/super_admin. Writes accept `{revision, config}` and reset accepts `revision`. Pydantic validates canonical countries, finite values, share totals, duplicate metric/year pairs and report references; each metric/year has one report basis. PostgreSQL category uniqueness and row locking serialize first writes and updates; stale revisions return 409. Configuration and before/after audit records commit in one transaction. Reset removes the override so subsequent reads use the current bundled defaults. `version` is a configuration revision, while `reviewed_at` is the supplied review date. Admin-curated data is not automatically labeled official. Source URLs are references, not scrape instructions.
`/api/v1/data-products/compute/facilities` reads successful snapshots with `compute_contract=v1`, applying country filters and per-source counts in SQL. Epoch excludes planned, decommissioned, superseded, and suspected duplicate records; TOP500 means list observations. Cross-source deduplication remains pending. Historical cutoffs use collection time, never pretending a recently fetched old source was available earlier. Geographic results are bounded and report truncation; total record counts are independent of this limit. Official site helpers are keyed by exact `source` + `source_id`; their access-map centers are site-level references only and retain their official URL in `source_note`.

View File

@@ -70,7 +70,10 @@ The unified event model is `earth.layer.changed`:
| --- | --- |
| `clear_then_reload` | Clear local frontend layer objects first, then force a refetch. Prefer this for deletes. |
| `reload` | Keep old objects until fresh data returns. Use it for location, metadata, or non-destructive updates. |
| `delta` | Used only for `earth_interactables`; upsert or remove objects by id. |
| `delta` | `earth_interactables` upserts/removes by id; ordinary vessel writes update confirmed state by MMSI through the dedicated `vessels` channel without clearing the layer. |
Vessel deletion still emits a `reload` reconciliation hint; individual `vessel_current_state` deletions also enter the vessel remove channel. Source notifications identify changed MMSIs, while transmitted values come from current state. Global subscriptions use `scope: "global"`; message-size limits split frames instead of discarding remaining vessels.
APIs must return HTTP 200 with an empty collection for real zero-data states; 5xx is reserved for real endpoint failures. After a delete event, if refetch fails, the frontend should keep the cleared state and show a lightweight error instead of restoring stale objects.

View File

@@ -321,7 +321,7 @@ The new vessel list entry point is no longer the legacy `/api/v1/visualization/g
GET /api/v1/vessels/snapshot?bbox=-180,-85.05112878,180,85.05112878&zoom=12&limit=3000
```
That endpoint reads `vessel_current_state`, returning the latest point per MMSI inside the freshness window. Raw `ais_raw_observations` remain available for tracks, audit, and situational analysis, but the display endpoint no longer scans and aggregates history on the fly. Earth sends a global bbox rather than the current camera viewport. If the `/ws` `vessels` channel is connected, it should act as a reload/dirty hint for merged refreshes, not as a per-AIS-delta full-layer rebuild path.
That endpoint reads `vessel_current_state`, returning the latest point per MMSI inside the freshness window. Raw `ais_raw_observations` remain available for tracks, audit, and situational analysis, but the display endpoint no longer scans and aggregates history on the fly. Earth sends a global bbox rather than the current camera viewport. Realtime updates use the global `/ws` `vessels` subscription to deliver canonical state by MMSI and update existing markers in place. Larger updates are split across frames without dropping vessels. Initial load, reconnection, and deletion reconciliation still use snapshots; see [Collector Architecture](backend-collectors.md).
## Custom REST / WebSocket Mapping Runtime

View File

@@ -1,5 +1,19 @@
# Earth Frontend Context
## Configurable compute observations
Compute cards read annual observations from `/api/v1/data-products/compute/series`; the admin Compute comparison section is the maintenance entry point. `compute-topic.js` fetches with no-store, listens for same-origin `planet-compute-configuration` notifications, and checks every 60 seconds while visible and enabled, plus when the page becomes visible. Observation refresh does not reload facility layers. `destroy()` releases the timer, channel and visibility listener. Country membership, full regional coverage and missing-value semantics remain backend-owned.
## Surface texture provenance
On 2026-09-21, `assets/8k_earth_daymap.jpg` was replaced with an 8192×4096 Web derivative of the paired Natural Earth I raster. Its loading path and rendering code remain unchanged. The source is a 16200×8100 EPSG:4326 image; the previous JPEG is preserved byte-for-byte under `resources/earth/archive/2026-09-21/`. Source/license records, the catalogue versus embedded-version discrepancy, SHA256 hashes, JPEG parameters, and recalculated world-file metadata live in `resources/earth/natural-earth-1/`. Large originals remain in Git-ignored `downloads/earth/`, outside the frontend release bundle.
The base-map selector defaults to Original basemap, listed first and saves the source chosen in the display menu. Natural Earth I remains available alongside “Original basemap”, a byte-identical copy of the September 21, 2026 archive; NASA 21K is source archive material only, while runtime delivery stays at one 8K texture. A new source replaces the old GPU texture only after successful loading and releases the old one then; failures retain the old texture and stale responses cannot overwrite the current source. This is a source switch, not an additional rendering layer. The physical texture is JPEG sRGB, does not bake political borders, and detailed boundary alignment is not yet validated. The starfield is restored byte-for-byte to commit `83a10a6c`: `celestial.js` matches that commit, `skyTint` and `skyBrightness` are removed, and `brightStarOpacity` is back to `0.88`; star density and color are no longer calibrated. The HD overlay uses a white `MeshBasicMaterial` with opacity `1` and `toneMapped:false`, with `texture-colors.js` centralizing sRGB read/write. The 0.128 path uses `encoding` plus `LinearTosRGB`; the 0.160.1 path uses `colorSpace` plus the normal colorspace fragment. There is no Phong white-light or colored twilight overlay, blue night tint, `cameraBoost`, or shoulder curve; daylight neutral scaling keeps brightness at or below the source maximum, and dark mode only applies neutral scaling. The original texture asset is unchanged. Natural Earth is intrinsically pale; this does not claim missing detail was restored. Earth still supports the Three 0.128 import-map runtime and frontend package Three 0.160.1 runtime, and WebGL1 derivatives remain enabled for `fwidth`.
Day/night shading uses a narrow smooth transition centered on the solar horizon (normal dot product ±0.12). The day side is capped at the source brightness; night-side linear illumination is 0.24, rising smoothly to at most 0.60 in source shadows to retain dark ocean and forest detail. Shading uses a neutral scalar without added exposure, colored veils, rendering passes, or textures.
Basemaps and TV channels share `hud-select-menu.js` for the HUD popover, list, selection mark, keyboard navigation, and positioning; `tv-source-menu.js` only adapts channel copy. Basemaps disable search, pagination, and counts. The trigger owner identifies popover clicks as internal to the mobile drawer.
This document describes the current real structure of the Earth display frontend. The focus is on helping future changes to the HUD, layers, media panel, real terrain, and BGP visualization avoid repeating past structural and state-sync pitfalls.
Related references:
@@ -159,6 +173,14 @@ Each module is responsible for its own:
`tv.js` owns the live / aggregation-news tabs inside `media-panel`. Toolbar open and tab-switch actions write back through `earth:tv-visibility-change` and `earth:tv-tab-change`: panel visibility remains viewport-scoped at `views.<scope>.panelVisibility.media-panel`, while the active tab is stored at `shared.mediaPanelActiveTab`. Refreshing the page therefore restores the user's last live/news state. Temporary hides from `closeTransientMobileOverlays()` carry `persist:false` and do not overwrite the preference.
`tv-source-menu.js` reuses HUD and legend-list styling in a popover with search, a scrolling list, and a fixed count footer. `tv.js` requests 50 entries from `/api/v1/tv/streams` and caches channel details; search runs against the complete backend catalog, and pagination uses `next_offset`. Cancellation and a request generation counter prevent stale results from replacing a newer search. Catalog refresh uses `selected_id` to restore a channel outside the first page.
`tv-playback-policy.js` owns live recovery decisions: all non-fatal HLS events remain with the player, while fatal network/media failures receive at most three recovery attempts. Switching requires automatic fallback to be enabled, no explicit user channel selection, and an embeddable backup. Async callbacks in `tv.js` verify player identity and channel; native video errors no longer bypass HLS recovery. `tv_streams.py` replaces the old bundled CCTV-4 CDRM URL, including saved copies, with WCETV's America-edition HLS stream, names the edition explicitly, and preserves custom streams. An HTTP success does not establish video decodability.
Sources with verified CORS support may explicitly set `use_proxy:false` for direct browser playback without segment forwarding. Other sources continue to use the TV proxy by default. The bundled WCETV source uses direct playback.
Source information reuses `createHUDPanel` and its existing 0.22-second CSS transition. The shared controller measures content into `--hud-body-content-height`, avoiding the non-interpolable change from `max-height: none` to zero. ResizeObserver updates content measurements and compensates the top position of upward-expanding dragged panels during animation, keeping their bottom edge fixed without disabling the body transition. The shell's top transition is paused during compensation to prevent drift from double interpolation.
`brand.js` manages Earth HUD brand resources. Static assets provide the default brand; runtime overrides come from `/api/v1/earth/brand`, and uploaded images are served from `/earth-brand-assets/...`. The frontend must treat logo/title images and text fallback separately: if an image fails, show the text title; if text fields are empty, rely on backend defaults so the HUD brand area never renders blank. The console Earth Content page owns saving and resetting brand configuration; the Earth frontend only consumes it.
`about.js` manages the About card inside Earth settings. Frontend defaults remain as a fallback, while runtime content is loaded from `/api/v1/earth/about`. If the request fails or fields are missing, the renderer must fall back per field so the settings page never renders an empty card. Admin exposes an Earth Content `About` tab; saving uses `PUT /api/v1/earth/about`, and restoring defaults uses `DELETE /api/v1/earth/about`.
@@ -179,6 +201,8 @@ The compute-center layer row has a notification badge for GeoJSON `unresolved` r
Location candidate state in the details card is cached in [info-card.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/info-card.js) by `entityType:entityId`. If the user closes the details card or unresolved queue and reopens the same compute center / BGP collector, previously collected candidates and status text are restored. Header-level `一键采用` prefers cached candidates, avoiding repeated online geocoding or LLM factcheck calls. After a location is saved, that entity's candidate list is cleared to a "refreshing layer" status so stale candidates do not keep misleading the user.
`runUnresolvedComputeCenterBatch()` owns a queue of entity contexts rather than panel DOM. `locationCollectStateCache` retains candidates, progress, and save results so rebuilt cards can hydrate the current state. `earth:compute-center-location-batch-change` updates the badge, `savedCandidate` excludes completed entries, and a ✓ entry remains after completion. Queue state lives only in the current Earth page: refresh, close, or route navigation interrupts unfinished work, while saved coordinates remain authoritative in the backend.
The `预览 / 保存` buttons on each candidate row use a single delegated `click` handler per candidate root (the `[data-collect-cache-key]` block in the details card, or `[data-unresolved-item]` in the unresolved queue), guarded by a `data-candidate-actions-bound` flag so it cannot be double-bound. Direct `pointerup` / `click` listeners on individual buttons and overlapping delegated handlers were removed. Candidate objects are no longer JSON-stringified into an HTML attribute and parsed back; buttons only carry `data-candidate-index`, and the handler resolves the candidate object from a module-level `Map` keyed by cache-key. This removes the entire class of failures caused by HTML entity escaping of `&` / `<` / `"` in candidate fields. Clicking `预览` dispatches `earth:preview-location-candidate`; `main.js`'s `previewLocationCandidate()` calls `showComputeCenterLocationPreview()`, which attaches a hollow breathing-ring sprite pair at the candidate coordinates (visually mirroring the BGP event ring) and focuses the camera on the candidate. Previewing another candidate replaces the ring; saving clears it and `spawnSavedComputeCenterLocation()` immediately spawns the formal compute-center interactable. Note that `main.js` has no module-level `earth` variable — every location-save / preview handler must call `const earth = getEarth();` first, otherwise the event handler throws a `ReferenceError` that the surrounding `.catch` swallows, producing the failure mode where the button "does nothing".
The `earth:compute-center-location-saved` reconciliation pipeline is deliberately silent on background-refresh failures. `spawnComputeCenterAfterLocationSave()` already presents the success toast and locked state; `refreshComputeCentersAfterLocationSave()` only reloads backend data when the scene is ready and no longer emits its own `已保存` toast. `handleComputeCenterLocationSaved()` runs refresh in the background after a successful spawn; only when spawn returns `null` (scene not ready) or throws does refresh take over the success toast. A refresh error is only `console.warn`'d — it must never surface as a `保存失败` message, because the save itself succeeded and the refresh is a follow-up sync.
@@ -191,7 +215,11 @@ The backend snapshot endpoint still requires `bbox`, but the Earth runtime treat
Vessel markers are rendered through `createInteractableLayer()` as batched `THREE.Points`, with `cluster` and `avoidance` explicitly disabled. Dense waterways may overlap. Dragging or inertial rotation skips hover picking; normal hover uses screen-space nearest-point picking. Do not reconnect vessels to dynamic screen clustering or per-frame Points rebuilds, because those interaction costs are what make a 3000-marker layer feel heavy.
If the `/ws` `vessels` channel is used by Earth, it should be a low-frequency reload/dirty hint only. Do not create multiple viewport subscriptions, and do not turn every AIS delta into a full layer rebuild.
AISStream and BarentsWatch share the `/ws` `vessels` delta channel. Earth subscribes with `scope: "global"` on its existing WebSocket, without changing subscriptions as the camera moves. The backend coalesces notifications by MMSI for one second, then reads confirmed `vessel_current_state` rows. Frames contain at most 1000 items; further frames carry the remaining vessels rather than truncating them. Raw source messages must not overwrite confirmed client positions. Deleting current-state rows also produces MMSI-based remove notifications.
The frontend coalesces short bursts per MMSI and uses `Interactable.updateItems()` to update position and color buffers in place. Heading-bucket changes touch only affected buckets, growing capacity when needed. Existing marker identities, selections, and materials survive. Initial entry, reconnects, deletion hints, and the minute reconciliation still use snapshots without first clearing the layer. A bounded snapshot must not treat truncated vessels as deleted, and older responses must not overwrite newer stream updates or removals received while the request was in flight. Snapshots include query-start `generated_at`; it is compared with stream frame time to prevent stale cached snapshots from rolling back new vessels or removals.
Ordinary vessel writes use the `delta` strategy on `earth_updates`; while the dedicated channel is connected, they no longer request whole-layer reloads. Deletion or disconnected reconciliation uses `reload` while reusing existing objects. Hiding the layer unsubscribes and clears queued changes. Tracks, conflicts, and audit data continue through the single-vessel historical APIs.
The legacy `/api/v1/visualization/geo/vessels` route has been removed. Frontend code should keep using `PATHS.vesselsApi` and can verify the current-state path through `diagnostics.source == "vessel_current_state"`.
@@ -221,6 +249,14 @@ The cruise sequencer handles generic logic: current target, queue order, camera
All material, layer, satellite, BGP, cable, terrain, celestial, and other style parameters are maintained here. Do not scatter magic numbers in module files.
## Render hot paths
`cable-batches.js` batches the complete cable and landing-point sets. `cables.js` still owns original business objects, picking, occlusion, and selection state. Do not make these interaction proxies draw individually again or return a render batch as the detail-card business object.
Satellite breathing runs in vertex shaders. `satellite-position-worker.js` moves full SGP4 position and initial trail calculations off the main thread; the main thread applies snapshots to interaction coordinates and render buffers. `satellite-propagation.js` supplies the same mathematics to the Worker, predicted orbits, and synchronous fallback. Reload, clear, and altitude changes must terminate prior work so stale snapshots cannot resurrect cleared layers.
The `i18n.js` MutationObserver translates only added subtrees, changed text, or changed attributes. Local clocks, status labels, and download progress must not synchronize all locale controls. Global locale synchronization belongs to initialization, locale changes, or subtrees containing new locale controls.
## Current Style Layers
CSS files in `frontend/public/earth/css/` each correspond to a specific component scope. Do not write global Earth styles into `base.css` unless they genuinely apply to everything.
@@ -333,7 +369,10 @@ If future cable, satellite, or news cruise is added, do not copy a new set of `m
[controls.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/controls.js) owns the Earth zoom state, and every zoom entry point must ultimately call `setZoomLevel()` to write the camera distance. Do not write `camera.position.z` from other modules, or the zoom percentage, drag sensitivity, and Interactable clustering thresholds will diverge again.
Interactable clustering is selected per layer through `cluster.strategy`. `stable-spherical` uses discrete zoom bands and local 3D bucket clustering, so BGP, compute centers, and Earth interactables do not regroup while the globe rotates inside the same band. `dynamic-screen` keeps the projection-based behavior for high-frequency realtime layers such as vessels, and `none` disables clustering. Stable cluster dots stay rigidly aligned to their 3D centroid projection and do not participate in 2D avoidance. See [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md) for strategy configuration and tuning.
`globe-navigation.js` maps zoom to `sqrt(R² + (D₀² - R²) / zoom²)` with minimum surface clearance and the matching inverse. The 100% camera pose remains unchanged; 500% stays outside the globe. Mouse and single-touch drags choose a mode on pointer-down from the actual camera frustum. A fully visible globe retains legacy screen-delta sensitivity, 0.18 smoothing and release inertia, including movement outside its silhouette. A viewport-clipped globe uses pixel-aligned ray/sphere grabbing only when the initial pointer hits its surface. Subsequent missed intersections pause surface updates and clear fling speed instead of using a closest-point substitute that can reverse rotation. Grabs starting in space use screen deltas. The mode stays fixed throughout the drag; auto-rotation is suspended during dragging and pinching. Returning from a pinch to one finger reselects the mode and rebases the anchor. Click suppression uses displacement from pointer-down, and cancellation clears inertia.
Interactable clustering is selected per layer through `cluster.strategy`. `stable-spherical` uses discrete zoom bands and local 3D bucket clustering, so BGP, compute centers, and Earth interactables do not regroup while the globe rotates inside the same band. `dynamic-screen` keeps projection-based clustering, and `none` disables it. Vessels explicitly disable clustering and avoidance so incremental updates do not rebuild cluster topology. Stable cluster dots stay rigidly aligned to their 3D centroid projection and do not participate in 2D avoidance. See [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md) for strategy configuration and tuning.
Wheel input has two paths. Traditional mouse wheels keep the 10% step and short animation, using `wheelZoomTarget` as the logical base for continuous wheel input. Trackpads and high-precision wheels use the pixel delta for continuous zoom and call `setZoomLevel()` directly instead of passing through the 10% stepped animation. The trackpad path also filters a short-window, old-direction residual delta after a real direction change so inertia tails do not pull a just-reversed zoom back in the previous direction.
@@ -369,3 +408,46 @@ Therefore:
For console structure, see:
- [Admin Frontend Context](/home/ray/dev/linkong/planet/docs/technical/en/frontend-admin-frontend-context.md)
## Spatial compute cards
`compute-topic.js` owns periods, measures, participants, and evidence. `compute_catalog.py` supplies canonical names, picker groups, and regional label anchors for 45 currently registered participants. Unknown historical values remain missing. APIs are no longer limited to CN/US. Group labels never become overlapping statistical denominators, and regional coordinates must not become facility locations.
Earth's existing Roboto Condensed face is now served locally from `assets/fonts/`, with its source and license in that directory and the shared declaration in `css/fonts.css`. The remote Google Fonts stylesheet no longer blocks the page; font family and weights are preserved.
`spatial-callouts.js` delegates layout to `spatial-callout-controller.js` and reuses `CalloutConnector` with stable region IDs. Layout is event-driven and retains valid old rectangles while work is pending; the projected source line follows the globe continuously. The layout reference is the direction of the actual source exit point, so a card stays near the object side instead of being pulled from a top-left geographic point to an opposite corner. Neighboring same-sector placements, shorter connectors, and fewer crossings are preferred; the solver may exchange free positions for at most three rounds rather than relayout everything each frame. In fullscreen Earth, nearby foreground positions are allowed so cards of normal size do not cross the whole globe merely to avoid it. In slow mode, sustained direction drift or connector crossings must persist for 1.8 seconds before proposing a joint reflow. Automatic proposals are at least 8 seconds apart and need an improvement of at least 18% plus a minimum absolute gain. Manual/inertial input retains the 72ms/12px response budget. Expansion and manual placement grant 12 seconds of reading time; hover, keyboard focus and dragging continue to lock a card, while residual pointer focus does not lock it forever. Manual placement is a preference: after its hold, automatic repair requires at least 45 degrees of source direction change. Restored positions are not permanent pins. Rear connectors retain the globe-outline cue without changing card identity. Local position/size persistence and reset-to-auto are part of the controller. A `ResizeObserver` handles resizing and collapse; configuration and source windows retain shared drag handles.
Object presentation is shared through `PresentationController` and the new object-callout anchor/layout path. A click on a BGP, compute, vessel, satellite, or cable object draws the connector before placing the adjacent card. The source anchor begins at the actual click or original region label, then follows a visible highlight geometry sample as rotation approaches occlusion. The stable `layoutReferencePoint` is derived from the actual connector exit direction, so `packDirectionalEdgeCallouts` keeps cards on the corresponding screen edge and leaves the globe's center open; a card may move inward only when that edge lane is unavailable, and it may never cover its own connector exit point. The bounded joint assignment considers up to 24 beam states and 32 candidates, avoids existing HUD rectangles, then allows at most three safe exchanges to reduce directional crossings; the previous solver remains a fallback and the work is not a full relayout every frame. A slow source-direction gate also checks whether the actual card target reverses more than 90 degrees from the source before committing a change. Manual drag and hover locks take priority. The source point itself remains a per-frame visual connection target; fixed objects track live positions while cards remain draggable. Rear-side objects retain a globe-outline direction cue, a subdued connector, and a hollow source point; these are orientation guides rather than geographic points. Country candidates are capped at 96 with a hard limit of 128 and cached, so GeoJSON is not scanned every frame. Compute cards keep their original point independent from dynamic source-point movement, and the shared card endpoint slides along the obstacle perimeter. Cable presentation prefers the true ray-hit local point, then samples highlighted geometry by `cableId`. Mobile keeps the existing mini popup and opens drawer details. A stale asynchronous presentation must never dismiss a newer selection.
`configVisible` and `cardsEnabled` are separate. X/Escape closes only configuration; disabling the switch releases connectors and restores the previous map query/layer state. Configuration and source windows bind HUD typography directly so reparenting cannot enlarge their text. The main card presents the selected compute metric as a large share percentage or an absolute value with unit, observation year, historical context, and source entry. Missing comparable data is explicit; no empty trend chart is rendered without observations. Facility records and coverage remain complete in the expanded auxiliary area, where sample counts are evidence context rather than a compute value or global coverage rate. Long lists open only when needed. Mobile preserves the same single metric meaning in its accordion. Local position/size persistence and reset-to-auto remain available. Stable DOM, dimensions, and reading state survive updates, and stale responses cannot re-enable cards.
Motion presentation uses the shared `main.js` input state. Sustained drift or crossings trigger a joint assignment, rather than each card competing independently for leftover slots. `callout-region-path.js` provides the short connection and terminal stub used both for rendering and intersection scoring. Candidate paths are precomputed and layout states are cloned only after pruning. Existing cards remain visible during planning; queued reflows survive frame throttling.
The shared `CalloutConnector` calculates polyline length from its points instead of reading `getTotalLength()` after each SVG write, avoiding repeated synchronous layout during motion. Initial draw animation is retained, so existing cruise presentations receive the same optimization.
On mobile, compute uses a compact regional overview in the existing drawer. Rows are an in-place accordion: tapping the same header collapses it, tapping another collapses the previous row, and at most one expanded card is shown. The expanded card keeps trends, region members, facilities, media, and sources; there is no region subpage, while Settings remains a separate on-demand page. The shared HUD collapsible panel animation is reused and respects reduced motion. Summary loading remains active when map cards are off; enabling cards shares the map loading event and cancels stale requests. Drawer handlers use `composedPath()` so inner HTML replacement cannot collapse the active row. Facility names remain in a scroll region capped at 240px, with left-aligned wrapping rows and at least 42px touch height.
`/series` retains source-backed annual observations; the default primary metric is global compute-device scale split into basic, intelligent, and supercomputer capacity for 2020–2023. Reviewed absolute observations are 302 for 2022 (`caict2023`) and 435 for 2023 (`caict2024`); Japan's global share is 6%/5%/4%/4% for 2020–2023. Product groups APAC excluding China and Other do not map directly to report Japan or “other”; Europe remains `null` until membership aligns with the report. Bilateral share keeps a fixed US + China denominator. Missing primary values may show the latest reported year (currently 2023), while an explicit 2026 selection stays missing. Unconfirmed comparability does not produce year-over-year change. `/facilities` filters and counts unique canonical country codes in successful collection snapshots; coverage counts remain auxiliary, and are not capacity or a global coverage rate. `compute_catalog.py` groups China, US, Europe, APAC excluding China, and Other, covering 45 canonical country codes; these display groups never become statistical denominators. `earth_geography_policy.py` shares display ownership with map policy, grouping Taiwan/Hong Kong/Macao source subdivisions under China while retaining `source_country`. API review metadata is `reviewed_at=2026-09-22` with the matching version date. An independent Epoch share mode is not implemented. Innovation differences are supplied only when both CN and US are selected.
### Boundary surface attachment
Normal borders, coastlines, claim lines, highlights and landing points use `earth-surface.js` to share the HD sphere UV-triangle surface. Borders draw after the texture but before landing points. Highlight lines are always transparent-queue objects with depth testing, preventing through-globe display or overpainting markers. Geometry is computed and cached at construction, not reprojected per frame. See the [render-layer specification](earth-render-layer-order.md).
### News ticker edge fading
A separate pseudo-element paints the ticker background and horizontal rules, fading to full transparency outside the text bounds. The shell has no rectangular backdrop filter, box shadow, or text mask. The region label sizes to its content and the scrolling text follows directly without fading. Alert states change only the gradient rule color so they cannot reintroduce rectangular shadow edges.
Compute cards directly reuse the shared `.hud-panel` background, border, shadow, and glass treatment. Summary, essentials, and detail areas add no separate black surface. Their headers share inset spacing with the layer panel through `.hud-panel__header--inset` / `.layer-panel-header`; primary numbers remain prominent and essential information stays visible.
Saved placement restoration migrates legacy central positions: only old central coordinate locks without the current position marker return to automatic layout, retaining dimensions. Existing edge positions are preserved and versioned. New drags save `positionVersion=2` and continue to honor deliberate placement. The existing storage key is retained instead of clearing every personalized position.
### First frame and spatial card entrance
`earth-runtime-plugin.mjs` bundles local Earth modules into content-hashed entries and shared chunks for both development and production. The existing import map still pins third-party versions; relative assets and workers retain the `/earth/js/` base. `earth-preload.js` requests the selected full-size map early. `earth-texture-image.js` decodes an ImageBitmap asynchronously and transfers that same image to Three.js without resampling or changing texture color/sampling settings. Replaced and destroyed textures release their bitmaps. The full-resolution surface precedes independent concurrent data layers; data refresh retains it. Versioned boundary geometry, picking indexes and claim lines use HTTP caching and parallel requests. The live player engine is imported on demand; media and realtime connections start after the surface frame.
The root attributes `data-earth-first-frame-ms` and `data-earth-surface-ready-ms` record navigation-to-render timings. Measure cold and warm caches separately; local warm results are not a guarantee for all first-visit networks.
Positioned cards stay hidden while the shared connector draws for 420 ms, then fade in using the HUD 180 ms opacity timing. Targets remain fixed during drawing; subsequent motion does not replay entrance. Reduced-motion preferences skip this animation.
Object clicks, search focus and gesture confirmation share `PresentationController`. Cable selection updates selection/legend state without independently opening the card. InfoCard reveal callbacks are versioned so staging or hiding invalidates older callbacks. Connector completion gates card reveal; cancelled cruises no longer hide a newer selection. Object details reuse the regional-card edge candidates and collision scoring, with dimensions measured while staged. Selecting a mobile search result collapses the search drawer so the focused object, connector and compact card remain visible.

View File

@@ -143,6 +143,8 @@ The land/ocean base is an Earth base-map asset and preloads at startup; the "Bor
## Submarine Cables and Landing Points
`cable-batches.js` batches cable lines and landing points separately. The Sprite material and size parameters below remain owned by picking and selection proxies in `cables.js`; their color, opacity, scale, and visibility update the style texture or instance attributes. Drawing uses `LineSegments` and instanced billboards with the existing textures, colors, render order, and globe occlusion rules.
| Name | Variable | Current Value | Location / Notes |
| --- | --- | --- | --- |
| Default cable color | `CABLE_COLORS.default` | `0xffff44` | Used when no data color available |

View File

@@ -1,5 +1,7 @@
# Earth Render Layer Order
The compute topic reuses the compute-center Interactable icons, depth behavior, and layer order. Country/year changes replace that layer's data and clear prior selection. Its HUD is screen UI excluded from Earth picking in `main.js`. Regional cards are tethered in screen space; rear anchors do not draw connectors through the globe. Facility details reuse the existing card and mobile details drawer. The existing outer atmosphere shell remains, while the starfield is restored to commit `83a10a6c` (`brightStarOpacity:0.88`; no tint/brightness calibration). The HD overlay is a white `MeshBasicMaterial` with opacity 1 and `toneMapped:false`; `texture-colors.js` selects the r128 `encoding`/`LinearTosRGB` path or r160 `colorSpace`/colorspace fragment. No Phong white light, colored twilight, blue night tint, camera boost, or shoulder curve is added. No new post-processing, GPU draw pass, or high-resolution texture was added; the compute topic adds 45 local SVG flag assets.
This document records the current Earth renderer's layer order and the intent of each layer. When adjusting `renderOrder`, radius offsets, depth strategy, or pointer interaction, update this document accordingly.
Note: the layer control panel order and the registration / startup load order are two separate semantics.
@@ -20,12 +22,12 @@ Note: the layer control panel order and the registration / startup load order ar
| 0.86 | Land/ocean base fill | `country-boundaries.js` | `landAltitudeOffset = 0.32`; ocean `#010609`, land `#080f1b` | Raycast disabled | Base map remains usable even when country borders are off; radius is separated from the base sphere to avoid far-zoom z-fighting. |
| 0.96 | HD Earth texture | `earth.js` | `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | Surface picking target when visible | HD texture always overlays the land/ocean base fill; radius must stay above the land/ocean base and far enough from the base sphere. |
| 1 | Atmospheric glow and clouds | `earth.js` | Atmosphere / cloud spheres | Not in normal object selection path | Cloud layer controlled by the "Cloud Layer" toggle. |
| 1 | Submarine cables | `cables.js` | `CABLE_CONFIG.line.renderOrder` | Cable picking path | Preserves existing cable layer level. |
| 1 | Submarine cables / landing points | `cables.js`, `cable-batches.js` | All cable segments share one `LineSegments`; all landing points use instanced billboards; render order remains `1`; cable altitude offset stays `0.2`, while landing-point anchors use the HD surface offset `0.48` | Original `Line` / `Sprite` objects retain per-item picking and selection state but no longer draw individually; landing-point sphere occlusion remains, with `depthTest: false` on the batch | Style textures and instance attributes preserve color, pulse, size, visibility, and click behavior. |
| 1.2 | Real terrain | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` plus terrain displacement | Raycast disabled | Terrain overlays HD texture; temporarily hidden when HD texture is off, restores to prior state when re-enabled. |
| 2.05 | Grid lines | `earth.js` | `CONFIG.earthRadius + 0.14` | Raycast disabled | Low-opacity lines over HD texture. |
| 2.2 | Country borders | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`; claim lines have no extra lift | Raycast disabled | Line geometry still has its own `renderOrder`, but it shares the exact same radius as the HD texture shell to avoid parallax while the globe rotates. |
| 2.29 | Country border hover glow | `country-boundaries.js` | Hover radius + glow offset | `depthTest: false`, raycast disabled | Additive glow to reinforce border edge and terrain hover visibility. |
| 2.3 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: false`, raycast disabled | Neon red-orange hover line; aligned with the normal borders and HD texture shell to avoid ghosting or floating; China and Taiwan share the same highlight group. |
| 0.97–0.99 | Country borders | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`; claim lines have no extra lift | Raycast disabled | Line geometry still has its own `renderOrder`, but it shares the exact same radius as the HD texture shell to avoid parallax while the globe rotates. |
| 0.98 | Country border hover glow | `country-boundaries.js` | Same surface mapping, no extra lift | `depthTest: true`, transparent render queue, raycast disabled | Additive glow to reinforce border edge and terrain hover visibility. |
| 0.99 | Country border hover line | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: true`, transparent render queue, raycast disabled | Neon red-orange hover line; aligned with the normal borders and HD texture shell to avoid ghosting or floating; China and Taiwan share the same highlight group. |
| 3 | Satellite footprint fill / Iridium coverage ring | `satellites.js`, `iridium-footprint-adapter.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-tested; Iridium adapter fill / ring use the same renderOrder | Footprint above land / texture / terrain and country borders, below compute centers and satellites. |
| 3-4.5 | BGP collectors, event rings, and event markers | `bgp.js`, `interactable.js` | Collector markers use `BGP_COLLECTOR_RENDER_ORDER = 4.4`; event markers use `BGP_EVENT_RENDER_ORDER = 4.5`; overlays remain in BGP-owned groups | Screen-space Interactable picking; BGP collectors/events may use collision avoidance where appropriate | BGP collectors share the surface-facility band with vessels; BGP events share the compute-center band. |
| 4.3 | AIS vessel track lines | `vessels.js` | `VESSEL_RENDER_ORDER - 0.1`; `CONFIG.earthRadius + VESSEL_CONFIG.track.altitudeOffset` | Follows vessel selection, not independently picked | Recent track for the selected vessel, below vessel markers. |
@@ -36,6 +38,14 @@ Note: the layer control panel order and the registration / startup load order ar
| 12+ | Satellite locked ring, halo, predicted orbit | `satellites.js` | `SATELLITE_CONFIG.overlayRenderOrder` and offsets; predicted orbit follows the same real-altitude toggle and fixes the lock-time globe pose to draw a closed inertial orbit; returns to same-sphere mode when real altitude is disabled | Satellite overlay path | Used for selected/locked satellite emphasis. |
| 98-100 | Sun / moon halo and sprite | `celestial.js` | Fixed renderOrder | Celestial picking disabled | Foreground celestial sprites. |
## Full-set rendering and updates
- Batching changes GPU submission, not the number of satellites, cable segments, or landing points. It does not filter data by the visible hemisphere. Every adjacent cable vertex pair remains present, and picking still returns the original business object.
- Satellite foreground and backdrop remain two `Points` draws. Vertex shaders compute breathing from static per-point parameters and a shared frame-time uniform; existing hover/locked state still controls point masking.
- `satellite-position-worker.js` uses the same Three.js / SGP4 versions as the main thread. `satellite-propagation.js` owns shared orbit, display-altitude, and fallback calculations. Full positions and initial trail samples use transferable arrays; only one calculation is in flight. Reload, clear, and altitude-mode changes terminate the previous Worker.
- Worker startup failure or `SATELLITE_CONFIG.workerStartupTimeoutMs` expiry falls back to the shared synchronous calculation. Counts, APIs, the existing update interval, and trail length are unchanged.
- Verify full draw ranges, per-item picking, locked overlays, trails, rear-side occlusion, visibility toggles, and resource release after clearing. Frame-time comparisons require identical data, view, and resolution.
## Toggle Behavior
| Toggle | Behavior |
@@ -65,7 +75,16 @@ Maintenance rules:
| --- | --- |
| Earth coordinate hover | When HD texture is visible, uses the HD texture overlay as the surface picking target; otherwise uses the Earth base sphere. |
| Country border hover | Converts surface pick coordinates to lat/lon, then uses GeoJSON point-in-polygon; the border hover line itself does not receive raycasts. |
| Country border hover visual | On hover, dims normal border lines and draws no-depth-test glow and solid lines. |
| Country border hover visual | On hover, dims normal border lines and draws depth-tested glow and solid lines below landing points. |
| China / Taiwan hover | `CHN` and `TWN` are grouped in the same hover highlight group; the tooltip still shows the actually-hit feature. |
| Terrain | Acts as a visual layer only; `terrain.raycast` is disabled. |
| Satellites | Uses screen-space satellite picking to prevent footprints or surface layers from blocking satellite clicks. |
### Surface attachment and landing point order
`earth-surface.js` projects geographic vertices onto the high-resolution SphereGeometry UV triangles at geometry construction time. Boundaries, coastlines, claim lines, hover outlines and landing points share this surface mapping. Long edges are subdivided using the surface grid spacing. Landing-point anchors now use the texture shell offset of 0.48; cable line offsets remain 0.2. The draw order is texture 0.96, boundaries/highlights 0.97–0.99, then landing points 1. Hover outlines always use the transparent queue with depth testing, so they do not draw through the globe or over landing markers. Geography is unchanged, and cached geometry is not rebuilt during rotation.
### Startup loading versus drawing order
The selected full-resolution surface loads before independent data requests; boundary construction no longer blocks its first frame. This does not change renderOrder, depth behavior, geometry precision or image resolution. Data refresh retains the surface, with texture disposal limited to successful replacement or scene teardown.

View File

@@ -1,5 +1,9 @@
# Admin Frontend Context
## Compute comparison configuration
The Earth content `compute_comparison` section mounts `ComputeComparisonSettings.tsx`. Structured forms maintain report sources, annual country observations and innovation gaps using the shared Panel, Input, Select, Button and confirmation dialog; operators do not edit JSON. Drafts survive section switches, full-page unload warns about unsaved work, and discarding a draft or restoring defaults requires confirmation. Management data loads on activation. Publishing sends the backend revision to prevent overwriting concurrent edits, and failures preserve the draft. The content area scrolls internally while the heading and publish controls stay in the first viewport. `compute-comparison-model.ts` owns types and same-origin update notifications; `compute-comparison.css` only owns this section's layout.
This document describes the current real structure of the console frontend. The goal is to help future page development, table refactoring, layout governance, and state consolidation quickly find the right entry points.
Related references:

View File

@@ -16,6 +16,10 @@ The refactored rules:
- Earth renders only city-level or better locations.
- Local JSON registries are not runtime candidate sources for compute centers or BGP collectors; persisted location facts live in database dimension tables.
For compute-center display, `precise` and `site` candidates keep their source coordinates. `city`, `estimated`, and `needs_confirmation` records that share a coordinate are separated only in `displayLatLon`: a stable hash of `source` and `source_id` selects an irregular cloud direction and a radius in the 0.4–6 km range. The display offset is deterministic, is neither a circle, grid, nor spiral, and never uses a database row ID or list order. Source coordinates are never written back. Tooltips and detail cards label these points “City-level approximate location (dispersed display)”.
The resolver priority is confirmation cache, source precise coordinate, official registered site, source city/site, then unconfirmed cache. A city-level approximation must never be presented as `precise` or as a guessed headquarters. Official registration helpers are keyed by exact `source` + `source_id`; the current entries cover Gyoukou 2.0/2.2 at JAMSTEC Yokohama and Sophia ALCF (three records across two campuses). Their official access-map centers are site points only, not rack/building survey points, and each candidate keeps the official URL in `source_note`.
## Core Interfaces
```python
@@ -51,7 +55,7 @@ class LocationResolver(Protocol):
def resolve(self, query: LocationQuery) -> ResolverOutput: ...
```
`LocationPipeline.collect_candidates()` returns sorted candidates plus `attempted_queries`; `resolve_best()` returns the best candidate with diagnostics. The default sort key ranks source, precision, and confidence, then deduplicates candidates with the same source and rounded coordinates.
`LocationPipeline.collect_candidates()` returns sorted candidates plus `attempted_queries`; `resolve_best()` returns the best candidate with diagnostics. The default sort key ranks source, precision, and confidence, then deduplicates candidates with the same source and rounded coordinates. Legacy records are reclassified through the same path as `country`, `region`, or `unknown`; non-renderable records remain in the complete `unresolved` list.
## Built-In Resolvers

View File

@@ -123,6 +123,9 @@ The default guide follows the BarentsWatch official tutorial and reminds you to
### AISStream Realtime Vessels
Once the vessel layer is open, subsequent AISStream and BarentsWatch positions update automatically. Selected vessels stay selected during updates, and reconnecting automatically reconciles the display without repeatedly toggling the layer.
`AISStream Realtime Vessels` is the global AIS WebSocket collector. A passing connection test only confirms API key + endpoint format. Actual global vessel data requires the backend `aisstream_vessels` collector to stay connected and write to `ais_raw_observations`.
Steps:
@@ -160,7 +163,11 @@ Providers and models accept presets or arbitrary custom IDs. Common fields:
- Max Tokens, Anthropic Version: keep defaults if unsure
- Timeout / Retry: timeout and retry attempts
The plug icon at the end of the Base URL input runs a connection test. A passing test echoes the model's short reply.
The plug icon at the end of the Base URL input runs a lightweight connection check: it checks the proxy and model catalog, then confirms the selected model is listed. Use Playground afterward to verify generation access and the model's reply; the catalog check itself does not generate a response.
Use the refresh icon at the top right to update the available models. A successful refresh saves the catalog for later visits; catalogs with release dates show newer models first. Refresh preserves the current model, credentials, URLs, and unsaved edits. Select a model and click Save to change the model used by the application. A failed refresh displays an error and keeps the previous catalog.
Refresh uses the base URL, protocol, and API key currently entered in the form, so you can fill new configuration and refresh before saving. Credentialed providers require a key for the matching service region; an empty Ollama catalog means no models are installed. Selecting a model with a known protocol also updates the protocol field. Custom OpenAI gateways retain the manually selected protocol.
### Tools
@@ -262,6 +269,8 @@ Earth at `http://localhost:3000/earth` is the public 3D situational page; no log
A single globe view of: BGP events and observations, satellites and tracks, cables and landing points, compute centers, country boundaries / graticules / high-res tiles / clouds / terrain, news live streams and situational news, search and focus details.
At 500% zoom the viewpoint remains outside the surface. Mouse or single-finger dragging keeps the grabbed surface under the pointer, with auto-rotation suspended while held. Release retains inertia; pausing before release stops it. After a pinch, the remaining finger can continue dragging immediately.
### Layer Control
The right-side layer panel toggles layers. Common layers: graticule, country boundaries, high-res tiles, atmospheric clouds, cables, compute centers, BGP, satellites, AIS vessels, terrain. Satellite tracks are no longer listed as a standalone layer; they are controlled from Settings.
@@ -272,6 +281,8 @@ Dependencies:
- Tracks depend on satellites
- With high-res tiles disabled, the globe shows the base map with edge highlighting
The base-map selector defaults to Original basemap, listed first. The display menu can switch among the available map sources and saves the selection locally. The existing Natural Earth I source and an “Original basemap” option remain available; the archived option is a byte-identical copy of the September 21, 2026 archive; NASA 21K is source archive material only, while runtime delivery stays at one 8K texture. Loading replaces the previous GPU texture only after success, releases the old texture then, keeps it on failure, and ignores stale responses. The switch does not add a rendering layer.
### Legend
The bottom-left legend follows the focused or enabled layer. Covered: cables, satellites, country boundaries, compute centers, BGP, AIS vessels.
@@ -282,11 +293,21 @@ AIS vessel legend colors by type: cargo, tanker, passenger, fishing, military, m
Search finds cables, landing points, satellites, compute centers, BGP events, BGP observers. Results jump to and focus the object.
Click a BGP, compute, vessel, satellite, or cable object on the globe to draw a connector from the object before revealing a nearby detail card. The card follows the source direction toward the nearest usable screen edge, leaving breathing room around the globe; it never covers its own connector exit point. As the globe rotates, the source moves smoothly toward a visible part of the highlight. Fully hidden objects retain an outline direction cue with a lighter line and hollow source point. Cards stay readable and remain draggable; their connection points adjust along the edge automatically. Mobile shows a compact popup first, with full details available in the drawer. Dropdown menus and tool lists retain their existing opening behavior.
### Choosing a News Live Stream
Open the live tab in the media panel and click the current channel to open the search menu. Search covers the complete catalog. The list loads 50 channels at a time and appends another page when you scroll to the bottom. A fixed footer shows the loaded and matching channel counts; a failed request can be retried below the list. The default source is Al Jazeera Mubasher, using its HLS playback URL. Other channels remain subject to availability of their playback service.
Buffering and recoverable playback errors do not switch channels. A manually selected channel remains selected for the current page session; refresh to retry after failure. An automatically started channel may switch only after fatal recovery is exhausted, automatic fallback is enabled, and a playable backup exists. The bundled CCTV-4 source uses WCETV's directly playable America-edition HLS stream. The edition is identified in the channel name because its schedule may differ from the Asia edition. The old bundled CDRM URL is no longer passed to the generic decoder; other custom CCTV-4 URLs remain unchanged.
Source information briefly expands when a channel loads, then collapses with the existing HUD animation. Manual toggles use the same component. Expanded height is measured from actual content; panels dragged to the bottom retain their bottom anchor while animating upward.
### Coordinate Candidate Collection
Compute center and BGP observer detail cards support automatic coordinate-candidate collection. Click the object then use "Collect Coordinate Candidates" or "Re-collect Coordinates". The backend assembles candidates from source coordinates, public-org registry APIs, and online geocoders. When regular sources have no candidate, the current default AI Provider runs one LLM factcheck fallback. BGP observers' stored coordinates only fill query context; they are not returned as candidates.
Candidates preview on Earth directly. Saving a compute-center candidate writes to the `compute_center_locations` dimension table and refreshes the layer immediately. The notification badge at the top-left of the compute-center layer shows the unresolved count; clicking it opens the queue, supports single collection, or "Adopt All" to save the top-confidence candidates from top to bottom. Records without candidates stay in the queue rather than being faked to country centroids.
Candidates preview on Earth directly. Saving a compute-center candidate writes to the `compute_center_locations` dimension table and refreshes the layer immediately. Precise and site candidates keep their source coordinates. City-level, estimated, and confirmation-required records that overlap are dispersed only for display using a stable source + source ID placement; the source coordinate is never changed. Their tooltip and detail card say “City-level approximate location (dispersed display)”. The notification badge at the top-left of the compute-center layer shows the unresolved count; clicking it opens the queue, supports single collection, or "Adopt All" to save the top-confidence candidates from top to bottom. Records without candidates stay in the complete queue rather than being faked to country centroids or headquarters.
Recommended single-object flow:
@@ -298,6 +319,8 @@ Recommended single-object flow:
Adopt All is for batch processing the compute-center unresolved queue. It starts from the top and adopts the highest-confidence candidate. Records without factual support remain in the queue. When WebSearch is disabled, single locate and Adopt All are disabled because location validation depends on factual lookup.
The batch continues within the current Earth page when you close the candidate panel, inspect another object, or switch browser tabs. Reopening the queue restores progress and results. Saved entries are not collected again, and a ✓ entry beside the layer remains available after completion. Refreshing, closing, or navigating away from Earth interrupts unfinished work; coordinates already saved remain stored.
### Settings
The settings panel is grouped into Runtime, Display, Panels, Motion, Shortcuts, and System. It covers rotate / cruise / motion mode, cruise modules (BGP/news/compute centers/vessels/cables/satellites), view (satellite display style, hover tooltip, satellite idle breathing, real satellite altitude, track display, compact dots, day-night mode, panel toggles), motion debug mode / input source / skeleton-only / recognized-gesture whitelist, shortcut enablement and remapping, default globe size, terrain opacity, reset.
@@ -386,3 +409,19 @@ Docs supports: category navigation, Markdown rendering, tables and code blocks,
- [Quickstart](/home/ray/dev/linkong/planet/docs/technical/en/quickstart.md)
- [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md)
- [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)
## Compute intelligence
Administrators maintain card data in **Earth content → Compute comparison**. Add a report title, URL, publisher, report year and page/figure locator under Sources, then add country values by observation year, metric and source under Observations. Save and publish applies the complete configuration. Global shares use the world denominator; absolute values use FP32 EFlop/s. Bilateral shares are derived from same-year US and China observations, and regions require every member. Do not mix FP16/FP64 capacity, regional estimates or facility counts. Omit missing countries; enter zero only when the source explicitly reports zero.
Innovation gap observations remain independent US-minus-China values. Referenced reports cannot be removed until their observations are removed or reassigned. Duplicate metric/year rows, missing sources and global shares totaling over 100% are rejected. Concurrent edits return a conflict while preserving the local draft. Restore defaults requires confirmation and restores the bundled report data. Publishing and restoring are audited. Same-origin Earth pages receive a refresh notification; other visible pages check once per minute and hidden pages check when visible again. Reports must still be reviewed manually; configuring a URL does not scrape a report.
Open the analysis icon beside Compute Centers, then enable Show region cards. Connectors draw to their target positions before the cards fade in; subsequent globe motion does not replay entrance. Closing the configuration window or pressing Escape leaves the cards enabled; the analysis icon reopens configuration, while the switch disables the cards. Anchors rotate with Earth while cards remain readable. Dragging or expanding grants reading time; after a substantial change in geographic direction, cards can move together when that materially reduces crossing or detours. Active interaction prevents movement, while manual positions remain preferences rather than permanent pins. Rear regions retain their state without a connector drawn through the globe. Existing layer, news, and media windows remain available.
The picker groups the canonical comparison catalog as China, US, Europe, APAC excluding China, and Other; the groups are display buckets, not statistical denominators. It offers period and metric selection. Bilateral share still uses US + China as its fixed denominator; unavailable measures stay missing for other participants. Regional anchors are label positions, never facility locations. Source buttons open report evidence and facility buttons reuse the existing detail card. Mobile uses a compact regional overview in the drawer. Each row is an accordion: tapping the same header collapses it, tapping another collapses the old row, and at most one long card is open. The expanded card keeps trends, region members, facilities, media, and sources; there is no region subpage. Settings remain a separate on-demand page.
The default national metric is global compute-device scale, covering basic, intelligent, and supercomputer capacity with observations from 2020–2023. The reviewed absolute observations are 302 for 2022 (`caict2023` evidence) and 435 for 2023 (`caict2024` evidence). Japan's global-share observations are 6%/5%/4%/4% for 2020/2021/2022/2023. Product groups APAC and OTHER do not mean the report's Japan or “other” categories, and EUROPE remains missing until its membership matches the report. Bilateral share always uses US + China as its denominator. A missing primary value may show the latest reported year (currently 2023); explicitly selecting 2026 still shows missing and does not backfill. Unconfirmed comparability does not produce year-over-year change. National report metrics remain separate from public facility samples; coverage counts remain auxiliary evidence, never computed capacity or a global coverage rate. TOP500 and Epoch may overlap, so source-record counts are not a global census. The bundled reports were reviewed on 2026-09-22; published overrides use their maintained review date and observations. An independent Epoch share mode is not implemented.
This prototype shows up to six content cards. The main card centers on the selected compute metric: a large share percentage or absolute value with its unit, observation year, historical context, and source entry. Missing comparable data is shown as “No comparable data yet”; a trend is omitted when there are no observations rather than rendered as an empty chart. Facility records and coverage metadata remain complete but move to the expanded auxiliary area; sample counts are evidence context, never a compute value or global coverage rate. Region previews may show member counts, and long lists open only when needed. Desktop compact cards are about 268px wide with content-driven height; expanded cards support user resizing up to 480px. Mobile keeps the same single metric meaning in the one-card accordion rather than promoting facility counts. Local card position and size are saved, with a reset-to-auto action. The content layout leaves room for later image and video integration, but those media data are not connected yet. When outside space is insufficient, cards can travel through the globe's foreground instead of teleporting or shrinking to labels.
When the map cards switch is off, the overview summary still loads. Turning it on shares the map loading event; a newer request cancels the previous one. Drawer event handling follows `composedPath()` so replacing inner HTML cannot accidentally collapse the active row. The shared HUD collapsible panel animation is reused and respects reduced-motion preferences. Facility names remain in a scroll region capped at 240px; rows are left-aligned, wrap long names, and retain at least 42px of touch height.

View File

@@ -60,6 +60,7 @@ This document standardizes terms used across Intelligent Planet, the console, ba
| `traffic_stats` | 流量统计 | Cloudflare Radar traffic statistics |
| `as_stats` | 自治系统统计 | Cloudflare Radar AS statistics |
| `compute_center` | 算力中心 | Aggregated Earth compute-center object |
| `compute_intelligence` | 算力与资源态势 | Compute intelligence: national observations, public samples, history, and sources |
| `generic` | 通用数据 | Generic collector output |
| `generic_records` | 通用记录 | Generic mapping output |

View File

@@ -1,6 +1,24 @@
# `planet.sh` Startup Performance Optimization
## Background
## Windows desktop launcher
The launcher prefers a new Windows Terminal window and explicitly runs PowerShell 7 (`pwsh.exe`), independently of the default Terminal profile. Without PowerShell 7 it falls back to Windows PowerShell; without Terminal it starts the available PowerShell directly. The CMD bootstrap also prefers PowerShell 7. Ordinary entries request elevation; entries already running as administrator still dispatch to the preferred terminal without another elevation request. The child validates its privileges and enters WSL directly, preventing recursive windows. The existing post-restart WSL shell is retained.
On first use, open the actual project folder in Windows Explorer and double-click the repository's `planet.cmd`. It derives the WSL distribution and Linux project directory from its own location, creates `Planet.cmd` on the current user's actual Windows desktop, then follows the normal restart flow. Both `\\wsl$\distribution\...` and `\\wsl.localhost\distribution\...` work. Local Windows-drive checkouts use the default WSL distribution and `wslpath`. Windows resolves the desktop location, including redirected desktops such as OneDrive; WSL, zsh and project dependencies must already be installed.
Subsequent desktop launches use the bound target, request the existing administrator elevation and call `planet.sh restart --allow-lan`. No test ports are embedded: defaults remain frontend 3000, backend 8000 and AI Provider 8010, subject to explicit `DEFAULT_*_PORT` environment overrides. Successful restarts open Earth using the actual `ports.env` values and retain the WSL shell; failures display their error without opening a success page. `scripts/windows/planet-launcher.ps1` owns path resolution, desktop installation and elevation; `scripts/windows/restart.zsh` only bridges to the existing lifecycle script.
Use `planet.cmd --install-only` on Windows to generate or validate a launcher without restarting services. Re-running the repository entry updates the same target's launcher. Other projects and unmanaged files are preserved by choosing a target-specific filename. After moving a checkout, regenerate from its new location; old launchers never guess another directory. Desktop launchers require the scripts in the checkout and are not standalone installation packages.
Do not copy the raw repository `planet.cmd` to the desktop by itself: first installation needs the adjacent `scripts/windows/` directory. On another computer, generate a new launcher from that computer's checkout instead of reusing an old desktop copy. Server paths such as `/home/linkong/planet` and personal workstation paths both work without machine-specific Git edits. Run under the Windows account that owns the WSL distribution; elevating under a different account may expose a different set of registered distributions. Installation itself does not require administrator privileges; elevation is requested for the actual restart.
Run Windows regression checks with `powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts\harness\test_windows_launcher.ps1`. They cover target resolution, special characters, generated launchers and name collisions. `scripts/harness/test_windows_restart.py` uses disposable command substitutes to check restart arguments, browser opening after success and shell retention after failure. The latter runs in quick-check without starting real services.
## Startup background
Use `zsh ./planet.sh start --non-motion-agent` for daily startup; use `init` when preparing a new environment. When investigating latency, distinguish initial dependency downloads, container readiness, and application initialization using the stage timestamps.
Before preparing the AI Provider image, startup verifies the backend's actual database connection and recreates a missing port mapping once while preserving the volume. A terminated backend process or Uvicorn initialization, ASGI loading, import, or syntax failure stops waiting and identical retries immediately. Normally slow initialization keeps its existing timeout budget. AI Provider readiness probes the host `/health` endpoint directly, without waiting for Docker's first scheduled health check.
`planet.sh` manages start, stop, restart, health checks, and logs for all local services. The previous implementation had several startup issues:
@@ -37,27 +55,9 @@ write_ai_provider_build_stamp() {
}
```
### Faster Fingerprint
### Fingerprint Scope
The previous implementation tarred the whole `aiprovider/` directory before hashing, which could take seconds in large trees. The new version uses `find + stat` and reads only file metadata:
```bash
compute_ai_provider_build_fingerprint() {
find aiprovider \
-type f \
! -path '*/__pycache__/*' \
! -name '.env' \
! -name '.env.*' \
! -name '*.pyc' \
! -name '*.pyo' \
| LC_ALL=C sort \
| xargs -r stat --format="%Y %s %n" 2>/dev/null
sha256sum docker-compose.yml docker-compose.simple.yml 2>/dev/null
python3 "$SCRIPT_DIR/scripts/compute_aiprovider_dependency_fingerprint.py" 2>/dev/null
}
```
This is roughly 10 times faster for many-small-file workloads while preserving the same practical rebuild signal. `.env` and `.env.*` are excluded because runtime model, key, and Base URL changes should not force an image rebuild.
The fingerprint hashes file contents from `aiprovider/`, the Dockerfile, the root manifest and lockfile, and the provider dependency information. It does not traverse frontend assets or downloaded data. `.env` and `.env.*` are excluded because they are runtime configuration. The Dockerfile applies its fingerprint label after dependency installation so a changed build marker alone does not invalidate dependency layers.
### Docker Build Context
@@ -83,16 +83,18 @@ The Dockerfile copies only AI Provider inputs:
```dockerfile
COPY pyproject.toml uv.lock /app/
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev
uv sync --frozen --only-group aiprovider
COPY aiprovider /app/aiprovider
```
`uv sync` uses a BuildKit cache mount. The first build may still depend on network speed, but later builds reuse `/root/.cache/uv`.
The `aiprovider` dependency group in the root `pyproject.toml` uses the same `uv.lock` and installs only the API, HTTP client, settings, and ASGI runtime dependencies. The image excludes backend collectors and OpenCV / MediaPipe motion dependencies. The build fingerprint label comes after dependency installation and code copying, so a fingerprint change alone does not invalidate dependency layers. The container starts the installed `.venv/bin/python` directly, without runtime dependency synchronization. Dependency changes must update this group and the lockfile and validate image imports and `/health`.
### Runtime Configuration
Before starting AI Provider, `planet.sh` generates a current-user runtime env-file and passes it to Compose or the manual `docker run` fallback. The default path is `${XDG_STATE_HOME:-$HOME/.local/state}/planet/aiprovider_runtime.env`. Configuration priority:
Before starting AI Provider, `planet.sh` generates a current-user runtime env-file and passes it to Compose; a failed Compose operation stops with its error instead of falling back to a manual launch. The default path is `${XDG_STATE_HOME:-$HOME/.local/state}/planet/aiprovider_runtime.env`. Configuration priority:
1. `aiprovider/.env`
2. simple `export AI_...=...` or `AI_...=...` lines from `~/.zshrc`
@@ -117,7 +119,7 @@ When the fingerprint matches, the script skips `docker compose build` and starts
docker start planet_aiprovider
```
`docker stop` stops the container without deleting the image. `cleanup_exit_containers` removes exited containers but not images, so the next `docker start` can reuse the existing image.
`docker stop` stops the container without deleting the image. `start` and `restart` retain stopped containers instead of scanning and deleting all exited containers on the host. An unchanged AI Provider can be reused, while Compose still reconciles database configuration. Existing recreation paths remain responsible for image or configuration changes.
## Issue 2: Slow Port Cleanup

View File

@@ -33,9 +33,9 @@ docker buildx version
## Database Initialization and Connection Checks
`init` reconciles PostgreSQL / Redis containers through Compose, including port configuration on existing containers. A plain `docker start` cannot apply configuration changes. Compose failures retain their specific errors, such as an occupied port, instead of falling back to an old container and reporting success.
`init` and `start` reconcile PostgreSQL / Redis containers through Compose, including port configuration on existing containers. A plain `docker start` cannot apply configuration changes. Compose failures retain their specific errors, such as an occupied port, instead of falling back to an old container and reporting success.
The container's `pg_isready` check only establishes that the server accepts connections; it does not validate the host backend's address and credentials. Once containers are healthy, `init` runs `scripts/check_database_connection.py` using the backend's effective `DATABASE_URL`. It checks the local PostgreSQL published port and executes a read-only `SELECT 1` before reporting database readiness or creating tables and seed data.
The container's `pg_isready` check only establishes that the server accepts connections; it does not validate the host backend's address and credentials. Once containers are healthy, both initialization and backend startup run `scripts/check_database_connection.py` using the backend's effective `DATABASE_URL`. It checks the local PostgreSQL published port and executes a read-only `SELECT 1`. Startup performs this check before preparing the AI Provider image and stops immediately on failure. Initialization only creates tables and seed data after the check passes.
- If the actual local port mapping is still missing or mismatched, the script recreates PostgreSQL once from Compose while preserving its data volume, then checks again. A second failure stops initialization.
- Authentication, database-name, and network failures stop before schema changes. Diagnostics show the host, port, and database name without passwords, full connection strings, or raw driver exceptions.
@@ -67,6 +67,8 @@ First startup seeds two default accounts (see `DEFAULT_LOGIN_USERS` in `backend/
Both seed accounts are created with `email_verified = TRUE` and can log into the console immediately. Any other account must either go through the public registration flow described in the Manual, or be created via `./planet.sh createuser`.
An isolated preview database does not receive default accounts when application lifespan is skipped. For a default-account 401, first verify the actual database and account existence. Run the existing `ensure_default_admin_user` against the confirmed database only when needed; it does not reset existing passwords. Verify both `/api/v1/auth/login` and authenticated `/api/v1/auth/me`: identity loading preserves full fields such as `email_verified` and `created_at`, and responses accept existing bootstrap `.local` addresses while registration and email updates stay strict. The AUTH catalog markers below require manually confirmed evidence; generic 401/500 responses alone do not establish these causes.
Specify custom ports:
```bash
@@ -274,6 +276,15 @@ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/"
default = true
```
Start and restart use fingerprint-based build detection by default. To explicitly reuse a local image for one command, add --no-build:
```bash
./planet.sh start --no-build
./planet.sh restart -a --no-build
```
This option affects only AI Provider and keeps other startup checks enabled. A missing local image is an error, and the old image is never stamped as containing current code. When Compose v2 is available, builds, startup and build-capability checks use v2 only and preserve its failure. Compose v1 is considered only when v2 is unavailable.
Diagnose slow builds:
| Symptom | Common cause | Fix |
@@ -291,6 +302,101 @@ When SMTP is unset, `POST /api/v1/auth/register` returns `503 EMAIL_PROVIDER_NOT
One-time codes are stored in Redis under `otp:{purpose}:{email}` with a 600-second TTL. The key is invalidated after 5 invalid attempts. Resend cooldown is 60 seconds, enforced via `otp_rate:{purpose}:{email}`.
## Error Cause Catalog
By default, planet.sh retains one specific error beneath the main status. With --verbose, log_error reads its diagnostic code, cause and remedy from the Chinese table. AI Provider build failures also inspect the full build log. Rows are matched in order using case-insensitive literal fragments separated by semicolons; specific errors precede summaries. Codes are diagnostic identifiers, not process exit codes; failure still returns a nonzero status. Matching fragments below intentionally retain the Chinese shell messages.
For each newly confirmed cause, update both language tables with a stable code, distinguishing evidence and a regression case before integrating it into the script. Unverified failures remain P_UNKNOWN; the script must not append guessed causes to documentation. scripts/lib/error-diagnostics.zsh reads the table. Do not put vertical bars in cells. scripts/harness/test_error_diagnostics.py checks classification, bilingual codes and runtime wording.
<!-- planet-error-catalog:start -->
| Error code | Matching fragments (semicolon-separated) | Cause / established failure | Remedy |
| --- | --- | --- | --- |
| P_WINDOWS_LAUNCHER_TARGET_MISSING | P_WINDOWS_LAUNCHER_TARGET_MISSING | The bound WSL project is unavailable or lacks launcher scripts, commonly after moving a checkout, losing access to a distribution or copying another machine's desktop launcher. | Run the repository planet.cmd from the current checkout to regenerate this machine's desktop launcher and verify the WSL distribution; do not hardcode another machine's path. |
| P_PROXY_EXTERNAL | PLANET_PROXY_EXTERNAL | Docker proxy settings belong to another configuration source or have been edited manually. | Inspect daemon.json, systemd proxy settings and the Planet ownership record before changing ownership; existing settings are preserved. |
| P_AUTH_BOOTSTRAP_MISSING | DEFAULT_LOGIN_USER_MISSING | The active backend database is confirmed to lack default accounts, often in an isolated preview that skipped initialization. | Verify database ownership and run the existing account initializer without resetting existing passwords or clearing data. |
| P_AUTH_PROFILE_INVALID | AUTH_IDENTITY_RESPONSE_INVALID | Login succeeds but the identity response lacks required fields or incorrectly rejects a stored local email address. | Load the complete persisted identity, separate response serialization from registration validation, and run default-account identity regressions. |
| P_EARTH_POLICY_MISSING | EARTH_DISPLAY_POLICY_MISSING | The backend image or deployment directory omitted the shared geography policy required when compute routes import. | Include config/earth-boundary-pov-policy.china-v1.json, the boundary builders and reviewed geometry through both Dockerfile and .dockerignore; rebuild the image without resetting the database. |
| P_PROXY_CHANGED | PLANET_PROXY_CONFIG_CHANGED | Docker proxy settings changed after detection. | Wait for concurrent configuration work to finish and retry without overwriting it. |
| P_PROXY_ROLLBACK | PLANET_PROXY_ROLLBACK_FAILED | Docker or previously running containers could not be restored after a failed proxy update. | Inspect daemon and container logs; the original configuration was restored. Use daemon.json.planet-proxy.bak for manual recovery if necessary. |
| P_PROXY_CONFIG | PLANET_PROXY_CONFIG_FAILED;Docker 构建代理检测失败;Docker 构建代理更新失败 | Automatic proxy detection, validation or service update failed. | Check Python 3, curl, Docker and sudo access; updates attempt rollback and proxy credentials are excluded from error output. |
| P_PROXY_NO_ROUTE | PLANET_PROXY_NO_ROUTE | No configured host proxy could reach the required build registries, and direct probes also failed. | Restore a working proxy or repair direct networking, DNS and registry addresses; disabling builds does not repair connectivity. |
| P_REGISTRY_RATE_LIMIT | 429 Too Many Requests;toomanyrequests;pull rate limit | The registry responded but imposed a request-rate or image-pull quota limit; consult the response for the specific limit. | Avoid repeated retries and follow upstream retry guidance. For anonymous pull quotas, check docker login identity and allowance; do not mistake throttling for an unreachable proxy. |
| P_DNS | no such host;temporary failure in name resolution;could not resolve host | Name resolution failed; the log alone does not identify the DNS configuration or upstream fault. | Check DNS and proxy resolution in the Docker environment; compare shell and daemon resolution. |
| P_TLS_CERT | x509:;certificate verify failed;certificate signed by unknown authority | TLS certificate validation failed. | Check time, certificate chain and proxy CA; install trusted certificates without disabling verification. |
| P_PROXY_AUTH | proxy authentication required;407 proxy | The proxy requires authentication and rejected the request. | Check daemon proxy credentials and permissions; keep credentials out of the repository and logs. |
| P_NETWORK_TIMEOUT | i/o timeout;tls handshake timeout;context deadline exceeded;deadlineexceeded | A connection or TLS handshake timed out; the log alone cannot distinguish proxy, DNS and IPv6 faults. | Compare direct and proxied requests; check daemon proxy settings, DNS and IPv6 routes. Shell proxy settings do not configure the daemon. |
| P_CONNECTION_REFUSED | connection refused | The destination refused the connection; its listener, address or port may be wrong. | Identify whether the target is a proxy, database or registry, then check its listener and service state. |
| P_REGISTRY_AUTH | pull access denied;unauthorized:;insufficient_scope;denied: requested access | Registry access was denied or the current identity cannot pull the image. | Verify the image name, repository permissions and docker login identity. |
| P_IMAGE_TAG | manifest unknown;manifest not found | The registry cannot find the requested manifest or tag. | Check PYTHON_IMAGE, UV_IMAGE and other tags, including architecture support. |
| P_DISK_FULL | no space left on device | Disk space or inodes are exhausted. | Check df -h, df -i and docker system df; remove only confirmed disposable data, never database volumes as a troubleshooting shortcut. |
| P_DOCKER_SOCKET | permission denied while trying to connect;刷新组权限后仍无法访问 Docker socket | The current user cannot access the Docker socket. | Check socket ownership and docker group membership; run ./planet.sh init for permissions and open a new terminal. |
| P_DOCKER_SERVICE | 没有可用的 docker.service;Docker Engine 启动失败;cannot connect to the docker daemon;无法连接 daemon | Docker is not ready or the client cannot reach the current endpoint. | Check systemctl status docker, docker context ls and journalctl -u docker.service; connection failure does not prove Docker is absent. |
| P_DOCKER_DESKTOP | 检测到 Docker Desktop,但当前 WSL | Docker Desktop or its WSL integration is unavailable. | Start Docker Desktop and enable WSL Integration for this distribution. |
| P_DOCKER_ENDPOINT | 当前 Docker 使用其他 context 或远程/rootless endpoint | The selected remote or rootless Docker endpoint is unavailable. | Check docker context ls, DOCKER_HOST and the target service; do not replace it with a local engine automatically. |
| P_BUILDX | 未检测到 docker buildx;buildx 0.17;buildx >=;buildx v;当前 docker compose 不支持 build;安装后 Docker CLI、Compose v2 | Docker build plugins are missing, too old or unable to provide build support. | Check docker buildx version and docker compose version; install or upgrade the plugins required by the project. |
| P_COMPOSE_MISSING | 未检测到可用的 Docker Compose | No usable Compose command was found. | Install the Compose v2 plugin and verify docker compose version; a v2 operation failure must not trigger v1 fallback. |
| P_LOCAL_IMAGE_MISSING | 已指定 --no-build,但本地没有 AI Provider 镜像 | Builds were disabled but the AI Provider image is not present locally. | Build or import the image before using --no-build; the option never builds automatically. |
| P_SUDO | 缺少 sudo | The privilege escalation tool required for dependency setup is unavailable. | Ask an administrator to install and authorize sudo, or provision the dependencies in advance. |
| P_UNSUPPORTED_OS | Docker 自动安装目前支持;未识别系统包管理器 | The current system is outside the automatic installation support scope. | Install dependencies through supported system procedures and retry. |
| P_LOCKFILE_CHANGED | 修改了 uv.lock | Dependency preparation unexpectedly changed the lockfile. | Check manifest and lockfile consistency; use frozen installation without implicit lockfile updates. |
| P_DEPENDENCIES | 安装失败;安装后仍不可用;安装完成后仍未找到;未找到 .venv/bin/python;自动安装后仍无法解析运行时;未找到 Vite Bun 入口;缺少 mediapipe/opencv-python;仍无法导入 mediapipe/opencv-python;需要 openssl | A required dependency failed to install, is missing or is unavailable in the active environment. | Inspect the installer log, network, package sources and PATH; use Bun for frontend and the project uv environment for Python. |
| P_ARGUMENT | 未知参数;非法端口;需要端口号;需要逗号分隔;--motion-agent-mode 需要;--motion-agent-wsl-usbipd-busid 需要;用法: ./planet.sh | The command or an argument does not match the supported format. | Check ./planet.sh usage and correct the arguments before retrying. |
| P_DB_PORT_OCCUPIED | PLANET_DB_PORT_OCCUPIED | Another container or host service occupies the Planet database or Redis port. | Identify the owner with docker ps; update the DATABASE_URL port or REDIS_PORT in backend/.env (process REDIS_URL takes precedence). planet.sh synchronizes Compose mappings without stopping other projects or deleting volumes. |
| P_PORT | 地址已被占用;端口仍不可用;清理失败,请检查占用进程;port is already allocated;address already in use | The requested port is occupied or cannot be bound in the host environment. | Check ss and Windows Get-NetTCPConnection; identify the owner before changing ports or stopping the service. |
| P_CAMERA | live 模式缺少可用摄像头;未找到可打开并能读帧的摄像头 | Motion Agent cannot capture frames from a usable camera. | Check hardware, permissions and WSL USB forwarding; use --non-motion-agent or explicit dry-run when live capture is not needed. |
| P_DB_CONNECTION | 后端数据库连接检查失败 | The backend database connection or published-port check failed. | Inspect the probe output and verify DATABASE_URL, credentials, database name and port; container health alone is insufficient. |
| P_DB_START | 数据库启动失败;数据库重启失败;PostgreSQL 启动失败;数据库初始化失败 | Database startup, health checks or initialization did not complete. | Inspect PostgreSQL and Redis logs and the specific database error; do not troubleshoot by deleting volumes. |
| P_BACKEND_START | 后端进程已退出;后端启动失败 | The backend exited or did not pass its health check. | Use ./planet.sh log -b and resolve import, configuration, database or application initialization errors first. |
| P_AI_START | AI Provider 启动失败 | AI Provider did not start or pass its health check. | Use ./planet.sh log -a and check runtime configuration, ports and container exit details. |
| P_FRONTEND_START | 前端启动失败 | The frontend did not pass its startup health check. | Use ./planet.sh log -f and check Bun, dependencies, the Vite entry point and ports. |
| P_MOTION_START | Motion Agent 启动失败 | Motion Agent failed to start. | Use ./planet.sh log -m and check dependencies, cameras and input mode. |
| P_ACCOUNT_INPUT | 用户名不能为空;密码不能为空;密码长度不能少于;两次输入的密码不一致 | User creation input failed validation. | Supply a username and matching passwords that satisfy the minimum length. |
| P_HTTP_HEALTH | 不可访问: | The specified HTTP endpoint failed its access check. | Check the URL, listener, firewall and local or LAN routing; check certificate trust for HTTPS. |
| P_DOCKER_MIRROR_FAILED | PLANET_DOCKER_MIRROR_FAILED | Mirror manifest validation or image pulling failed, so fallback could not complete. | Inspect the specific registry error, mirror availability, allowlist, architecture and digest. A reachable homepage is insufficient. Configure PLANET_DOCKER_MIRROR_PREFIX or disable fallback while repairing the origin. |
| P_DOCKER_COMMAND_TIMEOUT | PLANET_DOCKER_COMMAND_TIMEOUT | The current Docker command exceeded the script deadline; timeout alone does not establish a proxy, network or local engine fault. | Locate the stalled stage using its label, latest output and log. For silent commands, inspect that Docker command and daemon logs; increase the deadline only after confirming progress. |
| P_COMPOSE_FAILED | Docker Compose 执行失败;docker-compose v1 执行失败 | The selected Compose command failed without a more specific classified cause. | Inspect the preceding original error; repair an installed Compose v2 instead of installing v1 as a fallback. |
| P_BUILD_FAILED | AI Provider 镜像构建失败;failed to solve | Image build failed without evidence matching a known specific cause. | Inspect the earliest specific error in aiprovider_build.log and add the confirmed cause and a regression case to this catalog. |
| P_UNKNOWN | — | Unclassified; the available evidence does not establish the cause. | Retain the error and command; after verifying the root cause, update both language catalogs and add a regression case. |
<!-- planet-error-catalog:end -->
### Docker Progress and Deadlines
Docker probes, Compose startup, database connection checks and AI Provider builds update the secondary line beneath the existing main status. There are no extra start/completion records, version numbers or log paths. Success clears the secondary line. Silent waits retain the current step and update elapsed time. Failure leaves one specific error and exits; later summaries and cleanup notices do not overwrite it. Failed Compose operations do not retry; health checks after a successful container launch retain their existing budgets.
`PLANET_DOCKER_PROBE_TIMEOUT`, `PLANET_COMPOSE_TIMEOUT` and `PLANET_DOCKER_BUILD_TIMEOUT` default to 15, 180 and 900 seconds respectively. Each accepts positive seconds and bounds one command. Timeout terminates that CLI process group; the daemon may already have accepted a request, so inspect actual container state. A Compose probe timeout does not fall back to v1.
Detailed logs remain under `${PLANET_STATE_DIR:-${XDG_STATE_HOME:-$HOME/.local/state}/planet}`: `compose.*.log`, `aiprovider_build.log` and `database-check.*.log`. Logs use mode 600 and mask proxy usernames/passwords in URLs. `scripts/docker_command.py` owns recording, timing and process cleanup; `planet.sh` owns terminal rendering. Invoking system `docker compose` directly does not load this wrapper.
### Database Troubleshooting Before Demos
A confirmed failure occurs when another project's PostgreSQL and Redis occupy `5432` and `6379`. Planet containers may appear healthy without published host ports, and the backend can accidentally reach the other database and fail authentication. `scripts/check_database_connection.py --ports` identifies conflicts before startup and stops instead of repeatedly recreating containers.
For coexistence, change the port in `backend/.env`'s `DATABASE_URL` to an available port such as `15432`, and `REDIS_PORT` to an available port such as `16379`. Preserve credentials and the database name. If the process exports `REDIS_URL`, update it too. `planet.sh` derives `PLANET_POSTGRES_PORT` and `PLANET_REDIS_PORT` from effective backend settings and preserves named volumes when recreating port mappings. Direct Compose invocations must supply these variables explicitly. Database ports bind only to `127.0.0.1`.
Handle other errors according to evidence: authentication failures require checking the target and existing volume credentials; changing `POSTGRES_PASSWORD` does not reset an existing password. Missing published ports without a conflict allow one recreation preserving volumes. DNS, TLS, proxy authentication and registry rate limits have separate catalog remedies. Slow downloads or dependency installation are not failures; unchanged code and dependencies should reuse the image on subsequent starts.
### Temporary Docker Image Mirrors
The Tsinghua [Docker CE repository](https://mirrors.tuna.tsinghua.edu.cn/help/docker-ce/) distributes Docker installation packages, not Docker Hub images. Image fallback defaults to the [DaoCloud public image mirror](https://github.com/DaoCloud/public-image-mirror). Tsinghua PyPI fallback remains specific to local Python dependencies.
Build preparation tries the origin through proxy and direct routes, then the mirror route when neither works. A network failure during an actual registry request can also trigger one fallback after a successful origin probe. Authentication, certificate, rate-limit, port and Dockerfile package-installation failures do not switch image sources. Exhausted routes leave the specific error in the secondary line and exit. Original failures remain in adjacent `*.primary.log` files.
The specific mirror image must pass manifest inspection; execution pins its returned sha256 digest. An explicitly requested digest must match. Base images are overridden only through this build's arguments; original configuration, fingerprint and output image name remain unchanged. Database fallback pulls only missing images for requested services, tags them with the original local names, then retries the original Compose operation. Global registry-mirrors, source configuration and named volumes remain unchanged.
`PLANET_DOCKER_MIRROR_FALLBACK=0` disables fallback. `PLANET_DOCKER_MIRROR_PREFIX` defaults to `m.daocloud.io` and accepts a trusted registry/path prefix. `PLANET_DOCKER_MIRROR_TIMEOUT` defaults to 120 seconds for base-image verification. Only Planet's public Python, uv, PostgreSQL and Redis repositories are mapped; private/custom images are not sent to public mirrors. Public services have allowlists, quotas and cache delays, so mutable tags can lag. Use digests when strict version identity matters and build/cache images before demos.
`scripts/docker_mirror.py` owns mapping, manifest checks and database pulls; `scripts/lib/docker-mirror.zsh` owns network-error selection and temporary arguments. Both reuse the existing progress/deadline runner. Regressions cover private repositories, digest mismatches, non-network errors, a single fallback and preservation of original configuration.
### Docker Daemon Proxy and Build Networking
Shell HTTP_PROXY / HTTPS_PROXY settings do not automatically configure a running Docker daemon. When the shell can reach a registry through its proxy but Docker pulls time out, compare direct requests, proxied requests and daemon settings. HTTP 401, 403 and 429 establish a registry response, not pull authorization or remaining quota. Verify authentication and rate limits with the actual build; these responses must not cause a reachable proxy to be disabled.
Before an actual AI Provider build, the script reads HTTPS_PROXY, HTTP_PROXY and ALL_PROXY, including lowercase forms. It validates HTTP/HTTPS candidates against the registries selected by PYTHON_IMAGE and UV_IMAGE, respecting NO_PROXY. A working proxy is configured for the local Linux Docker Engine. Missing or unusable proxies cause a direct probe and removal of stale Planet-managed proxy settings. If neither origin route works, the mirror route is checked; P_PROXY_NO_ROUTE stops the build only when that also fails. A host without a proxy and a daemon already using direct access receives no proxy configuration. Fingerprint cache hits and --no-build skip probing. This is not a background monitor: proxy availability is checked on the next actual build.
scripts/docker_proxy.py manages the proxies section of /etc/docker/daemon.json and a root-only ownership record at /etc/docker/planet-proxy-state.json. Administrator settings, systemd proxy settings and manually edited proxies are not overwritten. Unchanged settings need neither elevation nor a restart. Updates use the existing sudo flow, preserve unrelated Docker settings, save daemon.json.planet-proxy.bak, validate configuration, restart Docker and start previously running containers. Failures trigger rollback; check container health after recovery. Credentials travel through environment variables or restricted files and are excluded from logs. Docker Desktop, remote and rootless daemons retain their own settings without local daemon.json changes. Proxy addresses come from the environment, never a machine-specific port in the repository.
The AI Provider Dockerfile uses the bundled BuildKit frontend to avoid a separate docker/dockerfile image fetch. Python and uv base images and package downloads still require network access. --no-build explicitly reuses a local image; it does not repair build networking. Build errors are recorded in ${XDG_STATE_HOME:-$HOME/.local/state}/planet/aiprovider_build.log. Verify restored services with ./planet.sh health.
## Troubleshooting Order
```bash
@@ -372,3 +478,13 @@ uv run pytest backend/tests/test_otp_service.py
- [System Service Control](/home/ray/dev/linkong/planet/docs/technical/en/backend-system-service-control.md)
- [Docker + Compose + Buildx Upgrade](/home/ray/dev/linkong/planet/docs/technical/en/ops-docker-compose-buildx-upgrade.md)
- [Data Collectors](/home/ray/dev/linkong/planet/docs/technical/en/backend-collectors.md)
## Shipping the selectable Earth basemaps
These changes from 0.74.7 add no database migration or dependency-lock update. Restart the existing Bun development services; static/container deployments must rebuild the frontend and relevant images rather than keep an old dist after pulling. New modules, four runtime maps, flags, policy fallback geometry and the shared policy must be delivered together. Committing only already-tracked files omits required new assets.
The old `frontend/public/earth/assets/8k_earth_daymap.jpg` path remains but now contains Natural Earth I. The exact 0.74.7 image is retained at `frontend/public/earth/assets/maps/archived-original-20260921-8k.jpg`, exposed as Original basemap. Pull/start does not delete the archive. Settings without a map-source field use the Original basemap default. `resources/earth/candidates` contains provenance/reference originals and is not a frontend runtime dependency.
High-precision boundary outputs live in a Git-ignored generated directory. Pulling does not deliver them automatically. Outputs lacking the required new policy metadata are rejected in favor of the shipped policy fallback; regenerate and publish the static artifacts through the existing boundary workflow to retain high precision. This fallback does not reduce basemap resolution.
Run `PLANET_HARNESS_DOCKER_SMOKE=1 .venv/bin/python scripts/harness/test_backend_runtime_assets.py` to exercise the actual Docker context and production COPY rules, verify the shared policy/geometry and builder imports, and exclude environment files. This probe does not start the full application stack.

View File

@@ -33,7 +33,7 @@ The default role is `viewer`: you can sign in but only see public pages. For col
After landing on the `/admin` dashboard, here's a recommended walk-through:
1. `/collection-management?section=collector_credentials`: pick a collector and click the plug icon to test connectivity. Free collectors (e.g. open BGP) usually work right away; credential-bearing ones like `AISStream` or `BarentsWatch` need an API key / client secret first
2. `/ai?section=integrations`: fill an LLM provider (e.g. `minimax` / `openai`), model, base URL, API key, and click the plug at the end of the base URL to test. WebSearch / OCR tools are optional
2. `/ai?section=integrations`: fill an LLM provider (e.g. `minimax` / `openai`), model, base URL, API key, and click the plug at the end of the base URL to check the model catalog; verify generation in Playground. The refresh icon updates and saves the available model catalog while preserving your form; select a model and click Save to apply it. WebSearch / OCR tools are optional
3. `/earth-content?section=brand`: maintain the Earth HUD logo and title image in Branding. The Upload button inside each URL field opens a file picker, and image files can also be dropped directly onto the matching field. Save the brand configuration after upload
4. `/datasources` or `/data`: check whether collectors have produced data. Use `/datasources -> Built-in Sources` for finite collectors: with no rows selected, click `Trigger All`; after selecting rows, the primary button becomes `Trigger Selected N`. The top-right queue button shows progress. Use `/datasources -> Realtime Sources` for AISStream / WebSocket health and counters
5. `/alerts/system`: verify system alerts look right
@@ -50,8 +50,12 @@ Once in, verify:
- The globe renders, and the right-side layer panel can toggle layers
- Search finds cables, satellites, compute centers, BGP events
- Compute-center and BGP collector detail cards can collect coordinate candidates and preview them on Earth
- Close the candidate panel or switch browser tabs during batch location, then return to see progress; refreshing or leaving Earth interrupts unfinished work
- The vessel layer receives ongoing AISStream / BarentsWatch positions and reconciles after reconnecting
- The live-stream menu searches the complete channel catalog and loads more on scroll; Al Jazeera Mubasher is the default
- Mouse drag, wheel zoom, and the zoom percentage indicator work
- The settings panel can switch rotate / cruise / motion modes; motion settings can select the input source and allowed gestures; view settings can switch hover tooltip content, and satellite settings can toggle real-altitude layering and track display
- The base map defaults to Original basemap, listed first; the display menu can switch and save the map source. Natural Earth I remains available, NASA 21K is source archive material only, and runtime uses one 8K texture at a time.
## 5. Recover a Lost Password
@@ -67,3 +71,9 @@ Open `/forgot-password`, enter your email, receive a code, then enter the code p
- Troubleshooting and configuration questions: [FAQ](/home/ray/dev/linkong/planet/docs/technical/en/faq.md)
- Detailed Earth coordinate candidate flow: see the Earth section in [Intelligent Planet Manual](/home/ray/dev/linkong/planet/docs/technical/en/manual.md)
- Deployment / operations commands: [Intelligent Planet Ops Runbook](/home/ray/dev/linkong/planet/docs/technical/en/ops-runbook.md)
## Try compute intelligence
To maintain card values, an administrator opens **Earth content → Compute comparison**, adds report sources, enters comparable annual country observations, then chooses **Save and publish**. Omit missing countries; facility counts never replace compute shares. Bundled defaults can be restored and Earth cards automatically check published configuration.
Open the analysis icon beside Compute Centers and enable Show region cards. Closing configuration keeps the cards active; the switch controls their visibility. Resize or collapse desktop cards and inspect observed history, sources, and facility details. On mobile, use the Compute drawer tab. National indicators and facility samples are separate. “—” means missing evidence, not zero capacity.

View File

@@ -51,6 +51,7 @@
支持的请求适配器:
- `openai-completions`
- `openai-responses`
- `anthropic-messages`
- `ollama-generate`
@@ -95,12 +96,37 @@ AI 配置页使用的接口:
- `POST /api/v1/settings/integrations/ai-provider/connect`
- `GET /api/v1/settings/integrations/ai-provider/secrets`
- `GET /api/v1/settings/integrations/ai-provider/presets`
- `POST /api/v1/settings/integrations/ai-provider/presets/{provider}/refresh`
- `GET /api/v1/settings/ai-prompts`
- `PUT /api/v1/settings/ai-prompts/{task_key}`
- `POST /api/v1/settings/ai-prompts/{task_key}/reset`
这些接口都需要用户登录。`secrets` 接口只用于配置页点击显示 key/token 时取回明文,隐藏时前端恢复为脱敏预览。
模型目录刷新和轻量连通性测试共用 `backend/app/services/llm_model_catalog.py`,直接请求当前供应商的模型接口,不再依赖 models.dev。使用当前表单草稿中的基础地址、协议和凭证;不传草稿时读取已保存配置。国内、国际和自定义网关地址保持各自的地域与路径。缺少必需凭证时返回 400 并提示配置 API Key;上游超时、鉴权或响应错误返回安全提示,并保留上次目录。
成功目录和 `refreshed_at` 按供应商保存到 `system_settings` 的 `llm_provider_preset:<provider>` 分类。该动态分类使用对应供应商预设作为默认值,提交前校验分类,避免已提交却返回失败。列表优先返回已保存目录,未刷新过的供应商显示内置建议。刷新不修改 `external_integrations` 的当前模型、协议、地址或凭证;页面保留草稿,目录加载失败时保留已显示模型并明确提示失败。
### 模型目录接口
以下接口于 2026-09-13 对照官方文档核对。表中的 URL 是模型列表地址,配置表单仍填写生成接口的基础地址。需要凭证的目录必须使用对应服务地域的 API Key。
| 供应商 | 模型列表 GET 接口 | 鉴权与解析 |
| --- | --- | --- |
| [MiniMax](https://platform.minimax.io/docs/api-reference/models/anthropic/list-models) | `https://api.minimaxi.com/anthropic/v1/models`;国际站使用 `api.minimax.io` | `x-api-key`;`data[].id`;处理 `has_more/last_id` |
| [OpenAI](https://developers.openai.com/api/reference/resources/models/methods/list) | `https://api.openai.com/v1/models` | Bearer;`data[].id` |
| [Anthropic](https://platform.claude.com/docs/en/api/models/list) | `https://api.anthropic.com/v1/models` | `x-api-key`、`anthropic-version`;按 `after_id` 翻页 |
| [DeepSeek](https://api-docs.deepseek.com/api/list-models) | `https://api.deepseek.com/v1/models` | Bearer;`data[].id` |
| [阿里百炼](https://help.aliyun.com/zh/model-studio/list-models) | 同地域主机的 `/api/v1/models` | Bearer;`output.models[].model`;按 `page_no/page_size/output.total` 翻页,筛选 `capabilities=TG` |
| [Moonshot / Kimi](https://platform.kimi.ai/docs/api/list-models) | `https://api.moonshot.ai/v1/models`;国内站使用 `api.moonshot.cn` | Bearer;`data[].id` |
| [OpenRouter](https://openrouter.ai/docs/api/api-reference/models/list-all-models-and-their-properties) | `https://openrouter.ai/api/v1/models` | 支持公共目录;有凭证时附带 Bearer;`data[].id` |
| [OpenCode Go](https://opencode.ai/docs/go/#models) | `https://opencode.ai/zen/go/v1/models` | 支持公共目录;有凭证时附带 Bearer;`data[].id` |
| [Ollama](https://docs.ollama.com/api/tags) | 本机或配置服务器的 `/api/tags` | 本地无需 Key;`models[].model/name`;空目录表示尚未安装模型 |
百炼的北京、东京、法兰克福、弗吉尼亚新接口要求实际业务空间主机,例如 `<WorkspaceId>.cn-beijing.maas.aliyuncs.com`;新加坡使用 `dashscope-intl.aliyuncs.com`,香港使用 `cn-hongkong.dashscope.aliyuncs.com`。系统只替换目录路径,不猜测业务空间或跨地域切换凭证。旧北京域名若不再接受账号凭证,应按控制台给出的业务空间地址更新基础地址。
完整分页和重试受 30 秒总超时保护;网络故障及 502/503/504 最多尝试两次,鉴权失败不重试。按接口提供的创建/发布日期倒序排列;日期相同时保留上游顺序,不截断为固定数量。公共目录只表示供应商公开的模型集合,不代表账号已获得每个模型的生成权限。
Admin 的 AI 页面按业务信息架构组织为:
- `模型供应商`
@@ -283,22 +309,11 @@ Admin 的密钥状态必须按 provider / tool 精确判断:
### 轻量连通性测试
Admin 的插头按钮走轻量连通性测试,不承担保存职责。业界常见做法是分两层:
- 快速检查 provider 目录或低成本 endpoint,确认 base URL、鉴权和当前模型是否可达。
- 只有在用户明确运行 Playground 或业务任务时才发完整模型请求。
因此,连通性测试应尽量使用低成本请求,并返回明确状态:
- `ok`: 鉴权、路由和模型目录可用。
- `warning`: 服务可达,但当前模型不在目录或能力声明不完整。
- `error`: 鉴权失败、网络失败、协议错误或模型不可用。
错误 toast 标题必须和结果一致,不能在失败时显示“连通性正常”。
Admin 插头按钮检查代理服务配置,再查询当前供应商目录;只有目录请求成功且包含所选模型时才返回 `success=true`。404、鉴权失败、无效响应和缺失模型都不会因命中内置建议而被改判成功。提示明确区分“找到模型”和“实际生成成功”;完整调用由 Playground 或业务任务验证。测试不保存表单或切换默认供应商。
### OpenCode Go 路由模型
OpenCode Go 这类订阅通道不要靠前端硬编码模型集合判断协议。推荐在 provider catalog 或后端能力发现中记录每个模型的协议能力,例如 `chat_completions`、`anthropic_messages`、`models_endpoint` 和是否需要订阅 key。前端只展示能力结果;后端负责把 provider、base URL、model 和协议适配映射为实际请求。
OpenCode Go 的模型协议映射由后端目录集中维护:MiniMax M3/M2.7/M2.5 和已核对的 Qwen3.6/3.7/3.8 模型使用 Anthropic Messages;GPT-5.6 Luna、Grok 4.6 和 Muse Spark Contributor 使用 Responses;其余已支持模型使用 Chat Completions。官方 `/models` 当前只提供模型 ID,新增模型的协议仍需对照官方端点表核对。前端选择模型时同步对应协议,后端覆盖旧的已知错误映射。Responses 请求使用 `input`、`max_output_tokens` 和 `store=false`;回复解析文本及推理摘要。OpenCode 请求携带应用 User-Agent 和会话标识;Playground 同一会话保持标识稳定。MiniMax M3 的通用 `thinking=enabled` 转为官方 `adaptive` 格式。
#### 配置页行为

View File

@@ -100,7 +100,13 @@ AIS 船只类采集器和其它 `CollectedData` 采集器的落库路径不同
智能星球国界不再属于采集器体系。它是智能星球静态渲染资产,由控制台 `运维与配置 -> 智能星球内容 -> 国界精度` 维护源配置,并由 `/api/v1/earth/boundaries/*` 构建 `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`。本地没有高精 PMTiles 时,前端会使用仓库内置的低精度 GeoJSON 作为 fallback,不会向 `CollectedData` 写入国界记录。
TOP500 和 Epoch AI 算力数据的公开源不总是提供可用经纬度。Earth 统一算力中心接口在主地图启动链路中只使用源数据自带坐标或 `compute_center_locations` 维表坐标;缺少坐标的记录会进入 `unresolved`,不会通过本地注册表、国家质心或猜测城市自动渲染。用户手动采集候选时,后端会用源字段调用 ROR 组织注册 API 和 Nominatim/OpenStreetMap 在线搜索;候选经前端保存后写入 `compute_center_locations`,后续地图刷新再从维表渲染。
TOP500 和 Epoch AI 算力数据的公开源不总是提供可用经纬度。Epoch 官方 CSV 的 `Location` 可能只有国家或为空,即使行中存在经纬度,也不能仅凭它当作精确 site。Earth 统一算力中心接口在主地图启动链路中只使用有效源坐标或 `compute_center_locations` 维表坐标;不可渲染记录保留在完整 `unresolved` 列表,不通过本地注册表、国家中心或猜测总部自动渲染。`precise`/`site` 保持源坐标;城市级和待确认点只可按 `source` + `source_id` 生成确定性的展示偏移,原坐标不改写。用户手动采集候选时,后端会用源字段调用 ROR 组织注册 API 和 Nominatim/OpenStreetMap 在线搜索;候选经前端保存后写入 `compute_center_locations`,后续地图刷新再从维表渲染。
### 新闻直播频道目录
`news_live_streams` 的 IPTV-org 适配器保留现有新闻分类筛选,`max_sources` 默认值为 `0`,表示不截断匹配频道;显式正数仍限制采集数量。已有配置若保留旧的 `120`,需要改为 `0` 并重新采集才能补齐目录。
`GET /api/v1/tv/streams` 由 `tv_catalog.py` 提供数据库分页:`offset` 默认 `0`,`limit` 默认 `50`、最大 `100`,`q` 按空白拆词并匹配频道名、来源、地区和语言。内置及配置源优先,采集源按频道标识去重并排除配置覆盖项,再在数据库中搜索、计数、排序和分页,避免全表读取后切片。响应的 `total` 是匹配总数,`source_count` 是完整可用目录数量;后续页使用 `next_offset` 和 `has_more`。`selected_id` 可额外取得不在当前页的已选频道,不占本页配额。默认和兜底源继续由 `tv_streams.py` 统一管理。
## 四、数据格式 (统一存储到 CollectedData 表)
@@ -382,7 +388,11 @@ GET /api/v1/visualization/vessels/{mmsi}/conflicts
`/api/v1/vessels/snapshot` 必须携带 `bbox` 和 `zoom`,后端最大 `limit=5000`。Earth 前端使用全球 bbox 读取当前状态,不随相机视口变化反复请求。接口消费 `vessel_current_state`,并在 `diagnostics.source` 返回 `vessel_current_state`;旧 `/api/v1/visualization/geo/vessels` 路由已移除。
高频 AIS 更新不要直接推送成每条 delta 的整层重建。`/ws` 的 `vessels` channel 如用于 Earth,应广播低频 reload/dirty 提示,由前端合并刷新 snapshot;轨迹和冲突详情仍按单船接口读取历史事实。
AISStream 与 BarentsWatch 共用 `/ws` 的 `vessels` 增量通道。Earth 使用现有 WebSocket 连接订阅 `scope: "global"`,不跟随镜头改变订阅范围。后端按 MMSI 合并一秒内的通知,再读取 `vessel_current_state` 的已确认状态;单帧最多 1000 项,超过时继续发送后续帧,不截断不同船只。原始源消息不能直接覆盖客户端的确认位置。当前状态删除也会产生按 MMSI 的 remove 通知。
前端将短时间内同一 MMSI 的变更合并为最新值,通过 `Interactable.updateItems()` 原位修改位置和颜色缓冲。航向分桶变更只更新受影响的桶,容量不足才扩容;保留已有 marker 身份、锁定状态和材质。首次进入、重连、删除提示及每分钟巡检仍用 snapshot 校准,但不先清空整个图层。收到有数量上限的快照时,不能把被截断的船只当成删除;快照返回期间到达的较新更新和删除也不能被旧响应覆盖。快照携带查询开始时的 `generated_at`,与实时帧时间一起用于识别缓存旧快照,避免新船或删除状态被回滚。
`earth_updates` 中船舶普通写入的策略是 `delta`;专用通道连通时不再触发整层重拉。删除或断连后的校准使用 `reload`,并复用现有对象。关闭图层会取消船舶订阅并清空待应用变更。轨迹、冲突详情和审计继续按单船接口读取历史事实。
### 图层接口与全量统计分离
@@ -481,3 +491,15 @@ POST /api/v1/datasources/trigger-batch
请求体可以传 `source_ids` 精确触发选中项;如果不传 `source_ids`,后端按 `product`、`module`、`is_active`、`run_status`、`collected`、`credential_status` 和 `q` 过滤后触发。接口会跳过禁用源、正在运行且未 `force` 的源,以及未到频率窗口的源,并返回 `triggered`、`skipped`、`failed` 三组结果。
**核心文件**: `backend/app/api/v1/datasources.py`
## 算力来源合同与专题接口
Epoch 改用官方公开 CSV,保留 FP16 理论性能、原始日期、状态、匿名/取整标记与扩建关系。匿名记录以内容指纹和同值序号保留快照内的多条记录,不声称跨快照识别设施,不参与定位。未知国家保留缺失标记;它们不进入已选国家的统计。TOP500 以系统详情 ID 标识对象,榜期从榜单 URL 读取,性能按表头或单元格单位转换为 FLOP/s 并保留 FP64 LINPACK 口径。公开页面可能只提供部分榜单,不能宣称已取得完整 TOP500。
两个采集器不再生成示例回退。空原文、转换后空结果、结构错误或相对上次快照减少超过 40% 会失败并保留旧快照;写入失败回滚不完整记录。TOP500 身份切换仅在旧位置名称、规范国家完全一致且唯一时复制位置到新系统 ID,旧位置保留用于历史查询,绝不按名次迁移。
`compute_history_defaults.json` 提供原有报告数据,`compute_configuration.py` 从 `system_settings` 的 `compute_comparison` 分类读取完整数据库覆盖;没有记录时使用默认值,不需要迁移或人工初始化。`compute_history.py` 只处理同年聚合,参考期与发表时间分开;未知具体发表日保持未提供。`/api/v1/data-products/compute/series` 支持全球份额、固定中美分母的双边份额及 FP32 绝对量,逐次读取配置并返回 `Cache-Control: no-store`。创新分差独立返回,不反推单国指数。空配置保留缺失,不回填默认观测。
同前缀的 `GET/PUT /configuration` 和 `POST /configuration/reset` 仅允许 admin/super_admin。写入体为 `{revision, config}`,重置只需要 `revision`。Pydantic 校验国家目录、有限数值、比例总和、重复指标年份和报告引用;每个指标年份只允许一个报告口径。PostgreSQL 唯一分类与行锁串行化首次写入和更新,旧版本返回 409。审计记录和配置在同一事务提交;重置移除覆盖,下次读取随版本默认值。`version` 为配置修订标识,`reviewed_at` 为录入的核验日期,不能将管理员配置一概称为官方报告。来源链接只作引用,不触发抓取。
`/api/v1/data-products/compute/facilities` 仅读取 `compute_contract=v1` 的成功快照,数据库执行国家过滤和按来源计数。Epoch 默认排除规划、退役、已被替代和疑似重复记录;TOP500 表示榜单观测,跨源仍未合并。历史选择按采集时间截止,不将最近抓到的旧资料冒充过去已知的快照。地理结果有返回上限与截断标记,记录统计不受该上限影响。官方 site helper 按精确 `source` + `source_id` 匹配;access map 中心只表示 site 级参考点,并在 `source_note` 保留官方 URL。

View File

@@ -70,7 +70,10 @@ DB 变化不再默认创建 `earth_refresh` 任务,因此不会被同 source
| --- | --- |
| `clear_then_reload` | 先清前端本地图层对象,再强制重拉接口。删除数据时优先使用。 |
| `reload` | 保留旧对象直到新数据返回,适合定位、元数据或非破坏性更新。 |
| `delta` | 只用于 `earth_interactables`,按 id upsert 或 remove。 |
| `delta` | `earth_interactables` 按 id upsert/remove;船舶普通写入由专用 `vessels` 通道按 MMSI 更新确认状态,不触发整层清空。 |
船舶删除仍发出 `reload` 校准提示,`vessel_current_state` 的逐项删除同时进入船舶 remove 通道。源通知只决定哪些 MMSI 需要更新,推送值以当前状态表为准。全局订阅使用 `scope: "global"`;消息大小上限用于拆包,不用于丢弃其余船只。
接口在真实 0 数据时必须返回 200 和空集合;只有真实接口异常才返回 5xx。前端收到删除事件后,如果重拉失败,应保持已清空状态并显示轻量错误,不恢复旧对象。

View File

@@ -323,7 +323,7 @@ AISStream 使用 WebSocket 实时流,采集器只写入 `ais_raw_observations`
GET /api/v1/vessels/snapshot?bbox=-180,-85.05112878,180,85.05112878&zoom=12&limit=3000
```
该接口查询 `vessel_current_state` 当前状态表,只返回有效窗口内每个 MMSI 的最新点;原始 `ais_raw_observations` 继续保留给轨迹、审计和态势分析,但不再由展示接口临时扫描聚合。Earth 前端统一传全球 bbox,不随当前镜头视口反复请求。实时更新如接入 `/ws` 的 `vessels` channel,应作为 reload/dirty 提示触发合并刷新,不能把每条 AIS delta 直接变成整层重建。
该接口查询 `vessel_current_state` 当前状态表,只返回有效窗口内每个 MMSI 的最新点;原始 `ais_raw_observations` 继续保留给轨迹、审计和态势分析,但不再由展示接口临时扫描聚合。Earth 前端统一传全球 bbox,不随当前镜头视口反复请求。实时更新使用 `/ws` 的 `vessels` 全局订阅,按 MMSI 推送确认状态并原位更新;超出单帧上限时拆包,保留所有更新。首次加载、重连和删除校准仍使用快照,详见[采集器架构](backend-collectors.md)。
## 自定义 REST / WebSocket 映射运行时

View File

@@ -1,5 +1,15 @@
# 智能星球前端结构
## 地表贴图资源记录
2026-09-21,`assets/8k_earth_daymap.jpg` 替换为 Natural Earth I 配套栅格的 8192×4096 Web 派生图,加载路径和渲染代码未变。源图为 EPSG:4326、16200×8100;旧 JPG 在仓库 `resources/earth/archive/2026-09-21/` 逐字节保留。来源、许可、源包/目录版本差异、SHA256、JPEG 参数及按导出尺寸重算的定位文件见 `resources/earth/natural-earth-1/`。原始大文件位于 Git 忽略的 `downloads/earth/`,不进入前端发布包。
底图选择器默认使用“原始底图”,并将该选项放在首位,显示菜单可切换并在本地保存选择。已有 Natural Earth I 保留可用,并增加“原始底图”选项,使用 2026-09-21 归档原图的逐字节副本;NASA 21K 只作为源归档,运行时统一使用一张 8K 贴图。新贴图成功加载后才替换并释放旧 GPU 纹理,失败时保留旧纹理,过期响应不会覆盖当前结果;这是源切换,不新增渲染层。物理贴图为 JPEG sRGB,未烘焙政治边界,也未完成逐段边界对齐验收。星空逐字节恢复到上次提交前的 commit `83a10a6c`:`celestial.js` 与该提交一致,删除 `skyTint`、`skyBrightness`,`brightStarOpacity` 恢复为 `.88`,不再校准星空密度和颜色。高清 overlay 使用白色 `MeshBasicMaterial`、opacity 1、`toneMapped:false`,由 `texture-colors.js` 统一 sRGB 读写;r128 使用 `encoding`/`LinearTosRGB`,r160 使用 `colorSpace`/colorspace fragment。不叠加 Phong 白光、彩色暮光、蓝色夜染、`cameraBoost` 或 shoulder 曲线;daylight 中性亮度不超过原图最大值,dark 仅做 neutral 缩放,原图资源不变。Natural Earth 原图本身偏浅,不声称恢复不存在的细节。为 WebGL1 的 `fwidth` 开启 derivatives。
昼夜明暗使用以太阳地平线为中心的窄平滑过渡(法线点积 ±0.12)。日面保持原图亮度上限;夜面线性光照系数为 0.24,原图暗部最高保留 0.60,按源亮度连续过渡,以保持暗色海洋和森林细节。只使用中性标量,不增加曝光、彩色罩层、额外渲染通道或纹理。
地表底图与新闻频道复用 `hud-select-menu.js` 的 HUD 浮层、列表、选中标记、键盘导航和定位;`tv-source-menu.js` 仅保留频道文案适配。底图关闭搜索、分页和计数,菜单通过触发按钮归属参与移动抽屉的内部点击判断。
本文件描述当前智能星球前端的真实结构,重点是帮助后续继续改 HUD、图层、媒体面板、真实地形、BGP 可视化时,不再重复踩结构和状态同步上的坑。
相关规则建议一起参考:
@@ -9,6 +19,8 @@
## 当前目标
算力卡片的年度观测由 `/api/v1/data-products/compute/series` 返回,后台“算力对比”配置为唯一维护入口。`compute-topic.js` 以 no-store 请求读取数据,接收同源 `planet-compute-configuration` 发布通知,并在可见且专题启用时每 60 秒及页面恢复可见时检查更新;刷新观测不重拉设施图层。`destroy()` 释放计时器、通知通道和可见性监听。国家目录、区域完整覆盖条件与缺失语义仍由后端统一处理。
Earth 前端不是普通管理页,它是独立的大屏展示前端。当前产品目标是:
- 维持地球视图的空间感和可读性
@@ -56,7 +68,7 @@ React 路由入口:
- 各图层集成
- Earth 级别状态同步
Earth 收到 `/ws` 的 `earth_updates` 时只把它当作刷新提示,真实数据仍通过 `/api/v1/visualization/...` 接口重新 GET。数据库驱动的刷新由后端 listener 直接清理缓存再广播,不再默认经过 `earth_refresh` 作业队列;前端收到 `database_changed` 后会按 layer 读取 `clear_then_reload`、`reload` 或 `delta` 策略。`clear_then_reload` 必须先清 Three.js 对象再 no-store 重拉,summary 只做一致性校验,不能用 `0` 作为跳过图层重拉的理由。技术链路见 [数据作业与 Outbox 技术架构](/home/ray/dev/linkong/planet/docs/technical/zh/data-job-earth-sync-architecture.md),业务数据流见 [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)。
Earth 收到 `/ws` 的 `earth_updates` 时只把它当作刷新提示,普通图层仍通过 `/api/v1/visualization/...` 接口重新 GET;船舶使用下文的 `vessels` 专用确认状态增量通道。数据库驱动的刷新由后端 listener 直接清理缓存再广播,不再默认经过 `earth_refresh` 作业队列;前端收到 `database_changed` 后会按 layer 读取 `clear_then_reload`、`reload` 或 `delta` 策略。`clear_then_reload` 必须先清 Three.js 对象再 no-store 重拉,summary 只做一致性校验,不能用 `0` 作为跳过图层重拉的理由。技术链路见 [数据作业与 Outbox 技术架构](/home/ray/dev/linkong/planet/docs/technical/zh/data-job-earth-sync-architecture.md),业务数据流见 [业务架构与数据流转](/home/ray/dev/linkong/planet/docs/technical/zh/platform-data-flows.md)。
### 3. 地球控制层
@@ -170,6 +182,14 @@ Browser Camera provider 的手势识别管线在 [motion-browser-provider.js](/h
`tv.js` 管理 `media-panel` 里的直播 / 态势新闻 tab。toolbar 打开或切换 TV/新闻时,会通过 `earth:tv-visibility-change` 和 `earth:tv-tab-change` 回写 Earth 设置:面板可见性仍按 desktop/mobile viewport 存在 `views.<scope>.panelVisibility.media-panel`,当前 tab 存在 `shared.mediaPanelActiveTab`,因此刷新页面后能恢复用户上次打开的直播或新闻状态。`closeTransientMobileOverlays()` 这类临时收起会带 `persist:false`,不会覆盖用户偏好。
`tv-source-menu.js` 通过共享 `hud-select-menu.js` 复用 HUD 面板与图例列表样式,使用 popover 显示搜索、滚动列表和固定计数栏。`tv.js` 按 50 条请求 `/api/v1/tv/streams`,维护当前频道缓存;搜索交给后端完整目录,翻页用 `next_offset`。菜单通过取消请求和请求代数屏蔽过期响应,不能让旧搜索覆盖新输入。刷新目录时通过 `selected_id` 恢复不在第一页的频道。
直播恢复策略由 `tv-playback-policy.js` 统一判断:所有非致命 HLS 事件交给播放器恢复;致命网络或媒体错误最多恢复三次。只有后台允许自动回退、用户未手动指定频道且备用源可内嵌播放时才允许换台。`tv.js` 的异步回调核对播放器实例与频道,原生 video 错误不再抢先绕过 HLS 恢复。`tv_streams.py` 将旧内置 CCTV-4 CDRM 地址(含已保存配置中的同一地址)更新为 WCETV 的美洲版 HLS,名称明确版本并保留其他自定义源;HTTP 成功不代表视频可以解码。
已验证支持 CORS 的直播源可显式设置 `use_proxy:false`,由浏览器直连以减少分片转发开销;未指定时仍默认使用 TV 代理。WCETV 内置源使用直连。
直播信息折叠继续复用 `createHUDPanel` 和原有 0.22 秒 CSS 过渡。共享控制器测量正文高度写入 `--hud-body-content-height`,避免 `max-height: none` 到 `0` 无法插值;内容尺寸变化由 ResizeObserver 更新。向上展开的已拖动面板通过尺寸观察补偿顶部位置,动画期间保持底边,不再关闭正文 transition;补偿期间暂停外框的 top 过渡,避免二次插值导致位置漂移。
`brand.js` 管理智能星球 HUD 品牌资源。默认品牌来自静态资源,运行时覆盖值来自 `/api/v1/earth/brand`,上传的图片通过 `/earth-brand-assets/...` 读取。前端必须把 logo/title 图片和文本 fallback 分开处理:图片加载失败时显示文本标题,文本字段为空时使用后端默认值,避免 HUD 品牌区空白。控制台的智能星球内容页负责保存和重置品牌配置,智能星球前端只消费结果。
`about.js` 管理智能星球设置里的“关于”卡片。默认内容仍保留在前端作为兜底,运行时优先读取 `/api/v1/earth/about`。接口失败或字段缺失时必须回退默认值,避免设置页出现空白。控制台的智能星球内容页提供“关于”tab,保存走 `PUT /api/v1/earth/about`,恢复默认走 `DELETE /api/v1/earth/about`。
@@ -209,6 +229,14 @@ TV 预览需要尽量复用 Earth 运行时的直播卡片结构和状态标签
新闻巡航摘要的未来计划保存在仓库路径 `docs/plans/earth-news-cruise-summary-plan.md`,不作为公开 Docs 页面入口。
## 渲染热路径
`cable-batches.js` 将全部海缆线段与全部登陆点分别合批绘制;`cables.js` 继续拥有原始业务对象、拾取、遮挡和选择状态。避免在后续功能中把这些代理对象重新加入逐对象绘制,或把绘制批次当作业务对象返回给详情卡。
卫星的逐点呼吸在 GPU 中计算。全量 SGP4 位置和初始轨迹计算通过 `satellite-position-worker.js` 离开主线程;主线程只接收位置快照、更新共享交互坐标及绘制缓冲。轨道数学由 `satellite-propagation.js` 同时供 Worker、预测轨道和同步降级路径使用,避免多套公式漂移。数据重载、清空和高度切换必须同步终止旧计算,不能让旧快照恢复已清空的图层。
`i18n.js` 的 MutationObserver 只翻译新增子树、变动文本或变动属性。局部时钟、状态和下载进度更新不能触发整页语言控件同步;全局语言同步只属于初始化、语言切换或包含新语言控件的子树。
## 当前样式分层
Earth 的 CSS 不是一份大样式表,而是分层管理:
@@ -344,7 +372,13 @@ AIS 船只图层入口:
AISStream 的 `PositionReport` 常带实时位置和 `MetaData.ShipName`,但船型通常来自低频 `ShipStaticData.Type`。后端会把 `MetaData.ShipName` 补进船名,并将类型码映射为 Cargo / Tanker / Passenger / Fishing / Military;仍缺失的船型需要等待静态 AIS 消息或后续船舶资料 enrichment,不能在前端凭颜色之外的信息臆造细分类。
旧 `/api/v1/visualization/geo/vessels` 路由已移除。前端打开船只图层时只应拉取一次全局 `/api/v1/vessels/snapshot`;API 参数里的 bbox 是后端接口约束,Earth 运行时传全球范围,不表示当前镜头视口。`/ws` 的 `vessels` channel 如启用,只作为低频 reload/dirty 提示,不能把每条 AIS delta 直接变成整层重建。后端通过 `diagnostics.source == "vessel_current_state"` 暴露当前状态链路。
旧 `/api/v1/visualization/geo/vessels` 路由已移除;初始数据仍通过全球 bbox 的 `/api/v1/vessels/snapshot` 读取。
AISStream 与 BarentsWatch 共用 `/ws` 的 `vessels` 增量通道。Earth 使用现有 WebSocket 连接订阅 `scope: "global"`,不跟随镜头改变订阅范围。后端按 MMSI 合并一秒内的通知,再读取 `vessel_current_state` 的已确认状态;单帧最多 1000 项,超过时继续发送后续帧,不截断不同船只。原始源消息不能直接覆盖客户端的确认位置。当前状态删除也会产生按 MMSI 的 remove 通知。
前端将短时间内同一 MMSI 的变更合并为最新值,通过 `Interactable.updateItems()` 原位修改位置和颜色缓冲。航向分桶变更只更新受影响的桶,容量不足才扩容;保留已有 marker 身份、锁定状态和材质。首次进入、重连、删除提示及每分钟巡检仍用 snapshot 校准,但不先清空整个图层。收到有数量上限的快照时,不能把被截断的船只当成删除;快照返回期间到达的较新更新和删除也不能被旧响应覆盖。快照携带查询开始时的 `generated_at`,与实时帧时间一起用于识别缓存旧快照,避免新船或删除状态被回滚。
`earth_updates` 中船舶普通写入的策略是 `delta`;专用通道连通时不再触发整层重拉。删除或断连后的校准使用 `reload`,并复用现有对象。关闭图层会取消船舶订阅并清空待应用变更。轨迹、冲突详情和审计继续按单船接口读取历史事实。
新的图层接口族是 `/api/v1/layers/*`,用于把地图渲染数据和聚合面板统计分开。地图层请求必须带 `bbox`、`zoom` 和受控 `limit`,响应会返回 `visible_count`、`returned_count` 和 `diagnostics`,其中 `degraded/truncated/limit_clamped` 用于前端提示降级。右侧聚合统计不要从图层响应累加,应读取 `/api/v1/data-products` 或 `/api/v1/data-products/{product_id}/status`,因为这些统计保持全量/全局口径,不随当前视口变化。
@@ -380,6 +414,8 @@ AISStream 的 `PositionReport` 常带实时位置和 `MetaData.ShipName`,但
详情卡里的坐标候选状态由 [info-card.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/info-card.js) 按 `entityType:entityId` 缓存在模块内存中。用户关闭详情卡或待定位列表后再次打开同一个算力中心 / BGP 观测站,已经采集到的候选和状态文案会恢复;`一键采用` 会优先使用缓存候选,避免重复调用在线地理编码或 LLM factcheck。保存成功后该实体的候选列表会清空为“正在刷新图层”状态,避免旧候选在刷新后继续误导用户。
批量定位由 `runUnresolvedComputeCenterBatch()` 持有实体上下文队列,不持有面板 DOM;`locationCollectStateCache` 保存候选、进度和保存结果,重建卡片时重新 hydrate。`earth:compute-center-location-batch-change` 同步任务状态与气泡入口,成功条目根据 `savedCandidate` 排除,全部完成后保留 ✓ 入口。状态只存在当前 Earth 页面内存中,刷新、关闭或路由离开会终止未完成队列;已保存坐标仍以后端为准。
候选行的 `预览 / 保存` 按钮采用单一的事件委托模型:每个候选根(详情卡里的 `[data-collect-cache-key]` 块,或待定位列表里的 `[data-unresolved-item]`)只挂一个 `click` 监听,由 `data-candidate-actions-bound` 幂等标记,不再混用 `pointerup` / `click` 直绑或重复委托。候选对象不再以 JSON 字符串塞进 HTML 属性后再 `JSON.parse`,按钮只携带 `data-candidate-index`,handler 通过 cache-key 在模块内存的 `Map` 里取出原对象,避开 HTML 实体转义对 `&` / `<` / `"` 的破坏。点击 `预览` 会派发 `earth:preview-location-candidate`,由 `main.js` 的 `previewLocationCandidate()` 调用 `showComputeCenterLocationPreview()`:在候选经纬度上挂双层空心呼吸 sprite(视觉参考 BGP 事件 ring),并把视角聚焦到候选坐标;切换到另一个候选会替换为新呼吸圈,保存时立即清除并由 `spawnSavedComputeCenterLocation()` 即时生成正式算力中心交互图标。注意 `main.js` 没有模块级 `earth` 变量,所有 location-save / preview 处理函数必须先 `const earth = getEarth();`,否则会在事件 handler 里抛 `ReferenceError` 被 `.catch` 静默掉,外观上等同于按钮“没有反应”。
`earth:compute-center-location-saved` 之后的图层校准链路对后台刷新失败保持沉默:`spawnComputeCenterAfterLocationSave()` 已经把 toast 和 locked 状态都给了用户,`refreshComputeCentersAfterLocationSave()` 只在场景就绪时重新拉取后端数据,本身不再吐 `已保存` toast;`handleComputeCenterLocationSaved()` 在 spawn 成功路径让 refresh 静默后台运行,只在 spawn 返回 `null`(场景未就绪)或抛错时才让 refresh 接管成功 toast,refresh 自身报错只走 `console.warn`,绝不冒泡成 `保存失败` 文案——保存请求本身已经成功,刷新失败属于后续同步问题。
@@ -392,7 +428,7 @@ asset 图标大小由 `Interactable` 的 `icon.fitSize` 控制。SVG / 图片文
跨 Interactable 的同坐标关系也在公共层记录,但真实位置必须始终以 `icon_base_position` 为准。缩放、避让、聚合和后续 spiderfy 展开都只能改变屏幕表现,不能写回 `marker.position` 或 `THREE.Points` 里的业务锚点;巡航定位、详情卡、搜索定位和 picking 返回对象都必须落回真实经纬度。多个图标归入同一个经纬度 key 时,公共层只写 `icon_avoidance_*` 元数据,供业务层弱化 halo 或显示聚合提示;真正的低缩放聚合应通过独立 cluster glyph / screen layout 层实现,而不是把对象沿地表切平面挪开。
`Interactable` 的单点显示只由全局地图缩放决定:170% 及以下强制显示小圆点,超过 170% 显示原图标。cluster 现在由 `cluster.strategy` 决定:`stable-spherical` 使用离散 zoom band 和 3D 球面分桶,BGP、算力中心和 Earth interactable 在同一 band 内旋转或细微缩放时不会重新计算聚合拓扑;`dynamic-screen` 保留屏幕空间聚类,适合船只这类实时高频图层;`none` 关闭聚类。稳定球面聚类的 cluster 圆点刚性落在成员 3D 质心投影上,不参与 2D 避让,避免缩放时被推离真实地理位置。cluster 圆点大小随包含对象数量增长,数量过多时按稳定地理顺序拆成多个较小圆点;数量默认只在 hover tooltip 中显示。这个过程只设置 `icon_cluster_*` 展示元数据和重建渲染 Points,不改变每个 marker 的真实经纬度。
`Interactable` 的单点显示只由全局地图缩放决定:170% 及以下强制显示小圆点,超过 170% 显示原图标。cluster 现在由 `cluster.strategy` 决定:`stable-spherical` 使用离散 zoom band 和 3D 球面分桶,BGP、算力中心和 Earth interactable 在同一 band 内旋转或细微缩放时不会重新计算聚合拓扑;`dynamic-screen` 保留屏幕空间聚类;`none` 关闭聚类。船只显式关闭聚类与避让,保证增量更新时不重建聚类拓扑。稳定球面聚类的 cluster 圆点刚性落在成员 3D 质心投影上,不参与 2D 避让,避免缩放时被推离真实地理位置。cluster 圆点大小随包含对象数量增长,数量过多时按稳定地理顺序拆成多个较小圆点;数量默认只在 hover tooltip 中显示。这个过程只设置 `icon_cluster_*` 展示元数据和重建渲染 Points,不改变每个 marker 的真实经纬度。
接口细节、生命周期和接入示例见:
@@ -409,17 +445,9 @@ asset 图标大小由 `Interactable` 的 `icon.fitSize` 控制。SVG / 图片文
[main.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/main.js) 只负责在双指捏合缩放和动捕缩放时调用 `setZoomLevel()` 和 `showZoomStatusCapsule()`。鼠标滚轮与缩放按钮的胶囊提示应继续放在 `controls.js`,避免同一种缩放反馈散落在多个模块。
拖拽地球的旋转灵敏度会根据当前缩放连续衰减,而不是按某个缩放阈值分段:
`globe-navigation.js` 统一投影缩放与表面拖动。相机距离采用 `sqrt(R² + (D₀² - R²) / zoom²)`,并保留最小地表净空;100% 视角不变,500% 仍在球外,反向换算也使用同一公式。滚轮、按钮、捏合、搜索与巡航继续共用 `setZoomLevel()`。
```javascript
const scale = THREE.MathUtils.clamp(
Math.pow(zoom, -CONFIG.dragRotationZoomExponent),
CONFIG.dragRotationScaleMin,
CONFIG.dragRotationScaleMax,
);
```
调参入口在 [constants.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/constants.js):`dragRotationFactorBase` 控制基础速度,`dragRotationZoomExponent` 控制放大后的衰减曲线,`dragRotationScaleMin` / `dragRotationScaleMax` 控制上下限。
鼠标和单指在按下时按实际相机视锥选择并固定拖动模式:球体完整放入视口时,使用旧版屏幕位移、缩放灵敏度、0.18 追赶平滑和松手衰减惯性,指针移出球面仍保持原方向;球体放大到被视口裁切且按下命中球面时,才用射线抓点与四元数逐像素跟手。大球拖动移出球面后暂停更新并清除甩动速度,不再用可能反向的最近点代替真实交点;从球外开始则使用屏幕位移拖动。一趟拖动中不切换模式,自转不叠加。双指进入缩放时清除惯性,回到单指时重新判断模式并建立抓点。拖动阈值仍按起点累计位移判断,取消手势不产生惯性。
### `data-status-target`
@@ -510,3 +538,46 @@ SGP4 传播输出是惯性系位置,不能直接当成 Earth 的经纬度固
- [cruise-sequencer.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/cruise-sequencer.js)
- [callout-connector.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/callout-connector.js)
- [bgp-cruise-adapter.js](/home/ray/dev/linkong/planet/frontend/public/earth/js/bgp-cruise-adapter.js)
## 算力空间卡片
Earth 原有 Roboto Condensed 字体已随 `assets/fonts/` 本地分发,许可证与来源同目录保存,`css/fonts.css` 统一声明;页面不再等待 Google Fonts 远程样式表。字体家族与字重保持原样。
`compute-topic.js` 管理参考期、口径、参与方和来源,读取后端 `compute_catalog.py` 提供的规范名称、分组与地区标签锚点。当前目录覆盖 45 个已有坐标的参与方,未知历史指标返回缺失。参与方接口不再只接受中美;区域分组不参与分母计算,坐标不能复用为设施定位。
`spatial-callouts.js` 将布局委托给新的 `spatial-callout-controller.js`,按稳定地区 ID 复用 `CalloutConnector` 管理卡片。布局采用事件驱动;新计算尚未完成时保留有效旧矩形,投影 source 连线持续跟随地球。布局参考点改为实际出线方向,因此卡片保持在对象同侧,不会被左上地理点牵到右下角。solver 优先同扇区的邻近位置、较短连线和较少交叉;最多交换三轮空位改善 assignment,不在每帧全量重排。全屏地球允许附近前景位置,正常大小尽量留在球外,不为了完全避开球体而跨越整盘面。慢速模式下,方向明显偏移或连线持续交叉 1.8 秒后才能提议整体重排,两次自动提议至少间隔 8 秒;只有整体代价降低至少 18% 且超过最低收益门槛才采用。手动/惯性仍在 72ms/12px 预算内响应。展开与手动摆放获得 12 秒阅读缓冲;鼠标悬停、键盘焦点及拖动继续锁定,普通点击留下的焦点不永久锁卡。手动位置保持为偏好,缓冲结束且地球出线方向变化至少 45° 后才允许自动调整,旧保存位置不作为永久锁定。背面保留球体轮廓方向提示且不改变卡片身份。本地位置/尺寸保存与“恢复自动布局”由控制器处理。`ResizeObserver` 响应卡片调整尺寸和收起;选择器与来源窗复用共享拖动句柄。
对象展示统一通过 `PresentationController` 及新的 object-callout anchor/layout 路径。点击真实 BGP、算力中心、船只、卫星或海缆对象时,先绘制连接线,再把卡片放到对象旁边。源锚点从实际点击位置或原始区域标签开始;接近遮挡时,随着旋转自动平滑移向仍可见的高亮几何采样。`layoutReferencePoint` 取实际出线点的方向参考,`packDirectionalEdgeCallouts` 优先把卡片放到对应方向的屏幕边缘,给地球中间留空;边缘 lane 不可用时才内收,卡片不能盖住自己的出线点。有界联合分配最多使用 24 个 beam 状态和 32 个候选,避开已有 HUD,再最多三轮安全交换减少方向交叉;旧 solver 作为 fallback,不逐帧全量重排。slow 模式还检查实际卡片目标相对 source 反向超过 90° 后才提交变化;manual/hover 锁优先。source point 仍每帧更新为连线目标;固定对象跟踪动态位置,卡片仍可拖动。完全背面的对象保留球体轮廓方向提示、较淡连线和空心源点,这些是方向引导而非地理点。国家候选最多 96 条、硬上限 128,并使用缓存,不在每帧扫描 GeoJSON。算力卡继续让原始 point 独立于动态 source point,保持阅读稳定;共享卡片端接点沿障碍物周界滑移。海缆展示优先真实 ray-hit 的 local point,再按 `cableId` 采样高亮几何。移动端沿用 mini popup,点击后打开抽屉详情。旧异步 presentation 不能 dismiss 新的 selection。
配置窗的 `configVisible` 与卡片的 `cardsEnabled` 分离;X/Escape 只关闭配置,开关停用时才释放连线并恢复专题查询前的图层状态。配置和来源窗直接绑定 HUD 字号,避免拖到 `container` 后继承大号正文。主卡围绕当前选择的算力指标,显示大号百分比或带单位的绝对量、观测年份、历史上下文和来源入口。缺少可比数据时明确提示,不在没有观测值时绘制空趋势图。设施记录和覆盖信息仍完整保留在展开后的辅助区,样本数只是证据上下文,不能当作算力值或全球覆盖率。长列表仅在需要时展开;手机手风琴保持同一单一占比/绝对量语义。本地位置/尺寸保存与“恢复自动布局”仍可用;无变化的 DOM、尺寸、阅读状态应保留,过时响应不能重新启用卡片。
动捕/地球运动模式由 `main.js` 的 `isDragging`、`activeTouchPoints`、`hasActiveGlobeInertia` 接入。方向错位或交叉触发的是一组卡片联合优化,而非逐卡抢占剩余空位。`callout-region-path.js` 提供短直连加末端短折线;实际绘制、交叉评分及候选搜索共用路径几何。候选路径只计算一次,裁剪候选后才复制布局状态,避免重排期间不必要的分配。新布局计算期间保留现有卡片;队列不会因帧间隔而丢失重排请求。
共享 `CalloutConnector` 直接从折线点计算长度,不在每次写入 SVG 点后调用 `getTotalLength()`,避免多条移动连线触发反复同步布局。原有首次绘线动画保留;这一优化同时适用于原巡航展示。
移动端算力使用现有抽屉里的紧凑地区概览。同一列表采用原地手风琴:重复点击卡头收起,点击另一张先收起旧卡,最多同时展开一张长卡。展开详情保留趋势、区域成员、设施、媒体和来源,不设地区二级页面;设置仍是单独按需页。复用共享 HUD 折叠面板动画,并尊重 reduced-motion。地图卡片开关关闭时仍加载摘要,打开后共享地图加载事件并取消旧请求;抽屉事件使用 `composedPath()`,通过 innerHTML 换节点也不会误收起当前卡片。设施名单使用最高 240px 的内部滚动区域,名称左对齐并正常换行,每行至少保留 42px 触控高度。
`/series` 保留可追溯年度数据;默认主指标为全球计算设备总规模,拆分基础计算、智能计算和超级计算,观察期为 2020—2023。已核对绝对量为 2022 年 302(`caict2023`)和 2023 年 435(`caict2024`);日本全球份额为 2020—2023 的 6%/5%/4%/4%。产品分组 APAC(不含中国)和 OTHER 不能直接套报告里的日本或“其他”;EUROPE 在成员口径对齐前保持 `null`。双边份额固定使用美国加中国分母。主指标缺失时可显示最近报告年份(当前为 2023),明确选择 2026 仍保持缺失;不可比性未确认时不展示同比变化。`/facilities` 按成功采集快照过滤,并按规范目录中的唯一国家编码计数,覆盖 count 仅作为辅助信息,不能当作算力值或全球覆盖率。`compute_catalog.py` 将中国、美国、欧洲、不含中国的亚太和其他作为展示分组,覆盖 45 个规范国家编码;分组不作为统计分母。`earth_geography_policy.py` 与地图配置共用显示归属,台湾、港澳来源分区归入中国,同时保存 `source_country`。API 审计字段为 `reviewed_at=2026-09-22`,版本与该日期一致。独立 Epoch 份额模式尚未实现。宏观报告的统计范围仍按其原始方法说明,创新分差只在中美同时选择时提供。
### 国界贴地与登陆点
普通国界、海岸、主张线、高亮及登陆点使用 `earth-surface.js` 对齐高清球面的 UV 三角形。国界与高亮在贴图之后、登陆点之前绘制,高亮开启深度测试且始终位于透明渲染队列;不再透过球体显示或盖住登陆点。几何构建时计算并缓存,不增加每帧重投影。具体顺序见[渲染图层规范](earth-render-layer-order.md)。
### 新闻栏端部渐变
新闻栏的底色与上下细线由独立伪元素绘制,两端在正文范围之外渐变到完全透明;外框不使用矩形 backdrop-filter、box-shadow 或文字遮罩。地区标签按内容占宽,正文紧接标签,滚动文字不做渐隐。警报状态只改变渐变细线的颜色,避免重新引入矩形阴影边界。
算力卡片外观直接复用共享 `.hud-panel` 的背景、边框、阴影和玻璃效果,摘要、基础信息及详情区不再叠加单独黑底。标题栏与图层面板共用 `.hud-panel__header--inset` / `.layer-panel-header` 的内边距;主数字继续突出,基础信息保持常显。
保存位置恢复时会迁移旧版居中拖放坐标:只解除位于屏幕核心区域、且没有新版位置标记的旧坐标锁,保留宽高;旧版边缘位置保持不动并更新标记。新拖放保存 `positionVersion=2`,以后仍尊重用户的新位置。迁移沿用原存储键,避免因全量清空布局而丢失所有个性化位置。
### 首帧与空间卡片入场
Earth 由 `earth-runtime-plugin.mjs` 将本地模块合并为带内容哈希的入口与共享块,开发服务和正式构建使用同一路径;第三方依赖仍由原 import map 固定版本,`/earth/js/` 的相对资源及 Worker 路径不变。`earth-preload.js` 在应用初始化前请求选中的原尺寸底图,`earth-texture-image.js` 使用 ImageBitmap 异步解码,并把同一个解码结果交给 Three.js;不重采样、不改变色彩空间和采样设置,替换/销毁纹理时释放位图。高清地表先于数据图层就绪,刷新数据保留已有贴图。独立数据图层并行请求,国界版本化资源使用 HTTP 缓存,基础几何、拾取索引与声明线并行读取。直播引擎按需导入,媒体及实时连接在地表首帧后启动。
`html` 的 `data-earth-first-frame-ms` 和 `data-earth-surface-ready-ms` 记录本次导航至首次绘制、完整地表首次绘制的毫秒数;冷缓存与热缓存应分别实测,不能把本机热缓存结果承诺为任意网络下的首访时间。
空间卡片定位后保持隐藏,先复用 `CalloutConnector` 的 420 ms 绘线动画,再按共享 HUD 的 180 ms 淡入时长显示卡片。绘线期间固定目标位置,后续跟随运动不重播入场;系统减少动态效果时直接呈现。
物件点击、搜索定位及手势确认统一通过 `PresentationController`。海缆选择处理器只更新选择与图例,不再绕过控制器提前显示卡片;共享 InfoCard 的显示回调带版本标记,新的隐藏/预布局请求会使旧回调失效。所有连线卡片以绘线完成事件作为显示门槛,取消的旧巡航不能关闭新的选择卡片。物件详情复用算力卡的边缘候选与避让评分,按隐藏状态下测得的真实尺寸定位置;手机搜索结果被选中时收起搜索抽屉,让地图定位、引导线与精简卡片可见。

View File

@@ -149,6 +149,8 @@
## 海缆与登陆点
`cable-batches.js` 将海缆线与登陆点分别合批绘制。下表的 Sprite 材质和尺寸仍属于 `cables.js` 中的拾取、选择代理;每帧把颜色、透明度、缩放和可见性同步到样式纹理或实例属性。实际绘制使用 `LineSegments` 和实例化 billboard,沿用原纹理、色彩、层级与球体遮挡规则。
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
| --- | --- | --- | --- |
| 默认海缆颜色 | `CABLE_COLORS.default` | `0xffff44` | 无数据颜色时使用 |

View File

@@ -12,6 +12,8 @@
## 地表图层栈
算力专题沿用原算力中心 Interactable 图标、深度与层次顺序;国家/年份切换会替换该层数据并清除旧选中状态。专题 HUD 属于屏幕 UI,在 `main.js` 的指针排除列表中,不参与地球 picking;区域数据卡在屏幕空间由地理锚点牵引,背面不画穿球连线,设施详情复用原详情卡和移动端详情抽屉。外层大气壳保持原有实现;星空逐字节恢复到 commit `83a10a6c`,`brightStarOpacity` 为 `.88`,删除 `skyTint`/`skyBrightness`,不再校准密度和颜色。高清 overlay 使用白色 `MeshBasicMaterial`、opacity 1、`toneMapped:false`;`texture-colors.js` 统一选择 r128 的 `encoding`/`LinearTosRGB` 或 r160 的 `colorSpace`/colorspace fragment。不叠加 Phong 白光、彩色暮光、蓝色夜染、camera boost 或 shoulder 曲线。没有新增后处理、GPU draw pass 或高分辨率纹理;算力专题新增 45 个本地 SVG 旗帜资源。
| 顺序 | 图层 | 来源 | 渲染 / 半径策略 | 深度 / 交互策略 | 备注 |
| --- | --- | --- | --- | --- | --- |
| -1000 | 天体背景 mesh | `celestial.js` | 背景球 | 不参与地表拾取 | 位于所有智能星球内容之后。 |
@@ -21,12 +23,12 @@
| 0.86 | 海陆基座填充 | `country-boundaries.js` | `landAltitudeOffset = 0.32`; 海洋 `#010609`,陆地 `#080f1b` | 禁用 raycast | 即使国界线关闭,基座地图仍保持可用;半径与基座球拉开以避免远距 z-fighting。 |
| 0.96 | 高清 Earth 材质 | `earth.js` | `textureOverlayAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | 可见时作为地表拾取目标 | 高清材质始终压过海陆基座填充;半径必须高于海陆基座并与基座球保持足够间距。 |
| 1 | 大气辉光和云图 | `earth.js` | 大气 / 云层球 | 不走普通对象选择路径 | 云图由“大气云图”图层开关控制。 |
| 1 | 海缆 / 登陆点 | `cables.js` | 海缆线和登陆点都使用 `renderOrder = 1`;半径偏移都为 `0.2`;登陆点是专用 `THREE.Sprite` 黄色扁平球 | 海缆走海缆拾取路径;登陆点 `depthTest: false` 保持球体完整,并用相机到球心的球体遮挡判断避免背面穿透 | 登陆点和海缆同层贴地,避免地表设施层的凌空感。 |
| 1 | 海缆 / 登陆点 | `cables.js`, `cable-batches.js` | 海缆全量线段合并为一个 `LineSegments`;登陆点全量使用实例化 billboard;`renderOrder = 1`、海缆半径偏移仍为 `0.2`;登陆点锚点改为高清地表的 `0.48` | 原始 `Line` / `Sprite` 仅作为逐项拾取和选择状态对象,材质不再单独绘制;登陆点仍按球体遮挡判断背面可见性,批量材质 `depthTest: false` | 样式通过线缆样式纹理和登陆点实例属性同步;保留颜色、脉冲、尺寸、显隐与点击语义。 |
| 1.2 | 真实地形 | `earth.js`, `terrain.js` | `TERRAIN_CONFIG.baseRadiusOffset` 加地形位移 | 禁用 raycast | 地形压过高清材质;高清材质关闭时临时隐藏,重新开启后恢复原状态。 |
| 2.05 | 经纬线 | `earth.js` | `CONFIG.earthRadius + 0.14` | 禁用 raycast | 低透明度显示在高清材质上。 |
| 2.2 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`;claim 线不再额外抬高 | `depthTest: true`,禁用 raycast | 线层使用独立 line geometry 与 `renderOrder` 控制,但半径与高清材质壳完全一致,避免转动地球时与高清贴图出现视差。 |
| 2.29 | 国界 hover 光晕 | `country-boundaries.js` | hover 半径加 glow 偏移 | `depthTest: false`,禁用 raycast | 用 additive 光晕增强交界边和地形开启时的 hover 可见性。 |
| 2.3 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: false`,禁用 raycast | 霓虹红橘 hover 线;与普通国界线和高清材质同源半径对齐,避免重影和漂浮感;中国和中国(台湾)共享高亮组。 |
| 0.97–0.99 | 国界线 | `country-boundaries.js` | `lineAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48`;claim 线不再额外抬高 | `depthTest: true`,禁用 raycast | 线层使用独立 line geometry 与 `renderOrder` 控制,但半径与高清材质壳完全一致,避免转动地球时与高清贴图出现视差。 |
| 0.98 | 国界 hover 光晕 | `country-boundaries.js` | 与地表锚点相同,无额外抬高 | `depthTest: true`,透明绘制队列,禁用 raycast | additive 光晕处于登陆点下方,背面由球体遮挡。 |
| 0.99 | 国界 hover 实线 | `country-boundaries.js` | `hoverAltitudeOffset = EARTH_SURFACE_TEXTURE_ALTITUDE_OFFSET = 0.48` | `depthTest: true`,透明绘制队列,禁用 raycast | 霓虹红橘 hover 线;与普通国界线和高清材质同源半径对齐,避免重影和漂浮感;中国和中国(台湾)共享高亮组。 |
| 3 | 卫星 footprint 填充 / Iridium coverage ring | `satellites.js`, `iridium-footprint-adapter.js` | `GROUND_FOOTPRINT_RENDER_ORDER` | depth-tested;Iridium adapter 的 fill / ring 也使用同一 renderOrder | Footprint 在 land / texture / terrain 和国界线之上,但在算力中心和卫星之下。 |
| 3-4.5 | BGP 观测站、事件扩散圈和事件 marker | `bgp.js`, `interactable.js` | BGP 观测站和事件 marker 均使用 `Interactable` 批量 `THREE.Points`;事件 marker 使用 `BGP_EVENT_RENDER_ORDER = 4.5`;观测站主图标使用 `BGP_COLLECTOR_RENDER_ORDER = 4.4` 和 `BGP_CONFIG.collectorAltitudeOffset = 0.2`;事件 overlay 进入 `bgp-event-overlay-layer`;观测站 halo 和覆盖扇形进入 `bgp-collector-radar-layer` | BGP 事件和观测站都通过 `Interactable` 屏幕空间 picking,并参与同坐标避让 | BGP 观测站主图标与船只同层;BGP 事件与算力中心同层;向外扩散圈、观测站雷达/覆盖动画继续由 BGP 业务逻辑驱动。 |
| 4.3 | AIS 船只轨迹线 | `vessels.js` | `VESSEL_RENDER_ORDER - 0.1`;`CONFIG.earthRadius + VESSEL_CONFIG.track.altitudeOffset` | 跟随船只显隐,不单独参与拾取 | 选中船只后显示最近轨迹,低于船只 marker。 |
@@ -37,6 +39,14 @@
| 12+ | 卫星锁定 ring、halo、预测轨道 | `satellites.js` | `SATELLITE_CONFIG.overlayRenderOrder` 及偏移;预测轨道使用同一真实高度开关,并固定锁定时刻的地球姿态来绘制闭合惯性轨道;关闭真实高度时回到同层球面 | 卫星覆盖层路径 | 用于选中 / 锁定卫星强调。 |
| 98-100 | 太阳 / 月亮 halo 和 sprite | `celestial.js` | 固定 renderOrder | 天体拾取禁用 | 前景天体 sprite。 |
## 全量绘制与更新约束
- 合批只改变 GPU 提交方式,不减少卫星、线段或登陆点数量,也不按相机半球裁剪数据。海缆每对相邻顶点都保留,拾取仍返回原始业务对象。
- 卫星普通点与背景点保持两个 `Points` 绘制;呼吸动画由顶点着色器使用静态逐点参数和每帧统一时间计算。hover / locked 的隐藏标记仍由原有选择状态控制。
- `satellite-position-worker.js` 使用与主线程相同的 Three.js / SGP4 版本,通过 `satellite-propagation.js` 共享轨道、显示高度和 fallback 计算;全量位置及初始轨迹以可转移数组交给主线程。主线程维持一个在途计算,不堆积过期帧;重载、清空和高度模式变更会终止旧 Worker。
- Worker 启动失败或超过 `SATELLITE_CONFIG.workerStartupTimeoutMs` 时退回共享的同步计算,避免图层无限等待。计数、数据接口、原有更新周期和轨迹长度不变。
- 验证时同时检查全量 draw range、逐项拾取、锁定覆盖层、轨迹、背面遮挡、开关及清空后的资源释放;比较帧耗时必须使用相同数据、相同视角和分辨率。
## 开关联动
| 开关 | 行为 |
@@ -66,8 +76,17 @@ Earth 的地表不是单一 mesh,而是多层近似同心球:基座球、海
| --- | --- |
| 智能星球坐标 hover | 高清材质可见时使用高清材质 overlay 作为地表拾取目标,否则使用智能星球基座球。 |
| 国界 hover | 先把地表拾取坐标转成经纬度,再用 GeoJSON 点面判断;国界 hover 线本身不接收 raycast。 |
| 国界 hover 视觉 | hover 时压暗普通国界线,并绘制无深度测试的光晕和实线。 |
| 国界 hover 视觉 | hover 时压暗普通国界线,并绘制有深度测试的光晕和实线;不覆盖登陆点。 |
| 中国 / 台湾 hover | `CHN` 和 `TWN` 被归到同一个 hover 高亮组;tooltip 仍显示鼠标实际命中的 feature。 |
| 地形 | 只作为视觉层参与,`terrain.raycast` 已禁用。 |
| 卫星 | 使用屏幕空间卫星拾取,避免 footprint 或地表层挡住卫星点击。 |
| 船只 | 不使用对象级 sprite raycast。`main.js` 会在拖动 / 惯性期间跳过 hover picking,平时将正面船只投影到屏幕坐标,用像素半径命中最近船只;点击后可加载轨迹线。 |
### 国界贴地与登陆点顺序
`earth-surface.js` 在几何构建阶段按高清 SphereGeometry 的 UV 三角形计算贴地点,供普通国界、海岸、主张线、高亮和登陆点共用。长线段按地表网格间距细分,避免把理想球面顶点之间的长弦当作地表。高亮始终进入透明队列并启用深度测试:地表贴图 0.96 → 国界/高亮 0.97–0.99 → 登陆点 1。只调整地表附着与遮挡,不改变地理数据或经纬度归属;构建后缓存,不在转动时重新投影几何。
### 启动加载与绘制顺序
运行时优先完成选中的完整高清地表,再启动相互独立的数据图层请求;国界构建不再阻塞地表首帧。该启动优化不改变各图层的 renderOrder、深度策略、几何精度或贴图分辨率。数据刷新不清除当前地表,纹理仅在替换成功或场景销毁时释放。

View File

@@ -1,5 +1,9 @@
# 控制台前端结构
## 算力对比配置
“智能星球内容”的 `compute_comparison` 分区挂载独立 `ComputeComparisonSettings.tsx`,通过结构化表单维护来源、年度国家观测及创新分差,复用现有 Panel、Input、Select、Button 和确认弹窗,不让运营人员编辑 JSON。草稿在同页切换分区时保留,整页离开提示未保存内容;手动重新加载和恢复默认需确认。组件仅激活时读取管理接口,保存使用后端 revision 防止覆盖其他编辑者,失败保留草稿。表单内容区负责内部滚动,标题和发布操作留在首屏。`compute-comparison-model.ts` 集中类型与同源更新通知,`compute-comparison.css` 只负责该分区布局。
本文件描述当前控制台前端的真实结构,目标是帮助后续页面开发、表格改造、布局治理和状态收口时快速找到正确入口。
相关规则建议一起参考:

View File

@@ -16,6 +16,10 @@
- 只有达到城市级或更高精度的位置能渲染到 Earth。
- 本地 JSON registry 不作为算力中心或 BGP 观测站的运行时候选来源;持久事实写入数据库维表。
算力中心展示中,`precise` 和 `site` 候选保持源坐标。多个 `city`、`estimated`、`needs_confirmation` 记录坐标相同时,只在 `displayLatLon` 上分散:使用 `source` 与 `source_id` 的稳定 hash 选择不规则云状方向,并在 0.4–6 km 范围选择半径。偏移确定性与记录顺序无关,不使用数据库行 ID,也不是圆形、网格或螺旋;源坐标永不写回。tooltip 和详情卡标注“城市级近似位置(分散显示)”。
位置优先级为:确认缓存、源精确坐标、官方登记 site、源 city/site、未确认缓存。城市级近似不能标成 `precise`,也不能猜测总部。官方登记 helper 按精确 `source` + `source_id` 匹配;当前覆盖 Gyoukou 2.0/2.2 的 JAMSTEC Yokohama 与 Sophia ALCF,共 3 条、2 个园区。官方 access map 中心只作为 site 点,不表示 rack/building 测绘点,并在 `source_note` 保留官方 URL。
## 核心接口
```python
@@ -51,7 +55,7 @@ class LocationResolver(Protocol):
def resolve(self, query: LocationQuery) -> ResolverOutput: ...
```
`LocationPipeline.collect_candidates()` 返回排序后的候选和 `attempted_queries`;`resolve_best()` 返回最佳候选及诊断信息。默认排序按 source rank、precision rank、confidence,且对同 source 和同坐标候选去重。
`LocationPipeline.collect_candidates()` 返回排序后的候选和 `attempted_queries`;`resolve_best()` 返回最佳候选及诊断信息。默认排序按 source rank、precision rank、confidence,且对同 source 和同坐标候选去重。旧记录也经过同一条路径重新归类为 `country`、`region` 或 `unknown`;不可渲染记录保留在完整 `unresolved` 列表中。
## 内置 resolver

View File

@@ -126,6 +126,9 @@
### AISStream 实时船舶
在智能星球打开船只图层后,AISStream 与 BarentsWatch 的后续位置会自动更新。更新时会保留已选中的船只;短暂断线重连后会自动校准,无需反复关闭、开启图层。
`AISStream 实时船舶` 是全球 AIS WebSocket 采集器。连接测试通过只说明 API Key 和 endpoint 格式可用;真正的全球船只数据来自后台 `aisstream_vessels` collector 长连接运行并写入 `ais_raw_observations`。
操作步骤:
@@ -163,7 +166,11 @@ provider 和模型既可选预设也可直接输入自定义 id/name。常用字
- Max Tokens、Anthropic Version:可保持默认
- Timeout / Retry:超时和重试次数
Base URL 输入框尾端的插头图标会触发连接测试。测试通过会显示当前模型返回的简短回复。
Base URL 输入框尾端的插头图标会触发轻量连接测试:检查代理和模型目录,并确认当前模型出现在目录中。通过后可到 Playground 发起实际生成,核对账号权限和模型回复;目录检查本身不执行生成调用。
点击右上角的刷新图标可更新“可选模型”。成功后目录会保存,重新打开页面仍可使用;有发布日期的目录按新到旧排列。刷新保留当前模型、密钥、地址和未保存的修改。点击一个可选模型后,再点“保存”才会改变实际使用的模型。刷新失败时会显示错误并保留上次目录。
刷新使用当前表单中的基础地址、协议和 API Key,因此可以先填写新配置再刷新,无需提前保存。需要凭证的供应商必须填写对应地域的 Key;Ollama 的空目录表示尚未安装模型。选择已知协议的模型时,协议适配会同步到表单;自定义 OpenAI 网关保留手动选择的协议。
### 工具
@@ -261,6 +268,8 @@ Base URL 输入框尾端的插头图标会触发连接测试。测试通过会
在一个地球视图中观察:BGP 事件与观测态势、卫星和轨迹、海缆与登陆点、算力中心、国界线/经纬线/高清材质/云图/地形、新闻直播和态势新闻、搜索和聚焦对象详情。
放大至 500% 时视点仍在地表外。鼠标或单指拖动会让抓住的地表位置跟随指针;按住时不会叠加自转,松手保留惯性,停住再松手则停止。双指缩放后可直接用剩下的一根手指继续拖动。
### 图层控制
右侧图层面板用于打开或关闭图层。常见图层:经纬线、国界线、高清材质、大气云图、海缆、算力中心、BGP 观测、卫星、AIS 船只、地形。卫星轨迹不再作为单独图层出现在图层列表中,改在设置面板里控制。
@@ -271,6 +280,8 @@ Base URL 输入框尾端的插头图标会触发连接测试。测试通过会
- 轨迹依赖卫星
- 高清材质关闭时,地球显示基座地图和边缘识别效果
底图选择器默认使用“原始底图”,并将该选项放在首位,显示菜单可切换可用底图并在本地保存选择。已有 Natural Earth I 保留可用,并增加“原始底图”选项,使用 2026-09-21 归档原图的逐字节副本;NASA 21K 只作为源归档,运行时统一使用一张 8K 贴图。新贴图成功加载后才替换并释放旧 GPU 纹理,失败时保留旧纹理,过期响应不会覆盖当前结果;切换不新增渲染层。
### 图例
左下角图例会跟随当前聚焦或启用的图层切换。已覆盖:海缆、卫星、国界线、算力中心、BGP、AIS 船只。
@@ -281,11 +292,21 @@ AIS 船只图例按船型显示颜色:货轮、油轮、客船、渔船、军
支持查找海缆、登陆点、卫星、算力中心、BGP 事件、BGP 观测站。结果可快速定位并打开详情。
点击地球上的 BGP、算力中心、船只、卫星或海缆后,先从物件引出连接线,再沿同一方向把详情卡尽量放到屏幕边缘,中间给地球留出呼吸空间;卡片不能盖住自己的出线点。地球转动时,连线起点会自动平滑移向仍可见的高亮部分;整个对象转到背面后,保留球体轮廓上的方向提示,以较淡连线和空心端点区分。卡片位置保持稳定,仍可拖动,接点会随卡片沿边缘调整,无需手动编辑连线。手机上先显示精简卡片,点开后在抽屉中查看完整详情。下拉菜单与工具列表保持原有打开方式。
### 新闻直播源选择
打开媒体面板的直播页,点击当前频道可展开搜索菜单。搜索会检索完整频道库;列表每次加载 50 个频道,滚动到底部自动追加。底部固定显示已加载数量和搜索结果总数,加载失败时可在列表下方重试。默认新闻直播源为 Al Jazeera Mubasher(半岛电视台),使用 HLS 播放地址;其他频道仍取决于各自的播放服务是否可用。
缓冲和可恢复的播放错误不会切换频道。手动选定频道后,本次页面会话始终保留选择;失败时可刷新重试。自动启动的频道只有在致命错误重试耗尽、后台允许自动回退且存在可播放的备用源时才会切换。CCTV-4 内置源使用可直接播放的 WCETV 美洲版 HLS,频道名注明版本,节目编排可能与亚洲版不同;原内置 CDRM 地址不再直接交给通用播放器解码,控制台自定义的其他 CCTV-4 地址不受影响。
播放源信息在加载频道时短暂展开,随后沿用 HUD 的平滑折叠动画自动收起;手动展开和收起使用同一组件。展开高度按实际内容测量,拖动到屏幕底部后向上折叠仍保留动画与底边位置。
### 位置候选采集
算力中心和 BGP 观测站详情卡支持自动采集坐标候选。点击对象后用"自动采集坐标候选"或"重新自动采集坐标"按钮,后端会从源坐标、开放组织注册 API 和在线地理编码中整理候选;常规来源没有候选时使用当前默认 AI Provider 做 LLM factcheck 兜底。BGP 观测站的已存储位置只用于补齐查询上下文,不会作为候选直接返回。
候选可以直接在智能星球预览。算力中心候选点击"保存"后写入 `compute_center_locations` 维表并刷新图层。算力中心图层左上角的通知气泡显示无法渲染的待定位数量;点击查看列表,单条采集候选,或用"一键采用"从上到下保存最高置信候选。没有可用候选的记录会留在列表中,不会被国家中心点或硬编码 hint 伪造位置。
候选可以直接在智能星球预览。算力中心候选点击“保存”后写入 `compute_center_locations` 维表并刷新图层。`precise` 和 `site` 保持源坐标;城市级、estimated、待确认记录重叠时只在展示层按稳定的 source + source_id 分散,源坐标不会改写。tooltip 和详情卡显示“城市级近似位置(分散显示)”。算力中心图层左上角的通知气泡显示完整 unresolved 待定位数量;点击查看列表,单条采集候选,或用“一键采用”从上到下保存最高置信候选。没有可用候选的记录会留在列表中,不会被国家中心点或猜测总部伪造位置。
单个对象的推荐流程:
@@ -297,6 +318,8 @@ AIS 船只图例按船型显示颜色:货轮、油轮、客船、渔船、军
一键定位用于批量处理算力中心待定位队列。它会从列表顶部开始采用最高置信候选;仍没有事实依据的记录会保留在队列中。未开启 WebSearch 时,单个定位和一键定位会置灰,因为位置核验依赖事实查询。
在当前智能星球页面内,关闭候选面板、查看其他对象或切换浏览器标签页不会取消队列;返回候选列表后可继续查看进度和结果。已保存条目不会再次采集,全部完成后仍可通过图层旁的 ✓ 入口查看结果。刷新、关闭页面或跳转离开智能星球会中断尚未完成的队列,已经保存的坐标会保留。
### 设置
设置面板按分类组织:运行、显示、面板、动捕、快捷键、系统。里面包含旋转模式 / 巡航模式 / 动捕模式、巡航模块(BGP/新闻/算力中心/船只/海缆/卫星)、视图设置(卫星显示风格、悬停提示、卫星呼吸闪烁、真实卫星高度、轨迹显示、低缩放圆点、日夜模式、面板显示开关)、动捕调试模式 / 输入源 / 只显示骨骼 / 识别动作白名单、快捷键启用与改键、地球默认大小、地形透明度、重置设置。
@@ -385,3 +408,19 @@ AIS 船只图例按船型显示颜色:货轮、油轮、客船、渔船、军
- [快速开始](/home/ray/dev/linkong/planet/docs/technical/zh/quickstart.md)
- [常见问题](/home/ray/dev/linkong/planet/docs/technical/zh/faq.md)
- [智能星球运维手册](/home/ray/dev/linkong/planet/docs/technical/zh/ops-runbook.md)
## 算力与资源态势
管理员可在控制台“智能星球内容 → 算力对比”维护卡片数据。先在“报告来源”录入报告标题、链接、发布机构、报告年份与页码,再在“年度观测”按年份选择指标、来源和国家数值,最后“保存并发布”。占比以全球为分母,绝对量统一为 FP32 EFlop/s;中美双边份额由同年全球份额计算,区域仅在成员数据完整时汇总。不要混入 FP16/FP64、地区估计值或设施记录数;空缺国家应不录入,明确的零值才填 0。
“创新指数分差”独立维护美国减中国的观测,不反推单国指数。来源被观测引用时不能直接移除;重复指标年份、无效来源、占比总和超过 100% 等内容会拒绝发布。多人同时编辑时,后保存者收到冲突提示,草稿保留以便核对。“恢复默认”需要确认,会恢复随版本提供的报告数据;保存和恢复均有操作审计。同源页面收到发布通知后更新,其他正在展示的地球页每分钟检查一次,后台标签页恢复可见时也会检查。数值仍需人工核验报告,配置链接不会自动抓取报告。
点击算力中心图层行的“算力态势分析”打开配置窗,再开启“显示区域卡片”,地球周围会先绘制到目标位置的连接线,随后数据卡平滑出现;再次转动时不会重播入场。关闭配置窗或按 Escape 仅收起配置,卡片继续显示;再次点击分析按钮可重开配置,关闭显示开关才会停用卡片。区域锚点随地球旋转,卡片保持朝向读者;背面区域有状态提示,不绘制穿过地球的连线。其他图层、新闻和媒体窗口保持可用。
选择器按规范目录分为中国、美国、欧洲、不含中国的亚太和其他五组;这些分组只用于展示,不参与统计分母,可改变参考期与指标口径。双边份额的分母仍固定为美国加中国,其他参与方没有该口径时显示缺失。区域中心只用于卡片定位,不代表设施位置。点击来源查看报告,点击设施按钮使用原有详情卡。手机通过底部抽屉的“算力”页查看紧凑地区概览:同一列表中的短卡采用手风琴,重复点击卡头收起,点击另一张先收起旧卡,最多同时展开一张长卡。展开详情保留趋势、区域成员、设施、媒体和来源,不设地区二级页面;设置仍是单独按需页。
默认主指标为全球计算设备总规模,拆分基础计算、智能计算和超级计算,观察期为 2020—2023。已核对的绝对量为 2022 年 302(证据 `caict2023`)和 2023 年 435(证据 `caict2024`)。日本全球份额为 2020/2021/2022/2023 = 6%/5%/4%/4%。产品分组 APAC、OTHER 不能直接等同报告里的日本或“其他”;EUROPE 成员集合尚未与报告对齐,仍显示缺失。双边份额分母固定为美国加中国,不受日本加入影响。主指标缺失时可显示最近有报告的年份(当前到 2023),明确选择 2026 仍显示缺失,不回填最近值。不可比性未确认时不展示同比变化。国家报告指标与公开设施样本独立,覆盖 count 只保留在辅助证据区,不能当作算力值或全球覆盖率。TOP500 与 Epoch 可能重复收录,来源记录数不代表全球设施总数。默认报告核验日期为 2026-09-22;后台发布后以维护的核验日期和实际观测为准。独立 Epoch 份额模式尚未实现。
当前原型最多同时显示六张内容卡。主卡围绕当前选择的算力指标:大号百分比或带单位的绝对量、观测年份、历史上下文和来源入口。缺少可比数据时明确显示“暂无可比数据”;没有观测值时不绘制空趋势图。设施记录和覆盖信息仍完整保留,但移到展开后的辅助区;样本数只是证据上下文,不能当作算力值或全球覆盖率。区域预览可以显示成员数,长列表仅在需要时展开。桌面卡片约宽 268 像素,高度随内容自适应;展开卡片支持用户调整至最高 480 像素。手机手风琴单卡也保持同一占比/绝对量语义,不把设施条数提升为主数字。位置和尺寸保存于本地,并可恢复自动布局。拖动或展开后先留出阅读时间;地区随地球明显转到另一侧或上下方后,卡片才会在能减少连线交叉、绕行的情况下成组平滑换位。鼠标正在卡片上操作时不移动,手动位置不会被立即抢回,也不会永久阻止必要调整。图片、视频等内容可以沿用内容区布局,当前尚未接入这些媒体数据。空间不足时卡片可以经过球体前景移动,不再靠瞬移避让或缩成小标签。
地图卡片开关关闭时仍会加载概览摘要,打开后共享地图加载事件;新请求会取消旧请求。抽屉事件使用 `composedPath()`,即使通过 innerHTML 换节点也不会误收起当前卡片。复用共享 HUD 折叠面板动画,并尊重 reduced-motion。设施名单使用最高 240px 的内部滚动区域,名称左对齐并正常换行,每行至少保留 42px 触控高度。

View File

@@ -60,6 +60,7 @@
| `traffic_stats` | 流量统计 | Cloudflare Radar 流量统计 |
| `as_stats` | 自治系统统计 | Cloudflare Radar AS 统计 |
| `compute_center` | 算力中心 | 智能星球聚合展示对象 |
| `compute_intelligence` | 算力与资源态势 | 国家观测、公开设施样本、历史与来源专题 |
| `generic` | 通用数据 | 通用采集输出 |
| `generic_records` | 通用记录 | 通用映射输出 |

View File

@@ -1,6 +1,24 @@
# planet.sh 启动性能优化
## 背景
## Windows 桌面启动器
启动窗口优先使用 Windows Terminal,并显式运行 PowerShell 7(`pwsh.exe`),不依赖 Terminal 默认配置文件。未安装 PowerShell 7 时回退到 Windows PowerShell;未安装 Terminal 时直接启动可用的 PowerShell。CMD 初始引导同样优先 PowerShell 7。普通入口请求提权,已经以管理员运行的入口也会进入新 Terminal 窗口,但不会再次请求提权;子窗口校验权限后直接执行 WSL,避免递归打开窗口。重启完成后仍保留原有 WSL shell。
首次在 Windows 文件资源管理器中打开实际项目目录,双击仓库根目录的 `planet.cmd`。它从自身位置识别 WSL 发行版和 Linux 项目目录,在 Windows 当前用户的实际桌面生成 `Planet.cmd`,随后进入正常重启流程。支持 `\\wsl$\发行版\...` 与 `\\wsl.localhost\发行版\...`;仓库位于 Windows 本地磁盘时,使用默认 WSL 发行版及 `wslpath` 映射。系统桌面位置由 Windows 获取,兼容不同用户名和 OneDrive 等桌面重定向;WSL、zsh 及项目依赖仍须先准备好。
后续双击桌面副本,使用已绑定的发行版与目录,申请原有管理员权限,并调用 `planet.sh restart --allow-lan`。不写入测试端口;默认前端 3000、后端 8000、AI Provider 8010,显式 `DEFAULT_*_PORT` 环境配置仍有效。成功后按实际 `ports.env` 端口打开 Earth,终端保留 WSL shell;失败只显示错误,不打开成功页面。源文件负责生成入口,`scripts/windows/planet-launcher.ps1` 负责 Windows 路径、桌面安装与提权,`scripts/windows/restart.zsh` 只衔接现有重启流程。
只生成或检查副本而不重启服务,可在 Windows 中运行 `planet.cmd --install-only`。从仓库再次运行会刷新同一目标的副本;已有其他项目或非托管 `Planet.cmd` 时另存带目标标识的文件,不覆盖。目录移动后从新位置重新运行仓库入口,旧副本不会猜测新目录。桌面副本依赖项目中的脚本,不是可脱离仓库运行的安装包。
不要把仓库原始 `planet.cmd` 单独复制到桌面,它需要相邻的 `scripts/windows/` 才能完成首次生成。换电脑时应从新电脑的项目目录重新生成,不能直接沿用旧电脑的桌面副本。服务器的 `/home/linkong/planet` 与个人电脑的其他路径均可使用,不必修改提交到 Git 的路径。应在拥有目标 WSL 发行版的 Windows 账号下运行;若提权时改用其他账号,其 WSL 发行版注册可能不同。首次生成不要求管理员权限,真正重启时才申请提权。
Windows 回归命令为 `powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts\harness\test_windows_launcher.ps1`,覆盖路径识别、特殊字符、生成副本及名称冲突。`scripts/harness/test_windows_restart.py` 用临时替身验证重启参数、成功打开页面和失败保留 shell,已接入 quick-check,不启动实际服务。
## 启动背景
日常启动使用 `zsh ./planet.sh start --non-motion-agent`;新环境首次准备才需要 `init`。排查耗时时,应区分首次依赖下载、容器就绪和应用初始化,结合阶段日志时间判断。
当前启动流程在准备 AI Provider 镜像前验证后端实际数据库连接,端口映射缺失时保留数据卷重建一次。后端进程退出,或 Uvicorn 日志出现应用初始化失败、ASGI 加载失败、导入或语法错误时,会立即停止等待和重复启动;正常的慢启动仍保留原有等待预算。AI Provider 直接探测宿主机 `/health`,无需再等待 Docker 周期性健康检查首次运行。
`planet.sh` 管理所有服务的启动/停止/重启。原有实现存在以下问题:
@@ -37,27 +55,9 @@ write_ai_provider_build_stamp() {
}
```
### fingerprint 计算提速
### fingerprint 检查范围
原实现对整个 `aiprovider/` 打 tar 包再算 SHA,大目录下耗时可达数秒。改为 `find + stat`(只读文件元信息,不读内容):
```bash
compute_ai_provider_build_fingerprint() {
find aiprovider \
-type f \
! -path '*/__pycache__/*' \
! -name '.env' \
! -name '.env.*' \
! -name '*.pyc' \
! -name '*.pyo' \
| LC_ALL=C sort \
| xargs -r stat --format="%Y %s %n" 2>/dev/null
sha256sum docker-compose.yml docker-compose.simple.yml 2>/dev/null
python3 "$SCRIPT_DIR/scripts/compute_aiprovider_dependency_fingerprint.py" 2>/dev/null
}
```
速度提升约 10 倍(大量小文件场景),误报率相同(mtime+size 变化 ≡ 文件被修改)。
当前指纹使用内容 SHA,覆盖 `aiprovider/`、Dockerfile、根依赖清单与锁文件,以及代理服务相关依赖信息;不会遍历前端资源或下载数据。`.env` 配置不参与镜像内容指纹。构建标记只用于判断是否需要构建,Dockerfile 中的指纹标签位于依赖安装之后,以保留前置依赖层缓存。
`.env` 和 `.env.*` 被排除在 fingerprint 外。它们属于运行期配置,不应该因为修改模型、密钥或 Base URL 触发镜像重建。
@@ -85,16 +85,18 @@ Dockerfile 也从全仓复制改为只复制 AI Provider 代码:
```dockerfile
COPY pyproject.toml uv.lock /app/
RUN --mount=type=cache,target=/root/.cache/uv \
uv sync --frozen --no-dev
uv sync --frozen --only-group aiprovider
COPY aiprovider /app/aiprovider
```
`uv sync` 使用 BuildKit cache mount 后,首次构建仍可能受网络影响;后续构建会复用 `/root/.cache/uv`,依赖下载不再重复从零开始。
`aiprovider` 依赖组在根 `pyproject.toml` 定义,并由同一份 `uv.lock` 锁定,只安装 FastAPI、HTTP 客户端、配置读取和 ASGI 服务所需依赖。镜像不安装后端采集或 OpenCV / MediaPipe 动捕依赖。构建指纹标签放在依赖安装和代码复制之后,指纹改变不会单独使依赖层缓存失效。容器直接启动已安装的 `.venv/bin/python`,运行时不再执行 `uv sync`。修改代理服务的依赖时,应同步更新该依赖组和锁文件,并验证镜像导入及 `/health`。
### 运行期配置来源
`planet.sh` 启动 AI Provider 前会生成受当前用户保护的运行期 env-file,并把它传给 Compose 或手动 `docker run` fallback。默认路径位于 `${XDG_STATE_HOME:-$HOME/.local/state}/planet/aiprovider_runtime.env`。配置优先来自:
`planet.sh` 启动 AI Provider 前会生成受当前用户保护的运行期 env-file,并把它传给 Compose;Compose 失败时保留错误并停止,不再回退手动启动。默认路径位于 `${XDG_STATE_HOME:-$HOME/.local/state}/planet/aiprovider_runtime.env`。配置优先来自:
1. `aiprovider/.env`
2. `~/.zshrc` 中简单的 `export AI_...=...` 或 `AI_...=...` 行
@@ -119,7 +121,7 @@ fingerprint 一致时不执行 `docker compose build`,而是:
docker start planet_aiprovider # 启动已存在的容器,几秒内完成
```
`docker stop` 停容器,不删镜像;`cleanup_exit_containers` 删已退出容器,不删镜像。下次 `docker start` 会从现有镜像直接创建并启动容器。
`docker stop` 停容器,不删镜像。`start` 和 `restart` 保留已停止的容器,不再扫描删除全机已退出容器;未变化的 AI Provider 可直接复用,数据库仍由 Compose 同步配置。只有需要更新镜像或容器配置时才按原有流程重建。
## 问题二:杀端口速度慢

View File

@@ -33,9 +33,9 @@ docker buildx version
## 数据库初始化与连接检查
`init` 会先通过 Compose 同步 PostgreSQL / Redis 容器配置,包括已有容器的端口映射;仅执行 `docker start` 无法应用配置变化。Compose 同步失败时会保留具体错误,例如端口被占用,不会继续复用旧容器并报告成功。
`init` 和 `start` 会先通过 Compose 同步 PostgreSQL / Redis 容器配置,包括已有容器的端口映射;仅执行 `docker start` 无法应用配置变化。Compose 同步失败时会保留具体错误,例如端口被占用,不会继续复用旧容器并报告成功。
容器内部的 `pg_isready` 只检查服务是否接受连接,不能证明宿主机上的后端使用正确地址和密码。容器健康后,`init` 通过 `scripts/check_database_connection.py` 读取与后端相同的有效 `DATABASE_URL`,检查本地 PostgreSQL 的实际发布端口并执行只读 `SELECT 1`;通过后才显示“数据库服务已就绪”并创建表和默认数据。
容器内部的 `pg_isready` 只检查服务是否接受连接,不能证明宿主机上的后端使用正确地址和密码。容器健康后,`init` 和后端启动流程通过 `scripts/check_database_connection.py` 读取与后端相同的有效 `DATABASE_URL`,检查本地 PostgreSQL 的实际发布端口并执行只读 `SELECT 1`。启动流程在准备 AI Provider 镜像之前完成此检查;失败会立即停止。`init` 通过检查后才创建表和默认数据。
- 如果本地实际端口映射仍缺失或不匹配,脚本会保留数据卷,按 Compose 配置重建一次 PostgreSQL 并重新检查;再次失败就停止。
- 认证、库名或网络错误会在建表前停止,诊断只显示目标主机、端口和库名,不输出密码、完整连接串或驱动异常原文。
@@ -67,6 +67,8 @@ docker buildx version
两个默认账号 `email_verified` 为 `true`,可直接登录控制台。任何在该列表之外的账号都必须走公开注册 + 邮箱验证流程(见使用手册),或用 `./planet.sh createuser` 命令式创建。
独立预览库若跳过应用 lifespan,不会自动初始化默认账户。默认账号返回 401 时,先确认后端实际连接的库及账户是否存在;仅在确认缺失后,对该库执行现有 `ensure_default_admin_user`,不会重置已有账户密码。登录验证应同时检查 `/api/v1/auth/login` 和携带访问令牌的 `/api/v1/auth/me`:身份加载保留完整的 `email_verified`、`created_at` 等字段,响应接受既有默认账户的 `.local` 邮箱,注册和修改邮箱仍严格校验。下表 AUTH 标记用于已人工确认的诊断,不能仅凭通用 401/500 推断原因。
可指定端口:
```bash
@@ -274,6 +276,15 @@ url = "https://mirrors.tuna.tsinghua.edu.cn/pypi/web/simple/"
default = true
```
启动和重启默认按指纹判断是否需要构建。需要明确复用已有镜像时,可对本次命令增加 --no-build:
```bash
./planet.sh start --no-build
./planet.sh restart -a --no-build
```
该选项只影响 AI Provider,不会关闭其他服务的启动检查;缺少本地镜像时直接报错,也不会把旧镜像的指纹更新成当前代码。已有 Compose v2 时,构建、启动和能力检查只使用 v2,失败直接保留错误;只有未检测到 v2 时才考虑 v1。
构建较慢时按层排查:
| 现象 | 常见原因 | 处理方式 |
@@ -291,6 +302,101 @@ default = true
OTP 一次性验证码走 Redis,key 格式 `otp:{purpose}:{email}`,TTL 600 秒。错误尝试 5 次后该 key 失效;重发冷却 60 秒,由 `otp_rate:{purpose}:{email}` 控制。
## 错误原因对照表
`planet.sh` 默认只保留状态主行下的具体错误;使用 `--verbose` 时,`log_error` 从下表读取错误编号、原因和处理建议;AI Provider 构建失败还会匹配完整构建日志。匹配按表中顺序进行,不区分英文大小写,分号分隔多个字面关键片段,具体错误优先于汇总错误。错误编号不是进程退出码,失败仍返回非零状态。
确认新的故障原因时,必须更新中英文表,补充稳定编号、可辨识的日志片段和回归用例,再接入脚本。未确认的错误归入 `P_UNKNOWN`,不得自动把未知日志当成已验证原因写入手册。表由 `scripts/lib/error-diagnostics.zsh` 读取,不要在单元格中使用竖线;`scripts/harness/test_error_diagnostics.py` 验证匹配、双语编号和输出一致性。
<!-- planet-error-catalog:start -->
| 错误编号 | 匹配片段(分号分隔) | 原因/已确认的故障现象 | 处理建议 |
| --- | --- | --- | --- |
| P_WINDOWS_LAUNCHER_TARGET_MISSING | P_WINDOWS_LAUNCHER_TARGET_MISSING | 桌面启动器绑定的 WSL 项目目录不可访问或缺少启动脚本,常见于项目移动、发行版不可用或复制旧机器的副本。 | 从当前项目目录运行仓库 planet.cmd 重新生成本机桌面副本,检查 WSL 发行版;不修改成其他电脑的固定路径。 |
| P_PROXY_EXTERNAL | PLANET_PROXY_EXTERNAL | 当前 Docker 代理由其他配置管理,或已被人工修改,不能安全自动覆盖。 | 检查 daemon.json、systemd 代理及 Planet 管理记录,确认归属后再调整;脚本保留现有配置。 |
| P_AUTH_BOOTSTRAP_MISSING | DEFAULT_LOGIN_USER_MISSING | 已确认当前后端数据库缺少默认账户,常见于跳过初始化的隔离预览。 | 核对数据库归属后运行现有默认账户初始化,不重置已有账户或清空数据库。 |
| P_AUTH_PROFILE_INVALID | AUTH_IDENTITY_RESPONSE_INVALID | 已确认登录成功但身份响应字段缺失,或响应错误拒绝既有本地邮箱。 | 从真实用户记录加载完整身份,区分响应序列化与注册输入校验;运行默认账户身份回归。 |
| P_EARTH_POLICY_MISSING | EARTH_DISPLAY_POLICY_MISSING | 后端镜像或发布目录遗漏共享地理规则,算力路由导入时无法读取必需文件。 | 同步更新 Dockerfile 与 .dockerignore,交付 config/earth-boundary-pov-policy.china-v1.json、国界构建脚本及审核几何;重新构建镜像,不清空数据库。 |
| P_PROXY_CHANGED | PLANET_PROXY_CONFIG_CHANGED | 检测后 Docker 代理配置发生了变化。 | 等待其他配置操作结束后重试,避免覆盖并发修改。 |
| P_PROXY_ROLLBACK | PLANET_PROXY_ROLLBACK_FAILED | 自动代理更新失败后,Docker 或原有容器未能完成恢复。 | 检查 Docker 服务日志和容器状态;原配置已还原,必要时依据 daemon.json.planet-proxy.bak 手动恢复服务。 |
| P_PROXY_CONFIG | PLANET_PROXY_CONFIG_FAILED;Docker 构建代理检测失败;Docker 构建代理更新失败 | 自动代理检测、配置校验或服务更新失败。 | 检查 Python 3、curl、Docker 状态和 sudo 权限;配置更新会尝试回滚,代理凭据不会写入错误输出。 |
| P_PROXY_NO_ROUTE | PLANET_PROXY_NO_ROUTE | 未找到能访问构建所需仓库的主机代理,直连探测也未通过。 | 恢复可用代理或修复直连网络、DNS及仓库地址后重试;不要把禁止构建当成网络修复。 |
| P_REGISTRY_RATE_LIMIT | 429 Too Many Requests;toomanyrequests;pull rate limit | 镜像仓库已响应,但请求频率或拉取配额触发限流;具体限制需依据仓库响应确认。 | 停止反复重试,按上游提示等待;若为匿名拉取配额,核对 docker login 身份及额度。不要把限流误判为代理不可达。 |
| P_DNS | no such host;temporary failure in name resolution;could not resolve host | 域名解析失败,尚不能确定是 DNS 配置还是上游解析异常。 | 检查 Docker 所在环境的 DNS 和代理解析;对比终端与 Docker 的解析结果。 |
| P_TLS_CERT | x509:;certificate verify failed;certificate signed by unknown authority | TLS 证书校验失败。 | 检查系统时间、证书链和代理 CA;安装可信 CA,不要关闭证书校验。 |
| P_PROXY_AUTH | proxy authentication required;407 proxy | 代理要求认证,当前请求未通过认证。 | 检查 Docker 服务的代理凭据和代理端权限,不要把凭据写入仓库或日志。 |
| P_NETWORK_TIMEOUT | i/o timeout;tls handshake timeout;context deadline exceeded;deadlineexceeded | 网络连接或 TLS 握手超时,单凭日志不能认定是代理、DNS 或 IPv6 中的哪一项。 | 对比直连与代理请求;检查 Docker 服务自身的代理、DNS 和 IPv6 路由,终端代理不等于 Docker 服务代理。 |
| P_CONNECTION_REFUSED | connection refused | 目标地址拒绝连接,服务可能未监听或地址、端口配置不匹配。 | 检查被拒绝的目标是代理、数据库还是镜像仓库,再确认监听端口和服务状态。 |
| P_REGISTRY_AUTH | pull access denied;unauthorized:;insufficient_scope;denied: requested access | 镜像仓库拒绝访问或当前身份无拉取权限。 | 核对镜像名、仓库权限及 docker login 使用的身份。 |
| P_IMAGE_TAG | manifest unknown;manifest not found | 镜像仓库中找不到指定的镜像清单或标签。 | 核对 PYTHON_IMAGE、UV_IMAGE 或其他镜像标签,确认目标架构受支持。 |
| P_DISK_FULL | no space left on device | 磁盘空间或 inode 不足。 | 检查 df -h、df -i 和 docker system df;确认用途后定向清理,不要删除数据库卷。 |
| P_DOCKER_SOCKET | permission denied while trying to connect;刷新组权限后仍无法访问 Docker socket | 当前用户无法访问 Docker socket。 | 检查 socket 属组和 docker 组成员资格;执行 ./planet.sh init 配置权限后重新打开终端。 |
| P_DOCKER_SERVICE | 没有可用的 docker.service;Docker Engine 启动失败;cannot connect to the docker daemon;无法连接 daemon | Docker 服务未就绪,或客户端无法连接当前 endpoint。 | 检查 systemctl status docker、docker context ls 和 journalctl -u docker.service;不要把连接失败当成未安装。 |
| P_DOCKER_DESKTOP | 检测到 Docker Desktop,但当前 WSL | Docker Desktop 或当前 WSL 集成不可用。 | 启动 Docker Desktop,并启用当前发行版的 WSL Integration。 |
| P_DOCKER_ENDPOINT | 当前 Docker 使用其他 context 或远程/rootless endpoint | 当前远程或 rootless Docker endpoint 不可用。 | 检查 docker context ls、DOCKER_HOST 和目标服务;不要自动替换成本地引擎。 |
| P_BUILDX | 未检测到 docker buildx;buildx 0.17;buildx >=;buildx v;当前 docker compose 不支持 build;安装后 Docker CLI、Compose v2 | Docker 构建插件缺失、版本不足或构建能力不可用。 | 检查 docker buildx version 和 docker compose version,按项目要求安装或升级相应插件。 |
| P_COMPOSE_MISSING | 未检测到可用的 Docker Compose | 未发现可用的 Compose 命令。 | 安装 Compose v2 插件并验证 docker compose version;已有 v2 执行失败时不回退 v1。 |
| P_LOCAL_IMAGE_MISSING | 已指定 --no-build,但本地没有 AI Provider 镜像 | 禁止构建时,本地没有可复用的 AI Provider 镜像。 | 先成功构建或导入镜像,再使用 --no-build;该选项不会自动构建。 |
| P_SUDO | 缺少 sudo | 自动安装或配置系统依赖所需的提权工具不可用。 | 由管理员安装 sudo 并授予必要权限,或预先安装依赖。 |
| P_UNSUPPORTED_OS | Docker 自动安装目前支持;未识别系统包管理器 | 当前系统不在脚本自动安装的支持范围内。 | 按系统官方方式安装依赖,再重新执行脚本。 |
| P_LOCKFILE_CHANGED | 修改了 uv.lock | 依赖准备意外修改了锁文件。 | 检查依赖清单与锁文件的一致性;新环境使用 frozen 安装,不要隐式更新锁文件。 |
| P_DEPENDENCIES | 安装失败;安装后仍不可用;安装完成后仍未找到;未找到 .venv/bin/python;自动安装后仍无法解析运行时;未找到 Vite Bun 入口;缺少 mediapipe/opencv-python;仍无法导入 mediapipe/opencv-python;需要 openssl | 必需依赖安装失败、缺失或未进入当前运行环境。 | 查看对应安装日志,检查网络、软件源和 PATH;前端使用 Bun,Python 使用项目 uv 环境。 |
| P_ARGUMENT | 未知参数;非法端口;需要端口号;需要逗号分隔;--motion-agent-mode 需要;--motion-agent-wsl-usbipd-busid 需要;用法: ./planet.sh | 命令或参数不符合脚本支持的格式。 | 查看 ./planet.sh 用法,修正参数和值后重试。 |
| P_DB_PORT_OCCUPIED | PLANET_DB_PORT_OCCUPIED | Planet 的数据库或 Redis 宿主机端口已被其他容器或服务占用。 | 根据错误中的占用者核对 docker ps;调整 backend/.env 的 DATABASE_URL 端口或 REDIS_PORT(进程 REDIS_URL 优先),planet.sh 会同步 Compose 映射;不停止其他项目或删除数据卷。 |
| P_PORT | 地址已被占用;端口仍不可用;清理失败,请检查占用进程;port is already allocated;address already in use | 请求的端口被占用,或在宿主机/外部环境中不可绑定。 | 检查 ss 和 Windows Get-NetTCPConnection,确认占用者后调整端口或停止对应服务。 |
| P_CAMERA | live 模式缺少可用摄像头;未找到可打开并能读帧的摄像头 | Motion Agent 无法取得可用摄像头画面。 | 检查设备、权限和 WSL USB 转发;无需摄像头时使用 --non-motion-agent 或明确选择 dry-run。 |
| P_DB_CONNECTION | 后端数据库连接检查失败 | 后端实际数据库连接或发布端口检查未通过。 | 查看连接探测的具体输出,核对 DATABASE_URL、凭据、库名和端口;容器健康不等于后端能连接。 |
| P_DB_START | 数据库启动失败;数据库重启失败;PostgreSQL 启动失败;数据库初始化失败 | 数据库启动、健康检查或初始化未完成。 | 检查 PostgreSQL、Redis 容器日志及具体数据库错误;不要通过删除数据卷排障。 |
| P_BACKEND_START | 后端进程已退出;后端启动失败 | 后端进程退出或未通过健康检查。 | 查看 ./planet.sh log -b,优先处理导入、配置、数据库连接或应用初始化错误。 |
| P_AI_START | AI Provider 启动失败 | AI Provider 容器未正常启动或未通过健康检查。 | 查看 ./planet.sh log -a,检查运行配置、端口和容器退出原因。 |
| P_FRONTEND_START | 前端启动失败 | 前端未通过启动健康检查。 | 查看 ./planet.sh log -f,检查 Bun、依赖、Vite 入口和端口占用。 |
| P_MOTION_START | Motion Agent 启动失败 | Motion Agent 未正常启动。 | 查看 ./planet.sh log -m,检查依赖、摄像头和输入模式。 |
| P_ACCOUNT_INPUT | 用户名不能为空;密码不能为空;密码长度不能少于;两次输入的密码不一致 | 创建用户时输入不满足校验要求。 | 按提示重新输入用户名及满足长度要求且一致的密码。 |
| P_HTTP_HEALTH | 不可访问: | 指定 HTTP 端点未通过访问检查。 | 检查目标 URL、服务监听、防火墙和本机/局域网路由;HTTPS 还需检查证书信任。 |
| P_DOCKER_MIRROR_FAILED | PLANET_DOCKER_MIRROR_FAILED | 备用镜像的 manifest 校验或实际拉取失败,未能完成回退。 | 查看具体仓库错误;检查备用站可用性、镜像白名单、架构和 digest,不把首页可达当成镜像可用。可配置 PLANET_DOCKER_MIRROR_PREFIX 或关闭自动回退后修复原源。 |
| P_DOCKER_COMMAND_TIMEOUT | PLANET_DOCKER_COMMAND_TIMEOUT | 当前 Docker 命令超过脚本的等待上限;仅凭超时不能确定是代理、网络还是本地引擎问题。 | 根据阶段名称、最近输出和日志定位;无输出时检查对应 Docker 命令及 daemon 日志,确认仍有进展后再调整该阶段超时。 |
| P_COMPOSE_FAILED | Docker Compose 执行失败;docker-compose v1 执行失败 | 所选 Compose 命令失败,尚未识别更具体原因。 | 查看命令前面的原始错误;已有 Compose v2 时修复其错误,不安装 v1 作为回退。 |
| P_BUILD_FAILED | AI Provider 镜像构建失败;failed to solve | 镜像构建失败,现有证据未匹配已知的具体原因。 | 查看 aiprovider_build.log 中最早的具体错误,确认原因后补充本表和回归用例。 |
| P_UNKNOWN | — | 尚未归类,不能从现有证据确认原因。 | 保留完整错误和执行命令;确认根因后补充本表、中英文说明及回归用例。 |
<!-- planet-error-catalog:end -->
### Docker 等待进度与超时
Docker 检查、Compose 启动、数据库连接检查和 AI Provider 构建的最新步骤更新在现有状态主行下面的副行,不打印额外的开始/完成记录、版本号或日志路径。成功后清除副行;静默等待时保留当前步骤并更新耗时。失败时保留具体错误并退出,后续汇总错误和清理通知不会覆盖它。Compose 执行失败不再重复启动;容器正常启动后的健康等待继续沿用原有预算。
`PLANET_DOCKER_PROBE_TIMEOUT` 默认 15 秒,`PLANET_COMPOSE_TIMEOUT` 默认 180 秒,`PLANET_DOCKER_BUILD_TIMEOUT` 默认 900 秒,均为单次命令总时限,可设为正数秒。超时终止本次 CLI 进程组;daemon 可能已接受请求,仍应检查实际容器状态。Compose 检查超时不会回退到 v1。
详细日志位于 `${PLANET_STATE_DIR:-${XDG_STATE_HOME:-$HOME/.local/state}/planet}`:Compose 使用 `compose.*.log`,构建使用 `aiprovider_build.log`,连接检查使用 `database-check.*.log`。日志权限为 600,URL 中的代理用户名和密码会被遮盖。`scripts/docker_command.py` 负责记录、计时和进程清理,终端渲染由 `planet.sh` 统一负责。直接执行系统 `docker compose` 不会加载此包装器。
### 演示启动的数据库排障
已确认的冲突案例是另一个项目的 PostgreSQL 和 Redis 分别占用 `5432`、`6379`。Planet 容器可能显示健康,却没有实际发布宿主机端口;后端也可能误连另一套数据库并报认证失败。`scripts/check_database_connection.py --ports` 在启动前检测占用者,冲突时立即停止,不通过反复重建解决端口争用。
需要共存时,在 `backend/.env` 中将 `DATABASE_URL` 的端口改为可用端口(例如 `15432`),将 `REDIS_PORT` 改为可用端口(例如 `16379`)。保留账号、密码和库名;若进程设置了 `REDIS_URL`,也必须同步它。`planet.sh` 从后端实际配置导出 `PLANET_POSTGRES_PORT`、`PLANET_REDIS_PORT`,重建容器映射时保留命名数据卷。直接使用 Compose 时需显式提供这两个变量。数据库端口只发布到 `127.0.0.1`。
其他错误按证据分别处理:认证失败先核对连接目标和已有卷的密码,修改 `POSTGRES_PASSWORD` 不会重置已有密码;端口映射缺失且没有冲突时允许保留卷重建一次;镜像下载的 DNS、TLS、代理认证和限流错误按上表各自处理。正常下载或依赖安装的慢进度不算失败,代码与依赖未变化时后续启动应复用镜像。
### Docker 镜像的临时备用源
清华 [Docker CE 软件仓库](https://mirrors.tuna.tsinghua.edu.cn/help/docker-ce/) 只提供 Docker 安装包,不是 Docker Hub 镜像服务。Docker 镜像回退默认使用 [DaoCloud 公开镜像服务](https://github.com/DaoCloud/public-image-mirror),清华 PyPI 回退仍只用于本机 Python 依赖。
构建前先检测原仓库的代理和直连;均不可达时验证备用路径。原路径探测成功、实际拉取仍发生 DNS/连接超时等网络错误时,也只回退一次。认证失败、证书错误、限流、端口冲突及 Dockerfile 内部依赖安装失败不会触发镜像源切换。所有路径失败后保留错误副行并退出。原始失败日志保留为同目录的 `*.primary.log`。
备用站必须通过具体镜像的 manifest 检查,执行时锁定返回的 `sha256` digest;输入本身指定 digest 时必须一致。基础镜像只临时传入本次构建参数,原配置、构建指纹和最终镜像名不变。数据库只为本次请求中缺少的镜像执行备用拉取,成功后加回原始本地标签,再执行原 Compose 启动。不会修改全局 `registry-mirrors` 或源码配置,已有命名数据卷不变。
`PLANET_DOCKER_MIRROR_FALLBACK=0` 关闭回退;`PLANET_DOCKER_MIRROR_PREFIX` 默认 `m.daocloud.io`,可设置为可信镜像服务的 `registry/path` 前缀;`PLANET_DOCKER_MIRROR_TIMEOUT` 默认 120 秒,用于基础镜像验证。仅映射 Planet 使用的公开 Python、uv、PostgreSQL 和 Redis 仓库,自定义/私有镜像不会被自动送往公共站。公共镜像服务有白名单、限流和缓存延迟,可变标签可能滞后;需要严格固定版本时应使用 digest,演示前应完成构建并保留本地镜像。
`scripts/docker_mirror.py` 负责映射、manifest 校验及数据库镜像拉取,`scripts/lib/docker-mirror.zsh` 负责网络错误筛选和临时参数;它们复用现有命令进度与超时执行器。回归覆盖私有仓库保护、digest 不一致、非网络错误、一次回退与原配置保留。
### Docker 服务代理与构建网络
终端的 HTTP_PROXY / HTTPS_PROXY 不会自动配置已经运行的 Docker 服务。若终端通过代理能访问镜像仓库,而 Docker 拉取超时,应分别验证代理连接、直连和 Docker 服务的实际代理设置。探测收到 HTTP 401、403 或 429 表示仓库已响应,不等于已获得镜像拉取权限或剩余额度;认证及限流仍由实际构建验证,不能据此把可达代理切换掉。
实际构建 AI Provider 镜像前,脚本自动读取当前环境的 HTTPS_PROXY、HTTP_PROXY、ALL_PROXY(含小写形式),依次验证 HTTP/HTTPS 代理能否访问 PYTHON_IMAGE、UV_IMAGE 对应的仓库,并尊重 NO_PROXY。有可用代理才为本地 Linux Docker Engine 设置代理;没有代理或代理不可用时测试直连并清除脚本管理的旧代理。原源两种路径都不可用时继续检测备用镜像路径;备用路径也不可用时,用 P_PROXY_NO_ROUTE 明确停止。没有代理且 Docker 原本也是直连时,不新增代理配置。指纹命中跳过构建或使用 --no-build 时,不做联网探测;这不是后台监控,代理启停在下一次实际构建时检测。
`scripts/docker_proxy.py` 管理 `/etc/docker/daemon.json` 的 proxies,归属记录保存在仅 root 可读写的 `/etc/docker/planet-proxy-state.json`。不覆盖管理员配置、systemd 代理或人工修改过的代理。配置相同不提权、不重启;需要修改时使用现有 sudo 流程,保留其他 Docker 设置,备份到 daemon.json.planet-proxy.bak,校验后重启 Docker 并启动原先运行的容器。失败时回滚;恢复后应检查容器健康。代理凭据只经环境或受限文件传递,不输出到日志。Docker Desktop、远程和 rootless Docker 沿用自身设置,不修改本机 daemon.json。代理地址从环境读取,不硬编码某台机器的端口。
AI Provider Dockerfile 使用 BuildKit 内置解析器,避免额外拉取 docker/dockerfile 解析器镜像;Python、uv 基础镜像和依赖下载仍需网络。--no-build 只用于明确复用本地镜像,不是构建网络错误的修复。构建错误日志位于 `${XDG_STATE_HOME:-$HOME/.local/state}/planet/aiprovider_build.log`;重启恢复后的服务可用 ./planet.sh health 检查。
## 故障排查顺序
```bash
@@ -372,3 +478,13 @@ uv run pytest backend/tests/test_otp_service.py
- [系统服务控制](/home/ray/dev/linkong/planet/docs/technical/zh/backend-system-service-control.md)
- [Docker + Compose + Buildx 升级](/home/ray/dev/linkong/planet/docs/technical/zh/ops-docker-compose-buildx-upgrade.md)
- [数据采集系统](/home/ray/dev/linkong/planet/docs/technical/zh/backend-collectors.md)
## Earth 多底图升级交付
从 0.74.7 升级这批改动不新增数据库迁移或依赖锁文件。开发环境按原有 Bun 启动流程重启前后端;静态或容器部署必须重新构建前端及相关镜像,不能只 pull 后继续使用旧 dist。新模块、四张运行底图、国旗、基础国界和共享规则均需进入同一提交;仅提交已跟踪文件会漏掉新资源。
旧的 `frontend/public/earth/assets/8k_earth_daymap.jpg` 路径保留但内容改为 Natural Earth I;0.74.7 的原始图片逐字节保存在 `frontend/public/earth/assets/maps/archived-original-20260921-8k.jpg`,对应菜单“原始底图”。拉取和启动不会删除地图归档。旧设置没有地图选择字段时采用“原始底图”默认值。`resources/earth/candidates` 是来源及对照资料,不属于页面加载路径;不要把这些原图作为前端运行依赖。
高精国界产物位于被 Git 忽略的生成目录,不会随拉取自动交付。旧产物缺少新规则元数据时会被拒绝,页面使用已交付的政策基础国界;需保持高精效果时,应按既有国界构建流程重新生成并发布对应静态产物。底图分辨率不受这一回退影响。
容器资源回归使用 `PLANET_HARNESS_DOCKER_SMOKE=1 .venv/bin/python scripts/harness/test_backend_runtime_assets.py`。它通过真实 Docker 构建上下文验证生产 COPY 规则、共享规则/几何及构建脚本,同时检查环境文件未打包;不启动整套服务。

View File

@@ -33,7 +33,7 @@
进入 `/admin` 仪表盘后,建议按这个顺序熟悉控制台:
1. `/collection-management?section=collector_credentials`:选一个采集器,点插头图标做连接测试。免费 collector(开源 BGP 等)通常直接可用;像 `AISStream`、`BarentsWatch` 这类需要凭证的,需要先填 API Key/Client Secret
2. `/ai?section=integrations`:填一个 LLM provider(例如 `minimax` / `openai`)、模型名、Base URL、API Key,点 Base URL 末端的插头测试连接。WebSearch / OCR 工具可选
2. `/ai?section=integrations`:填一个 LLM provider(例如 `minimax` / `openai`)、模型名、Base URL、API Key,点 Base URL 末端的插头检查模型目录;实际生成在 Playground 验证。右上角刷新图标会更新并保存可选模型目录,同时保留当前表单;选中新模型后点“保存”生效。WebSearch / OCR 工具可选
3. `/earth-content?section=brand`:在“品牌标识”里维护智能星球的 Logo 和标题图;对应地址字段内的“上传”按钮支持选择文件,也支持把图片直接拖到字段上,保存后会应用到智能星球 HUD
4. `/datasources` 或 `/data`:看采集器是否已经产出数据。有限采集器看 `/datasources -> 内置源`,不勾选时点“触发全部”,勾选后主按钮会变成“触发已选 N”;右上角队列按钮可查看进度。AISStream / WebSocket 长连接看 `/datasources -> 实时源` 的健康状态和计数
5. `/alerts/system`:看系统告警是否正常
@@ -50,8 +50,12 @@
- 地球正常显示,右侧图层面板可以打开/关闭
- 搜索可以查找海缆、卫星、算力中心、BGP 事件
- 算力中心和 BGP 观测站详情卡可以自动采集坐标候选,并能在智能星球上预览
- 一键定位期间关闭候选面板或切换浏览器标签页,再返回可查看进度;刷新或离开智能星球页面会中断未完成队列
- 船只图层持续接收 AISStream / BarentsWatch 位置,断线重连后自动校准
- 直播菜单可搜索完整频道库,滚到底部继续加载;默认频道为半岛电视台
- 鼠标拖动、滚轮缩放、缩放百分比提示工作正常
- 设置面板的旋转 / 巡航 / 动捕模式可以切换;动捕设置可以选择输入源和允许识别的动作;视图设置里可以切换悬停提示,卫星相关设置里可以打开或关闭真实高度分层和轨迹显示
- 底图默认使用“原始底图”,并将该选项放在首位;显示菜单可以切换并保存底图源。Natural Earth I 仍保留可用,NASA 21K 只作源归档,运行时同一时间只使用一张 8K 贴图。
## 5. 找回密码
@@ -67,3 +71,9 @@
- 排障与配置疑问:[常见问题](/home/ray/dev/linkong/planet/docs/technical/zh/faq.md)
- 智能星球坐标候选采集流程见 [智能星球使用手册](/home/ray/dev/linkong/planet/docs/technical/zh/manual.md) 的智能星球章节
- 部署 / 运维相关命令:[智能星球运维手册](/home/ray/dev/linkong/planet/docs/technical/zh/ops-runbook.md)
## 试用算力专题
维护卡片数值:管理员打开“智能星球内容 → 算力对比”,先维护报告来源,再填写年度国家观测并“保存并发布”。只录入同年同口径数据,缺失国家不填,设施数量不会替代算力份额。可恢复内置默认值,地球卡片自动检查已发布配置。
在算力中心旁打开“算力态势分析”,开启“显示区域卡片”,选择地区并旋转地球。关闭配置窗会保留卡片,显示开关才控制启停。桌面卡片可调整大小、收起并查看真实历史图;点击来源或设施进入详情。手机使用底部“算力”页。国家指标与设施样本分开,“—”表示缺数据,不代表算力为零。

View File

@@ -16,12 +16,19 @@
## Current Version
- `main` 当前主线历史推导到:`0.16.5`
- `dev` 当前开发分支历史推导到:`0.74.3`
- `dev` 当前开发分支历史推导到:`0.75.2`
## Timeline
| Version | Type | Branch | Commit | Summary |
| --- | --- | --- | --- | --- |
| `0.75.2` | bugfix | `dev` | `v0.75.2` | Windows 启动器优先 Terminal 与 PowerShell 7,修复管理员入口分流,补齐兼容回退与提权测试 |
| `0.75.1` | improvement | `dev` | `v0.75.1` | Windows 启动器自动生成本机桌面入口,后台维护算力比较数据,恢复小球惯性与大球跟手分流并更新 agent 策略 |
| `0.75.0` | feature | `dev` | `v0.75.0` | 四图切换默认原始底图,新增算力区域态势卡片,统一空间连线与重排,优化启动和缩放拖动并补齐地理规则交付 |
| `0.74.7` | improvement | `dev` | `v0.74.7` | Docker 镜像网络故障临时回退,精简副行进度与失败显示,修复数据库端口冲突及启动重试并补齐实际验证 |
| `0.74.6` | improvement | `dev` | `v0.74.6` | 自动检测 Docker 构建代理与直连,统一运维错误诊断,支持跳过 AI Provider 构建并修复 Compose 回退与失败退出状态 |
| `0.74.5` | improvement | `dev` | `v0.74.5` | 恢复模型预设持久化,统一官方目录刷新与连通性检查,修复 M3 思考模式和 OpenCode 协议路由 |
| `0.74.4` | improvement | `dev` | `v0.74.4` | Earth 全量渲染与船舶增量更新优化,直播目录搜索和分页,定位队列状态恢复、模型目录保存及启动提速 |
| `0.74.3` | improvement | `dev` | `v0.74.3` | Ubuntu / WSL 初始化自动准备 Docker 及用户权限,修正启动诊断,并在建表前核对数据库端口、实际连接和认证 |
| `0.74.2` | bugfix | `dev` | `pending` | 收敛 agent harness 到根规则和 Codex skills,删除旧 Claude command 重复入口,并强化视觉证据路径解析与 OCR fallback 规则 |
| `0.74.1` | improvement | `dev` | `pending` | 将品牌标识上传收敛到 Logo/标题图字段内,新增字段级拖拽反馈和 Tactile UI primary 上传按钮,并同步中英文使用文档 |

View File

@@ -1,6 +1,6 @@
{
"name": "planet-frontend",
"version": "0.74.3",
"version": "0.75.2",
"private": true,
"packageManager": "bun@1",
"dependencies": {

Binary file not shown.

Before

Width:  |  Height:  |  Size: 4.4 MiB

After

Width:  |  Height:  |  Size: 3.9 MiB

Some files were not shown because too many files have changed in this diff Show More