release: bump version to 0.71.1
This commit is contained in:
81
AGENTS.md
Normal file
81
AGENTS.md
Normal file
@@ -0,0 +1,81 @@
|
|||||||
|
# Planet Agent Entry Point
|
||||||
|
|
||||||
|
This is the compatibility entry point for coding agents. The older root
|
||||||
|
`agents.md` file remains authoritative for repository-specific agent behavior;
|
||||||
|
do not delete or replace it.
|
||||||
|
|
||||||
|
## Read First
|
||||||
|
|
||||||
|
Read these files before changing code:
|
||||||
|
|
||||||
|
1. `rules.md` - mandatory repository rules. Always load `core`, `security`, and
|
||||||
|
`workflow`; load `docs`, `frontend`, `backend`, `earth`, `ai`, or `release`
|
||||||
|
when the task touches those areas.
|
||||||
|
2. `agents.md` - existing agent role, communication, and workflow guidance.
|
||||||
|
3. `project_context.md` - static project background. Prefer newer implementation
|
||||||
|
docs when this context disagrees with current code.
|
||||||
|
4. `README.md` - current architecture, startup, and toolchain summary.
|
||||||
|
5. `docs/HARNESS.md` - harness workflow, conflict policy, and validation tiers.
|
||||||
|
6. `CODEMAP.md` - codebase entry points, ownership boundaries, and deeper docs.
|
||||||
|
|
||||||
|
For documentation work, also read `docs/documentation-coverage-rules.md`.
|
||||||
|
|
||||||
|
## Start Safely
|
||||||
|
|
||||||
|
Before editing:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git status --short
|
||||||
|
scripts/harness/doctor.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Use focused context commands before reading large files:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rg -n "<symbol-or-term>" <path>
|
||||||
|
git diff --stat HEAD
|
||||||
|
git diff --name-only HEAD
|
||||||
|
git diff --unified=0 HEAD -- <path>
|
||||||
|
```
|
||||||
|
|
||||||
|
Preserve user changes already present in the worktree.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
Fast local harness validation:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/harness/quick-check.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Full local validation:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/harness/validate.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
`validate.sh` includes the quick check and the frontend Bun build. Docker image
|
||||||
|
smoke builds are intentionally opt-in:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## High-Risk Areas
|
||||||
|
|
||||||
|
- `planet.sh` owns local lifecycle, ports, WSL/LAN behavior, and destructive
|
||||||
|
`destroy` cleanup.
|
||||||
|
- Frontend package management is Bun-only. Do not use npm, pnpm, or yarn.
|
||||||
|
- `aiprovider` is a protocol/provider adapter; keep business prompts and product
|
||||||
|
workflows in the backend.
|
||||||
|
- Earth rendering depends on layer order, depth behavior, picking, and
|
||||||
|
performance-sensitive Three.js code.
|
||||||
|
- Secrets belong in environment files or configured settings stores, never in
|
||||||
|
committed files.
|
||||||
|
|
||||||
|
## Conflict Policy
|
||||||
|
|
||||||
|
Existing project rules and workflows win. If new harness guidance conflicts with
|
||||||
|
`rules.md`, `agents.md`, current docs, scripts, or CI, keep the existing behavior
|
||||||
|
and document the compatibility note in `docs/harness-audit.md` or
|
||||||
|
`docs/HARNESS.md`.
|
||||||
93
CODEMAP.md
Normal file
93
CODEMAP.md
Normal file
@@ -0,0 +1,93 @@
|
|||||||
|
# Code Map
|
||||||
|
|
||||||
|
This map gives agents and maintainers a quick orientation without replacing the
|
||||||
|
deeper architecture docs. Current implementation docs under `docs/technical/`
|
||||||
|
are the source of detail for specific subsystems.
|
||||||
|
|
||||||
|
## Top-Level Areas
|
||||||
|
|
||||||
|
| Path | Role | Notes |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `backend/` | FastAPI backend, auth, APIs, data collectors, AI task orchestration, persistence | Tests live in `backend/tests/`; run backend tests from `backend/` with the root uv project. |
|
||||||
|
| `frontend/` | React admin console, Docs UI, Web Earth shell, Vite build | Use Bun only. Public Earth assets live under `frontend/public/earth/`. |
|
||||||
|
| `aiprovider/` | Model provider/protocol adapter service | Keep it free of product-specific prompts and workflows. |
|
||||||
|
| `motion_agent/` | Motion capture protocol service used by `planet.sh` | Often dry-runs when cameras are unavailable, especially in WSL. |
|
||||||
|
| `scripts/` | Utility scripts and harness wrappers | Harness commands live in `scripts/harness/`. |
|
||||||
|
| `docs/` | Plans, technical docs, changelog, harness docs | Public technical docs are explicitly registered by the frontend Docs catalog. |
|
||||||
|
| `deploy/helm/planet/` | Helm chart for staging/deployment smoke paths | CI runs helm lint/template when delivery checks are available. |
|
||||||
|
| `.gitea/workflows/` | CI, release image build, staging deploy workflows | This repository uses Gitea workflow files, not `.github/workflows/`. |
|
||||||
|
| `planet.sh` | Main local lifecycle script | Owns init/start/restart/stop/health/log/createuser/destroy. |
|
||||||
|
|
||||||
|
## Runtime Entry Points
|
||||||
|
|
||||||
|
| Runtime | Entry Point | Validation |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Local full stack | `./planet.sh start` | `./planet.sh health` |
|
||||||
|
| Backend API | `backend/app/main.py` | `cd backend && uv run --frozen --group dev --project .. python -m pytest -q` |
|
||||||
|
| Frontend app | `frontend/src/main.tsx` and `frontend/vite.config.mts` | `cd frontend && bun run build` |
|
||||||
|
| AI Provider | `aiprovider/main.py` | `curl http://localhost:8010/health` after startup |
|
||||||
|
| Motion Agent | `python -m motion_agent` via `planet.sh` | `./planet.sh health` or dry-run startup |
|
||||||
|
| Docs UI | `frontend/src/pages/Docs/` | Docs catalog metadata plus frontend build |
|
||||||
|
|
||||||
|
## Ownership Boundaries
|
||||||
|
|
||||||
|
- Backend owns business state, auth, evidence collection, prompt selection, AI
|
||||||
|
task orchestration, and database persistence.
|
||||||
|
- `aiprovider` owns provider identity, request adapter style, model gateway
|
||||||
|
retries, and health/status endpoints only.
|
||||||
|
- Frontend owns operator workflows, Docs presentation, Web Earth orchestration,
|
||||||
|
and client-side state that mirrors backend truth.
|
||||||
|
- Web Earth rendering changes must preserve documented layer order, altitude
|
||||||
|
offsets, picking behavior, legend semantics, and performance constraints.
|
||||||
|
- `planet.sh` owns local environment bootstrap and service lifecycle. Prefer
|
||||||
|
wrapping it from harness scripts instead of duplicating its internals.
|
||||||
|
|
||||||
|
## Validation Commands
|
||||||
|
|
||||||
|
```bash
|
||||||
|
scripts/harness/doctor.sh
|
||||||
|
scripts/harness/quick-check.sh
|
||||||
|
scripts/harness/validate.sh
|
||||||
|
./planet.sh health
|
||||||
|
```
|
||||||
|
|
||||||
|
CI-equivalent local checks:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd backend
|
||||||
|
uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q
|
||||||
|
|
||||||
|
cd frontend
|
||||||
|
bun install --frozen-lockfile
|
||||||
|
bun run build
|
||||||
|
```
|
||||||
|
|
||||||
|
Optional delivery smoke, when Docker and Helm are available:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deeper Docs
|
||||||
|
|
||||||
|
| Topic | Start Here |
|
||||||
|
| --- | --- |
|
||||||
|
| Data products and flows | `docs/technical/zh/platform-data-flows.md` and `docs/technical/en/platform-data-flows.md` |
|
||||||
|
| Operations and local lifecycle | `docs/technical/zh/ops-runbook.md` and `docs/technical/en/ops-runbook.md` |
|
||||||
|
| `planet.sh` startup behavior | `docs/technical/zh/ops-planet-sh-startup.md` and `docs/technical/en/ops-planet-sh-startup.md` |
|
||||||
|
| AI Provider | `docs/technical/zh/agents-aiprovider.md` and `docs/technical/en/agents-aiprovider.md` |
|
||||||
|
| Admin frontend | `docs/technical/zh/frontend-admin-frontend-context.md` and `docs/technical/en/frontend-admin-frontend-context.md` |
|
||||||
|
| Earth frontend | `docs/technical/zh/earth-frontend-context.md` and `docs/technical/en/earth-frontend-context.md` |
|
||||||
|
| Earth render order | `docs/technical/zh/earth-render-layer-order.md` and `docs/technical/en/earth-render-layer-order.md` |
|
||||||
|
| Documentation rules | `docs/documentation-coverage-rules.md` |
|
||||||
|
| Harness workflow | `docs/HARNESS.md` |
|
||||||
|
|
||||||
|
## Known Sharp Edges
|
||||||
|
|
||||||
|
- `project_context.md` includes older roadmap-era assumptions such as Celery,
|
||||||
|
Kafka, TimescaleDB, MinIO, and UE5 being part of the active local stack. Treat
|
||||||
|
it as background unless current README/docs/code confirm the same behavior.
|
||||||
|
- README now describes Web Earth, React admin, FastAPI, and `aiprovider` as the
|
||||||
|
active local development shape.
|
||||||
|
- Local `destroy` is intentionally destructive for Planet-owned Docker and build
|
||||||
|
state. Never run it as a validation shortcut.
|
||||||
17
agents.md
17
agents.md
@@ -4,6 +4,23 @@
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Harness Compatibility
|
||||||
|
|
||||||
|
Common agent tools should start at `AGENTS.md`. This file remains the existing
|
||||||
|
behavior guide and must not be replaced by harness docs. For safe repository
|
||||||
|
orientation, use:
|
||||||
|
|
||||||
|
- `rules.md` for mandatory project rules
|
||||||
|
- `project_context.md` for static background
|
||||||
|
- `docs/HARNESS.md` for validation tiers and conflict policy
|
||||||
|
- `CODEMAP.md` for subsystem entry points and ownership boundaries
|
||||||
|
- `docs/harness-audit.md` for the latest harness compatibility notes
|
||||||
|
|
||||||
|
Existing project rules and workflows stay authoritative when they conflict with
|
||||||
|
new harness guidance.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Identity
|
## Identity
|
||||||
|
|
||||||
You are **opencode**, an AI coding assistant specialized in enterprise-level systems.
|
You are **opencode**, an AI coding assistant specialized in enterprise-level systems.
|
||||||
|
|||||||
@@ -2307,6 +2307,9 @@ def _diversify_news_items_for_locale(
|
|||||||
buckets: dict[str, list[ParsedNewsItem]] = {}
|
buckets: dict[str, list[ParsedNewsItem]] = {}
|
||||||
order: list[str] = []
|
order: list[str] = []
|
||||||
for item in ranked:
|
for item in ranked:
|
||||||
|
if active_region == "global":
|
||||||
|
key = item.feed_region or "global"
|
||||||
|
else:
|
||||||
key = _news_item_source_id(item) or item.source or item.feed_name or item.id
|
key = _news_item_source_id(item) or item.source or item.feed_name or item.id
|
||||||
if key not in buckets:
|
if key not in buckets:
|
||||||
buckets[key] = []
|
buckets[key] = []
|
||||||
@@ -2751,6 +2754,7 @@ async def get_earth_news_payload(
|
|||||||
)
|
)
|
||||||
await record_earth_news_sources_health(db, health_by_source)
|
await record_earth_news_sources_health(db, health_by_source)
|
||||||
ranked_fetched_items = _rank_and_trim_items(fetched_items, active_region=active_region)
|
ranked_fetched_items = _rank_and_trim_items(fetched_items, active_region=active_region)
|
||||||
|
await _enqueue_unverified_locations(ranked_fetched_items)
|
||||||
await upsert_earth_news_items(db, ranked_fetched_items)
|
await upsert_earth_news_items(db, ranked_fetched_items)
|
||||||
|
|
||||||
items = await _call_store_list_items(
|
items = await _call_store_list_items(
|
||||||
@@ -2802,7 +2806,8 @@ async def get_earth_news_payload(
|
|||||||
)
|
)
|
||||||
else:
|
else:
|
||||||
cruise_items = items
|
cruise_items = items
|
||||||
await _enqueue_unverified_locations(items)
|
enqueue_candidates = {item.id: item for item in [*items, *cruise_items] if item.id}
|
||||||
|
await _enqueue_unverified_locations(list(enqueue_candidates.values()))
|
||||||
stale = bool(errors and items)
|
stale = bool(errors and items)
|
||||||
|
|
||||||
return _build_payload(
|
return _build_payload(
|
||||||
|
|||||||
@@ -14,10 +14,14 @@ from app.core.logging import get_logger
|
|||||||
logger = get_logger(__name__, service="earth_news")
|
logger = get_logger(__name__, service="earth_news")
|
||||||
|
|
||||||
TARGET_LOCATION_STREAM = "earth_news:target_location:jobs"
|
TARGET_LOCATION_STREAM = "earth_news:target_location:jobs"
|
||||||
|
TARGET_LOCATION_PRIORITY_STREAM = "earth_news:target_location:priority"
|
||||||
TARGET_LOCATION_GROUP = "earth_news_target_location"
|
TARGET_LOCATION_GROUP = "earth_news_target_location"
|
||||||
TARGET_LOCATION_DEAD_LETTER_STREAM = "earth_news:target_location:dead"
|
TARGET_LOCATION_DEAD_LETTER_STREAM = "earth_news:target_location:dead"
|
||||||
TARGET_LOCATION_RESULT_TTL_SECONDS = 60 * 60 * 12
|
TARGET_LOCATION_RESULT_TTL_SECONDS = 60 * 60 * 12
|
||||||
TARGET_LOCATION_JOB_DEDUP_TTL_SECONDS = 60 * 60 * 6
|
TARGET_LOCATION_JOB_DEDUP_TTL_SECONDS = 60 * 60 * 6
|
||||||
|
TARGET_LOCATION_PRIORITY_JOB_DEDUP_TTL_SECONDS = 60 * 5
|
||||||
|
TARGET_LOCATION_PENDING_RECLAIM_IDLE_MS = 2 * 60 * 1000
|
||||||
|
TARGET_LOCATION_PRIORITY_READ_BLOCK_MS = 1
|
||||||
TARGET_LOCATION_MAX_ATTEMPTS = 3
|
TARGET_LOCATION_MAX_ATTEMPTS = 3
|
||||||
|
|
||||||
_redis_client: redis.Redis | None = None
|
_redis_client: redis.Redis | None = None
|
||||||
@@ -28,6 +32,7 @@ class NewsTargetLocationMessage:
|
|||||||
message_id: str
|
message_id: str
|
||||||
item_id: str
|
item_id: str
|
||||||
payload: dict[str, Any]
|
payload: dict[str, Any]
|
||||||
|
stream_name: str = TARGET_LOCATION_STREAM
|
||||||
attempts: int = 0
|
attempts: int = 0
|
||||||
|
|
||||||
|
|
||||||
@@ -44,7 +49,7 @@ class NewsTargetLocationQueue(Protocol):
|
|||||||
) -> list[NewsTargetLocationMessage]:
|
) -> list[NewsTargetLocationMessage]:
|
||||||
...
|
...
|
||||||
|
|
||||||
async def ack(self, message_id: str) -> None:
|
async def ack(self, message: NewsTargetLocationMessage) -> None:
|
||||||
...
|
...
|
||||||
|
|
||||||
async def retry_or_dead_letter(
|
async def retry_or_dead_letter(
|
||||||
@@ -71,6 +76,10 @@ def _queued_key(item_id: str) -> str:
|
|||||||
return f"earth_news:target_location:queued:{item_id}"
|
return f"earth_news:target_location:queued:{item_id}"
|
||||||
|
|
||||||
|
|
||||||
|
def _priority_queued_key(item_id: str) -> str:
|
||||||
|
return f"earth_news:target_location:priority_queued:{item_id}"
|
||||||
|
|
||||||
|
|
||||||
class RedisStreamsNewsTargetLocationQueue:
|
class RedisStreamsNewsTargetLocationQueue:
|
||||||
def __init__(self, client: redis.Redis | None = None) -> None:
|
def __init__(self, client: redis.Redis | None = None) -> None:
|
||||||
self.client = client or _get_redis_client()
|
self.client = client or _get_redis_client()
|
||||||
@@ -79,9 +88,10 @@ class RedisStreamsNewsTargetLocationQueue:
|
|||||||
async def _ensure_group(self) -> None:
|
async def _ensure_group(self) -> None:
|
||||||
if self._group_ready:
|
if self._group_ready:
|
||||||
return
|
return
|
||||||
|
for stream_name in (TARGET_LOCATION_PRIORITY_STREAM, TARGET_LOCATION_STREAM):
|
||||||
try:
|
try:
|
||||||
await self.client.xgroup_create(
|
await self.client.xgroup_create(
|
||||||
TARGET_LOCATION_STREAM,
|
stream_name,
|
||||||
TARGET_LOCATION_GROUP,
|
TARGET_LOCATION_GROUP,
|
||||||
id="0",
|
id="0",
|
||||||
mkstream=True,
|
mkstream=True,
|
||||||
@@ -94,19 +104,28 @@ class RedisStreamsNewsTargetLocationQueue:
|
|||||||
async def enqueue(self, *, item_id: str, payload: dict[str, Any], force: bool = False) -> bool:
|
async def enqueue(self, *, item_id: str, payload: dict[str, Any], force: bool = False) -> bool:
|
||||||
await self._ensure_group()
|
await self._ensure_group()
|
||||||
if force:
|
if force:
|
||||||
await self.client.delete(_result_key(item_id), _queued_key(item_id))
|
await self.client.delete(_result_key(item_id))
|
||||||
|
queued_key = _priority_queued_key(item_id)
|
||||||
elif await self.client.exists(_result_key(item_id)):
|
elif await self.client.exists(_result_key(item_id)):
|
||||||
return False
|
return False
|
||||||
|
else:
|
||||||
|
queued_key = _queued_key(item_id)
|
||||||
|
dedup_ttl = (
|
||||||
|
TARGET_LOCATION_PRIORITY_JOB_DEDUP_TTL_SECONDS
|
||||||
|
if force
|
||||||
|
else TARGET_LOCATION_JOB_DEDUP_TTL_SECONDS
|
||||||
|
)
|
||||||
queued = await self.client.set(
|
queued = await self.client.set(
|
||||||
_queued_key(item_id),
|
queued_key,
|
||||||
"1",
|
"1",
|
||||||
nx=True,
|
nx=True,
|
||||||
ex=TARGET_LOCATION_JOB_DEDUP_TTL_SECONDS,
|
ex=dedup_ttl,
|
||||||
)
|
)
|
||||||
if not queued:
|
if not queued:
|
||||||
return bool(await self.client.exists(_queued_key(item_id)))
|
return bool(await self.client.exists(queued_key))
|
||||||
|
stream_name = TARGET_LOCATION_PRIORITY_STREAM if force else TARGET_LOCATION_STREAM
|
||||||
await self.client.xadd(
|
await self.client.xadd(
|
||||||
TARGET_LOCATION_STREAM,
|
stream_name,
|
||||||
{
|
{
|
||||||
"item_id": item_id,
|
"item_id": item_id,
|
||||||
"attempts": "0",
|
"attempts": "0",
|
||||||
@@ -123,6 +142,32 @@ class RedisStreamsNewsTargetLocationQueue:
|
|||||||
block_ms: int,
|
block_ms: int,
|
||||||
) -> list[NewsTargetLocationMessage]:
|
) -> list[NewsTargetLocationMessage]:
|
||||||
await self._ensure_group()
|
await self._ensure_group()
|
||||||
|
streams = []
|
||||||
|
priority_claimed = await self._claim_stale_messages(
|
||||||
|
stream_name=TARGET_LOCATION_PRIORITY_STREAM,
|
||||||
|
consumer_name=consumer_name,
|
||||||
|
count=count,
|
||||||
|
)
|
||||||
|
if priority_claimed:
|
||||||
|
return priority_claimed
|
||||||
|
|
||||||
|
priority_messages = await self.client.xreadgroup(
|
||||||
|
TARGET_LOCATION_GROUP,
|
||||||
|
consumer_name,
|
||||||
|
{TARGET_LOCATION_PRIORITY_STREAM: ">"},
|
||||||
|
count=count,
|
||||||
|
block=TARGET_LOCATION_PRIORITY_READ_BLOCK_MS,
|
||||||
|
)
|
||||||
|
if priority_messages:
|
||||||
|
streams = priority_messages
|
||||||
|
else:
|
||||||
|
regular_claimed = await self._claim_stale_messages(
|
||||||
|
stream_name=TARGET_LOCATION_STREAM,
|
||||||
|
consumer_name=consumer_name,
|
||||||
|
count=count,
|
||||||
|
)
|
||||||
|
if regular_claimed:
|
||||||
|
return regular_claimed
|
||||||
streams = await self.client.xreadgroup(
|
streams = await self.client.xreadgroup(
|
||||||
TARGET_LOCATION_GROUP,
|
TARGET_LOCATION_GROUP,
|
||||||
consumer_name,
|
consumer_name,
|
||||||
@@ -131,31 +176,70 @@ class RedisStreamsNewsTargetLocationQueue:
|
|||||||
block=block_ms,
|
block=block_ms,
|
||||||
)
|
)
|
||||||
messages: list[NewsTargetLocationMessage] = []
|
messages: list[NewsTargetLocationMessage] = []
|
||||||
for _stream_name, stream_messages in streams:
|
for stream_name, stream_messages in streams:
|
||||||
for message_id, fields in stream_messages:
|
for message_id, fields in stream_messages:
|
||||||
|
message = await self._message_from_fields(stream_name, message_id, fields)
|
||||||
|
if message is not None:
|
||||||
|
messages.append(message)
|
||||||
|
return messages
|
||||||
|
|
||||||
|
async def _claim_stale_messages(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
stream_name: str,
|
||||||
|
consumer_name: str,
|
||||||
|
count: int,
|
||||||
|
) -> list[NewsTargetLocationMessage]:
|
||||||
|
try:
|
||||||
|
_next_id, claimed, _deleted = await self.client.xautoclaim(
|
||||||
|
stream_name,
|
||||||
|
TARGET_LOCATION_GROUP,
|
||||||
|
consumer_name,
|
||||||
|
TARGET_LOCATION_PENDING_RECLAIM_IDLE_MS,
|
||||||
|
start_id="0-0",
|
||||||
|
count=count,
|
||||||
|
)
|
||||||
|
except ResponseError:
|
||||||
|
return []
|
||||||
|
messages: list[NewsTargetLocationMessage] = []
|
||||||
|
for message_id, fields in claimed:
|
||||||
|
message = await self._message_from_fields(stream_name, message_id, fields)
|
||||||
|
if message is not None:
|
||||||
|
messages.append(message)
|
||||||
|
return messages
|
||||||
|
|
||||||
|
async def _message_from_fields(
|
||||||
|
self,
|
||||||
|
stream_name: str,
|
||||||
|
message_id: str,
|
||||||
|
fields: dict[str, str],
|
||||||
|
) -> NewsTargetLocationMessage | None:
|
||||||
raw_payload = fields.get("payload")
|
raw_payload = fields.get("payload")
|
||||||
item_id = fields.get("item_id")
|
item_id = fields.get("item_id")
|
||||||
if not raw_payload or not item_id:
|
if not raw_payload or not item_id:
|
||||||
await self.ack(message_id)
|
await self._discard_message(stream_name, message_id)
|
||||||
continue
|
return None
|
||||||
try:
|
try:
|
||||||
payload = json.loads(raw_payload)
|
payload = json.loads(raw_payload)
|
||||||
except json.JSONDecodeError:
|
except json.JSONDecodeError:
|
||||||
await self.ack(message_id)
|
await self._discard_message(stream_name, message_id)
|
||||||
continue
|
return None
|
||||||
attempts = int(fields.get("attempts") or 0)
|
attempts = int(fields.get("attempts") or 0)
|
||||||
messages.append(
|
return NewsTargetLocationMessage(
|
||||||
NewsTargetLocationMessage(
|
|
||||||
message_id=message_id,
|
message_id=message_id,
|
||||||
item_id=item_id,
|
item_id=item_id,
|
||||||
payload=payload,
|
payload=payload,
|
||||||
|
stream_name=stream_name,
|
||||||
attempts=attempts,
|
attempts=attempts,
|
||||||
)
|
)
|
||||||
)
|
|
||||||
return messages
|
|
||||||
|
|
||||||
async def ack(self, message_id: str) -> None:
|
async def ack(self, message: NewsTargetLocationMessage) -> None:
|
||||||
await self.client.xack(TARGET_LOCATION_STREAM, TARGET_LOCATION_GROUP, message_id)
|
await self.client.xack(message.stream_name, TARGET_LOCATION_GROUP, message.message_id)
|
||||||
|
await self.client.xdel(message.stream_name, message.message_id)
|
||||||
|
|
||||||
|
async def _discard_message(self, stream_name: str, message_id: str) -> None:
|
||||||
|
await self.client.xack(stream_name, TARGET_LOCATION_GROUP, message_id)
|
||||||
|
await self.client.xdel(stream_name, message_id)
|
||||||
|
|
||||||
async def retry_or_dead_letter(
|
async def retry_or_dead_letter(
|
||||||
self,
|
self,
|
||||||
@@ -163,7 +247,7 @@ class RedisStreamsNewsTargetLocationQueue:
|
|||||||
*,
|
*,
|
||||||
error: str,
|
error: str,
|
||||||
) -> None:
|
) -> None:
|
||||||
await self.ack(message.message_id)
|
await self.ack(message)
|
||||||
if message.attempts + 1 >= TARGET_LOCATION_MAX_ATTEMPTS:
|
if message.attempts + 1 >= TARGET_LOCATION_MAX_ATTEMPTS:
|
||||||
await self.client.xadd(
|
await self.client.xadd(
|
||||||
TARGET_LOCATION_DEAD_LETTER_STREAM,
|
TARGET_LOCATION_DEAD_LETTER_STREAM,
|
||||||
@@ -176,7 +260,7 @@ class RedisStreamsNewsTargetLocationQueue:
|
|||||||
)
|
)
|
||||||
return
|
return
|
||||||
await self.client.xadd(
|
await self.client.xadd(
|
||||||
TARGET_LOCATION_STREAM,
|
message.stream_name,
|
||||||
{
|
{
|
||||||
"item_id": message.item_id,
|
"item_id": message.item_id,
|
||||||
"attempts": str(message.attempts + 1),
|
"attempts": str(message.attempts + 1),
|
||||||
@@ -231,4 +315,4 @@ async def save_target_location_patch(item_id: str, patch: dict[str, Any]) -> Non
|
|||||||
TARGET_LOCATION_RESULT_TTL_SECONDS,
|
TARGET_LOCATION_RESULT_TTL_SECONDS,
|
||||||
json.dumps(patch, ensure_ascii=False),
|
json.dumps(patch, ensure_ascii=False),
|
||||||
)
|
)
|
||||||
await client.delete(_queued_key(item_id))
|
await client.delete(_queued_key(item_id), _priority_queued_key(item_id))
|
||||||
|
|||||||
@@ -19,6 +19,17 @@ from app.services.earth_news_classification import (
|
|||||||
normalize_breaking_scope,
|
normalize_breaking_scope,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
CRUISE_REGION_ORDER = (
|
||||||
|
"americas",
|
||||||
|
"europe",
|
||||||
|
"middle-east-africa",
|
||||||
|
"asia-pacific",
|
||||||
|
"global",
|
||||||
|
)
|
||||||
|
CRUISE_REGION_QUERY_MULTIPLIER = 12
|
||||||
|
CRUISE_REGION_QUERY_MIN_LIMIT = 240
|
||||||
|
CRUISE_REGION_QUERY_MAX_LIMIT = 1000
|
||||||
|
|
||||||
|
|
||||||
def _coerce_datetime(value: datetime | None) -> datetime | None:
|
def _coerce_datetime(value: datetime | None) -> datetime | None:
|
||||||
if value is None:
|
if value is None:
|
||||||
@@ -106,6 +117,41 @@ def _sort_parsed_news_items(items: list[ParsedNewsItem], *, active_region: str)
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def _diversify_parsed_news_items_by_region(
|
||||||
|
items: list[ParsedNewsItem],
|
||||||
|
*,
|
||||||
|
limit: int,
|
||||||
|
) -> list[ParsedNewsItem]:
|
||||||
|
if limit <= 0:
|
||||||
|
return []
|
||||||
|
sorted_items = _sort_parsed_news_items(items, active_region="global")
|
||||||
|
buckets: dict[str, list[ParsedNewsItem]] = {}
|
||||||
|
for item in sorted_items:
|
||||||
|
region = item.feed_region or "global"
|
||||||
|
buckets.setdefault(region, []).append(item)
|
||||||
|
|
||||||
|
ordered_regions = [
|
||||||
|
*[region for region in CRUISE_REGION_ORDER if buckets.get(region)],
|
||||||
|
*sorted(region for region in buckets if region not in CRUISE_REGION_ORDER),
|
||||||
|
]
|
||||||
|
diversified: list[ParsedNewsItem] = []
|
||||||
|
cursor = 0
|
||||||
|
while len(diversified) < limit:
|
||||||
|
added = False
|
||||||
|
for region in ordered_regions:
|
||||||
|
bucket = buckets.get(region) or []
|
||||||
|
if cursor >= len(bucket):
|
||||||
|
continue
|
||||||
|
diversified.append(bucket[cursor])
|
||||||
|
added = True
|
||||||
|
if len(diversified) >= limit:
|
||||||
|
break
|
||||||
|
if not added:
|
||||||
|
break
|
||||||
|
cursor += 1
|
||||||
|
return diversified
|
||||||
|
|
||||||
|
|
||||||
def _query_sort_key(active_region: str):
|
def _query_sort_key(active_region: str):
|
||||||
if active_region == "global":
|
if active_region == "global":
|
||||||
return (
|
return (
|
||||||
@@ -167,6 +213,8 @@ async def list_earth_news_items(
|
|||||||
[record_to_parsed_news_item(record) for record in records],
|
[record_to_parsed_news_item(record) for record in records],
|
||||||
active_region=active_region,
|
active_region=active_region,
|
||||||
)
|
)
|
||||||
|
if active_region == "global" and not source_ids:
|
||||||
|
return _diversify_parsed_news_items_by_region(items, limit=limit)
|
||||||
return items[:limit]
|
return items[:limit]
|
||||||
|
|
||||||
|
|
||||||
@@ -177,15 +225,20 @@ async def list_earth_news_cruise_items(
|
|||||||
categories: set[str] | None = None,
|
categories: set[str] | None = None,
|
||||||
source_ids: set[str] | None = None,
|
source_ids: set[str] | None = None,
|
||||||
) -> list[ParsedNewsItem]:
|
) -> list[ParsedNewsItem]:
|
||||||
|
query_limit = min(
|
||||||
|
max(limit * CRUISE_REGION_QUERY_MULTIPLIER, CRUISE_REGION_QUERY_MIN_LIMIT),
|
||||||
|
CRUISE_REGION_QUERY_MAX_LIMIT,
|
||||||
|
)
|
||||||
query = (
|
query = (
|
||||||
select(EarthNewsItem)
|
select(EarthNewsItem)
|
||||||
.order_by(
|
.order_by(
|
||||||
EarthNewsItem.region.asc(),
|
EarthNewsItem.published_at.is_(None),
|
||||||
EarthNewsItem.published_at.desc().nullslast(),
|
EarthNewsItem.published_at.desc().nullslast(),
|
||||||
EarthNewsItem.last_seen_at.desc(),
|
EarthNewsItem.last_seen_at.desc(),
|
||||||
|
EarthNewsItem.region.asc(),
|
||||||
EarthNewsItem.feed_name.asc(),
|
EarthNewsItem.feed_name.asc(),
|
||||||
)
|
)
|
||||||
.limit(min(max(limit * 4, limit), 200))
|
.limit(query_limit)
|
||||||
)
|
)
|
||||||
category_clause = _category_filter_clause(categories)
|
category_clause = _category_filter_clause(categories)
|
||||||
if category_clause is not None:
|
if category_clause is not None:
|
||||||
@@ -194,11 +247,10 @@ async def list_earth_news_cruise_items(
|
|||||||
if source_clause is not None:
|
if source_clause is not None:
|
||||||
query = query.where(source_clause)
|
query = query.where(source_clause)
|
||||||
result = await db.execute(query)
|
result = await db.execute(query)
|
||||||
items = _sort_parsed_news_items(
|
return _diversify_parsed_news_items_by_region(
|
||||||
[record_to_parsed_news_item(record) for record in result.scalars().all()],
|
[record_to_parsed_news_item(record) for record in result.scalars().all()],
|
||||||
active_region="global",
|
limit=limit,
|
||||||
)
|
)
|
||||||
return items[:limit]
|
|
||||||
|
|
||||||
|
|
||||||
async def get_earth_news_freshness(
|
async def get_earth_news_freshness(
|
||||||
|
|||||||
@@ -29,6 +29,9 @@ logger = get_logger(__name__, service="earth_news")
|
|||||||
WORKER_BATCH_SIZE = 4
|
WORKER_BATCH_SIZE = 4
|
||||||
WORKER_BLOCK_MS = 5000
|
WORKER_BLOCK_MS = 5000
|
||||||
WORKER_BACKOFF_SECONDS = 5.0
|
WORKER_BACKOFF_SECONDS = 5.0
|
||||||
|
WORKER_JOB_TIMEOUT_MIN_SECONDS = 20.0
|
||||||
|
WORKER_JOB_TIMEOUT_MAX_SECONDS = 90.0
|
||||||
|
WORKER_JOB_TIMEOUT_GRACE_SECONDS = 10.0
|
||||||
|
|
||||||
_worker_task: asyncio.Task | None = None
|
_worker_task: asyncio.Task | None = None
|
||||||
|
|
||||||
@@ -109,12 +112,25 @@ async def _run_target_location_worker() -> None:
|
|||||||
if not messages:
|
if not messages:
|
||||||
continue
|
continue
|
||||||
provider_client = await _build_provider_client()
|
provider_client = await _build_provider_client()
|
||||||
for message in messages:
|
job_timeout = _get_worker_job_timeout(provider_client)
|
||||||
|
|
||||||
|
async def handle_message(message: NewsTargetLocationMessage) -> None:
|
||||||
try:
|
try:
|
||||||
await process_target_location_message(message, provider_client=provider_client)
|
await asyncio.wait_for(
|
||||||
await queue.ack(message.message_id)
|
process_target_location_message(message, provider_client=provider_client),
|
||||||
|
timeout=job_timeout,
|
||||||
|
)
|
||||||
|
await queue.ack(message)
|
||||||
except asyncio.CancelledError:
|
except asyncio.CancelledError:
|
||||||
raise
|
raise
|
||||||
|
except TimeoutError as exc:
|
||||||
|
logger.warning_event(
|
||||||
|
"Earth news target location worker job timed out",
|
||||||
|
event="earth_news.target_location.worker_job_timeout",
|
||||||
|
context={"item_id": message.item_id, "timeout_seconds": job_timeout},
|
||||||
|
)
|
||||||
|
with suppress(Exception):
|
||||||
|
await queue.retry_or_dead_letter(message, error=str(exc) or "job timed out")
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.warning_event(
|
logger.warning_event(
|
||||||
"Earth news target location worker job failed",
|
"Earth news target location worker job failed",
|
||||||
@@ -124,6 +140,8 @@ async def _run_target_location_worker() -> None:
|
|||||||
with suppress(Exception):
|
with suppress(Exception):
|
||||||
await queue.retry_or_dead_letter(message, error=str(exc))
|
await queue.retry_or_dead_letter(message, error=str(exc))
|
||||||
|
|
||||||
|
await asyncio.gather(*(handle_message(message) for message in messages))
|
||||||
|
|
||||||
|
|
||||||
def start_earth_news_target_worker() -> None:
|
def start_earth_news_target_worker() -> None:
|
||||||
global _worker_task
|
global _worker_task
|
||||||
@@ -140,3 +158,15 @@ async def stop_earth_news_target_worker() -> None:
|
|||||||
with suppress(asyncio.CancelledError):
|
with suppress(asyncio.CancelledError):
|
||||||
await task
|
await task
|
||||||
_worker_task = None
|
_worker_task = None
|
||||||
|
|
||||||
|
|
||||||
|
def _get_worker_job_timeout(provider_client: AIProviderClient | None) -> float:
|
||||||
|
timeout = float(getattr(provider_client, "timeout", 0) or WORKER_JOB_TIMEOUT_MIN_SECONDS)
|
||||||
|
retry_attempts = float(getattr(provider_client, "retry_attempts", 1) or 1)
|
||||||
|
return min(
|
||||||
|
max(
|
||||||
|
timeout * retry_attempts + WORKER_JOB_TIMEOUT_GRACE_SECONDS,
|
||||||
|
WORKER_JOB_TIMEOUT_MIN_SECONDS,
|
||||||
|
),
|
||||||
|
WORKER_JOB_TIMEOUT_MAX_SECONDS,
|
||||||
|
)
|
||||||
|
|||||||
@@ -24,6 +24,7 @@ from app.services.earth_news import (
|
|||||||
from app.services.earth_news_queue import NewsTargetLocationMessage
|
from app.services.earth_news_queue import NewsTargetLocationMessage
|
||||||
from app.services.earth_news_worker import process_target_location_message
|
from app.services.earth_news_worker import process_target_location_message
|
||||||
from app.services.collectors.media_news_archive import MediaNewsArchiveCollector
|
from app.services.collectors.media_news_archive import MediaNewsArchiveCollector
|
||||||
|
from app.services.earth_news_store import _diversify_parsed_news_items_by_region
|
||||||
|
|
||||||
|
|
||||||
def test_serialize_item_includes_region_anchor_for_cruise():
|
def test_serialize_item_includes_region_anchor_for_cruise():
|
||||||
@@ -172,6 +173,68 @@ def test_diversify_news_items_prefers_display_ready_content_across_sources():
|
|||||||
assert [item.id.split(":", 1)[0] for item in result] == ["source-b", "source-c", "source-a"]
|
assert [item.id.split(":", 1)[0] for item in result] == ["source-b", "source-c", "source-a"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_cruise_news_diversity_keeps_regions_from_being_starved():
|
||||||
|
published_at = datetime(2026, 6, 26, 8, 0, tzinfo=UTC)
|
||||||
|
|
||||||
|
def make_item(region: str, index: int) -> ParsedNewsItem:
|
||||||
|
return ParsedNewsItem(
|
||||||
|
id=f"{region}:{index}",
|
||||||
|
title=f"{region} story {index}",
|
||||||
|
summary=f"{region} summary {index}",
|
||||||
|
url=f"https://example.com/{region}/{index}",
|
||||||
|
source=region,
|
||||||
|
feed_name=region,
|
||||||
|
feed_region=region,
|
||||||
|
homepage_url="https://example.com",
|
||||||
|
published_at=published_at - timedelta(minutes=index),
|
||||||
|
)
|
||||||
|
|
||||||
|
items = [
|
||||||
|
*[make_item("asia-pacific", index) for index in range(40)],
|
||||||
|
make_item("europe", 1),
|
||||||
|
make_item("middle-east-africa", 1),
|
||||||
|
make_item("americas", 1),
|
||||||
|
make_item("global", 1),
|
||||||
|
]
|
||||||
|
|
||||||
|
result = _diversify_parsed_news_items_by_region(items, limit=8)
|
||||||
|
regions = [item.feed_region for item in result]
|
||||||
|
|
||||||
|
assert "europe" in regions
|
||||||
|
assert "middle-east-africa" in regions
|
||||||
|
assert "americas" in regions
|
||||||
|
assert regions.count("asia-pacific") < len(regions)
|
||||||
|
|
||||||
|
|
||||||
|
def test_global_news_diversity_uses_same_region_balance():
|
||||||
|
published_at = datetime(2026, 6, 26, 8, 0, tzinfo=UTC)
|
||||||
|
|
||||||
|
def make_item(region: str, index: int) -> ParsedNewsItem:
|
||||||
|
return ParsedNewsItem(
|
||||||
|
id=f"{region}:global:{index}",
|
||||||
|
title=f"{region} story {index}",
|
||||||
|
summary=f"{region} summary {index}",
|
||||||
|
url=f"https://example.com/{region}/global/{index}",
|
||||||
|
source=region,
|
||||||
|
feed_name=region,
|
||||||
|
feed_region=region,
|
||||||
|
homepage_url="https://example.com",
|
||||||
|
published_at=published_at - timedelta(minutes=index),
|
||||||
|
)
|
||||||
|
|
||||||
|
items = [
|
||||||
|
*[make_item("asia-pacific", index) for index in range(24)],
|
||||||
|
*[make_item("europe", index) for index in range(2)],
|
||||||
|
*[make_item("middle-east-africa", index) for index in range(2)],
|
||||||
|
*[make_item("americas", index) for index in range(2)],
|
||||||
|
]
|
||||||
|
|
||||||
|
result = _diversify_parsed_news_items_by_region(items, limit=6)
|
||||||
|
regions = {item.feed_region for item in result}
|
||||||
|
|
||||||
|
assert {"europe", "middle-east-africa", "americas"}.issubset(regions)
|
||||||
|
|
||||||
|
|
||||||
def test_serialize_item_falls_back_to_global_anchor():
|
def test_serialize_item_falls_back_to_global_anchor():
|
||||||
item = ParsedNewsItem(
|
item = ParsedNewsItem(
|
||||||
id="custom:test",
|
id="custom:test",
|
||||||
|
|||||||
@@ -8,6 +8,24 @@ This project follows the repository versioning rule:
|
|||||||
- `improvement` -> `+0.0.1`(bugfix + 小功能混合)
|
- `improvement` -> `+0.0.1`(bugfix + 小功能混合)
|
||||||
- `bugfix` -> `+0.0.1`
|
- `bugfix` -> `+0.0.1`
|
||||||
|
|
||||||
|
## [0.71.1] — 2026-06-26
|
||||||
|
|
||||||
|
Released: 2026-06-26
|
||||||
|
|
||||||
|
### Highlights
|
||||||
|
- 修复 Earth 欧洲、美洲、中东与非洲等区域新闻被亚太来源和旧来源过滤饿死的问题,滚动条、面板和巡航重新回到同一批区域 payload。
|
||||||
|
- 将当前可见新闻和巡航新闻提升到目标位置/翻译优先队列,避免历史普通 Redis backlog 阻塞用户正在看的新闻精修。
|
||||||
|
- 新增 agent harness 入口、代码地图、验证脚本与双语技术说明,让后续维护能按现有 uv/Bun/Gitea 工作流检查而不替代项目规则。
|
||||||
|
|
||||||
|
### Added / Fixed / Improved
|
||||||
|
- EarthFeed 在全局和巡航队列中按区域轮转候选新闻,保留区域视图的“当前区域 + global”规则,并补充回归测试。
|
||||||
|
- `news.js` 按区域、类型、来源和数量隔离并发刷新请求,丢弃旧区域响应;跨区域时不再复用旧来源筛选。
|
||||||
|
- 新闻展示在中文本地化未完成时回退原始标题和摘要,避免出现有内容却显示“新闻汉化中”的卡片。
|
||||||
|
- 新闻目标位置 worker 新增优先 stream、pending reclaim、任务超时和并发处理;无效消息会确认并删除,减少队列堆积。
|
||||||
|
- 补充 Earth 新闻源、Earth 前端结构、harness 和版本历史文档,并移除控制台 auth store 的调试日志。
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## [0.71.0] — 2026-06-11
|
## [0.71.0] — 2026-06-11
|
||||||
|
|
||||||
Released: 2026-06-11
|
Released: 2026-06-11
|
||||||
|
|||||||
167
docs/HARNESS.md
Normal file
167
docs/HARNESS.md
Normal file
@@ -0,0 +1,167 @@
|
|||||||
|
# Agent Harness
|
||||||
|
|
||||||
|
This harness improves discoverability, repeatability, and agent safety for the
|
||||||
|
existing Planet project. It does not replace current project rules, scripts, CI,
|
||||||
|
or release workflows.
|
||||||
|
|
||||||
|
## Authority And Conflicts
|
||||||
|
|
||||||
|
Existing project rules are authoritative:
|
||||||
|
|
||||||
|
1. `rules.md`
|
||||||
|
2. `agents.md`
|
||||||
|
3. Current implementation docs under `docs/technical/`
|
||||||
|
4. Existing scripts, especially `planet.sh`
|
||||||
|
5. Existing Gitea workflow files under `.gitea/workflows/`
|
||||||
|
|
||||||
|
When harness guidance conflicts with any of the above, keep the existing rule,
|
||||||
|
do not overwrite the existing workflow, and add a compatibility note here or in
|
||||||
|
`docs/harness-audit.md`.
|
||||||
|
|
||||||
|
## Starting Work
|
||||||
|
|
||||||
|
Recommended startup flow:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git status --short
|
||||||
|
scripts/harness/doctor.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Then read only the relevant implementation docs:
|
||||||
|
|
||||||
|
- Backend/API/data work: `docs/technical/zh/backend-*.md` and matching English
|
||||||
|
docs when public docs are affected.
|
||||||
|
- Frontend/admin work: `docs/technical/zh/frontend-admin-frontend-context.md`.
|
||||||
|
- Earth work: `docs/technical/zh/earth-frontend-context.md`,
|
||||||
|
`docs/technical/zh/earth-render-layer-order.md`, and style docs when visual
|
||||||
|
semantics change.
|
||||||
|
- Operations work: `docs/technical/zh/ops-runbook.md` and
|
||||||
|
`docs/technical/zh/ops-planet-sh-startup.md`.
|
||||||
|
- AI Provider work: `docs/technical/zh/agents-aiprovider.md`.
|
||||||
|
- Documentation work: `docs/documentation-coverage-rules.md`.
|
||||||
|
|
||||||
|
Use focused inspection commands before broad reads:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
rg -n "<symbol-or-term>" <path>
|
||||||
|
git diff --stat HEAD
|
||||||
|
git diff --name-only HEAD
|
||||||
|
git diff --unified=0 HEAD -- <path>
|
||||||
|
```
|
||||||
|
|
||||||
|
## Existing Commands
|
||||||
|
|
||||||
|
| Purpose | Command |
|
||||||
|
| --- | --- |
|
||||||
|
| First setup | `./planet.sh init` |
|
||||||
|
| Start local stack | `./planet.sh start` |
|
||||||
|
| Start with LAN access | `./planet.sh start --allow-lan` |
|
||||||
|
| Restart all services | `./planet.sh restart` |
|
||||||
|
| Restart one area | `./planet.sh restart -b`, `-f`, `-a`, or `-d` |
|
||||||
|
| Health check | `./planet.sh health` |
|
||||||
|
| Logs | `./planet.sh log`, `./planet.sh log -b`, `-f`, `-a`, or `-m` |
|
||||||
|
| Create local user | `./planet.sh createuser` |
|
||||||
|
| Destructive local reset | `./planet.sh destroy` |
|
||||||
|
| Backend smoke tests | `cd backend && uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q` |
|
||||||
|
| Frontend build | `cd frontend && bun install --frozen-lockfile && bun run build` |
|
||||||
|
| Mock AIS WebSocket | `bun run mock:ais-ws` |
|
||||||
|
|
||||||
|
## Harness Commands
|
||||||
|
|
||||||
|
| Tier | Command | What It Does |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Doctor | `scripts/harness/doctor.sh` | Checks required files, required tools, optional delivery tools, and forbidden frontend lockfiles. |
|
||||||
|
| Quick | `scripts/harness/quick-check.sh` | Runs doctor, whitespace diff check, shell syntax checks, and CI backend smoke tests. |
|
||||||
|
| Full | `scripts/harness/validate.sh` | Runs quick check, frontend Bun install/build, optional Helm checks, and opt-in Docker image smoke builds. |
|
||||||
|
|
||||||
|
Docker image smoke builds are expensive and are off by default:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Environment Requirements
|
||||||
|
|
||||||
|
Required for normal development:
|
||||||
|
|
||||||
|
- `zsh` for `planet.sh`
|
||||||
|
- `uv` for Python dependency and test execution
|
||||||
|
- `bun` for frontend dependency and build execution
|
||||||
|
- Python resolved by `uv` from the root `pyproject.toml`
|
||||||
|
|
||||||
|
Required for full local stack operation:
|
||||||
|
|
||||||
|
- Docker and Docker Compose
|
||||||
|
- PostgreSQL and Redis containers started by `planet.sh`
|
||||||
|
|
||||||
|
Optional for delivery smoke:
|
||||||
|
|
||||||
|
- Docker daemon for image builds
|
||||||
|
- Helm for chart lint/template checks
|
||||||
|
|
||||||
|
If a required local tool is missing, do not install system software
|
||||||
|
automatically. Report the gap and point to `./planet.sh init` or
|
||||||
|
`scripts/bootstrap-dev.sh` as the existing bootstrap path.
|
||||||
|
|
||||||
|
## What Agents Must Not Change Automatically
|
||||||
|
|
||||||
|
- Do not replace Bun with npm, pnpm, or yarn.
|
||||||
|
- Do not migrate CI from `.gitea/workflows/` to `.github/workflows/`.
|
||||||
|
- Do not rewrite `planet.sh` lifecycle behavior as a parallel script.
|
||||||
|
- Do not run `./planet.sh destroy` unless explicitly requested.
|
||||||
|
- Do not commit `.env`, secrets, private keys, logs, or generated build output.
|
||||||
|
- Do not add external integrations, hooks, or new dependency managers just to
|
||||||
|
satisfy harness structure.
|
||||||
|
- Do not publish internal harness docs into the product Docs UI unless a
|
||||||
|
maintainer explicitly asks for it.
|
||||||
|
|
||||||
|
## Hooks And Reminders
|
||||||
|
|
||||||
|
No automatic hooks are installed in this phase. Manual reminders:
|
||||||
|
|
||||||
|
- Run `scripts/harness/quick-check.sh` before handing off small changes.
|
||||||
|
- Run `scripts/harness/validate.sh` before larger cross-subsystem changes.
|
||||||
|
- Add focused tests before modifying backend service behavior or frontend
|
||||||
|
workflows.
|
||||||
|
- For docs changes, run the checks listed in
|
||||||
|
`docs/documentation-coverage-rules.md`.
|
||||||
|
|
||||||
|
## Reusable Workflows
|
||||||
|
|
||||||
|
### Feature Work
|
||||||
|
|
||||||
|
1. Read `rules.md` modules for the touched area.
|
||||||
|
2. Check `CODEMAP.md` for entry points and ownership boundaries.
|
||||||
|
3. Inspect existing tests and docs before editing.
|
||||||
|
4. Make the smallest behavior-preserving or feature-scoped change.
|
||||||
|
5. Run `scripts/harness/quick-check.sh` or a narrower documented command.
|
||||||
|
6. Update relevant docs when behavior, workflow, or operations change.
|
||||||
|
|
||||||
|
### Bug Fix
|
||||||
|
|
||||||
|
1. Reproduce with a focused test or command.
|
||||||
|
2. Patch the owning module, not a caller-side workaround.
|
||||||
|
3. Run the focused regression test.
|
||||||
|
4. Run `scripts/harness/quick-check.sh` when the change is safe to validate
|
||||||
|
locally.
|
||||||
|
|
||||||
|
### Documentation Change
|
||||||
|
|
||||||
|
1. Read `docs/documentation-coverage-rules.md`.
|
||||||
|
2. Route docs by audience: UI users, operations, or second-party developers.
|
||||||
|
3. Keep Chinese and English public docs consistent when a public doc pair exists.
|
||||||
|
4. Run the repository-specific docs checks that match the changed files.
|
||||||
|
|
||||||
|
### Release Or Delivery Change
|
||||||
|
|
||||||
|
Use the existing release skill/workflow and `.gitea/workflows/` files. Harness
|
||||||
|
validation can smoke-check Helm and Docker locally, but it must not replace the
|
||||||
|
release process.
|
||||||
|
|
||||||
|
## Implementation Notes
|
||||||
|
|
||||||
|
- `docs/harness-audit.md` records the discovery pass that led to this harness.
|
||||||
|
- `AGENTS.md` is a compatibility entry point for tools that expect the uppercase
|
||||||
|
filename. The existing `agents.md` file remains in place.
|
||||||
|
- `CODEMAP.md` is intentionally high level; deeper subsystem docs stay in
|
||||||
|
`docs/technical/{zh,en}/`.
|
||||||
102
docs/harness-audit.md
Normal file
102
docs/harness-audit.md
Normal file
@@ -0,0 +1,102 @@
|
|||||||
|
# Harness Audit
|
||||||
|
|
||||||
|
Last audited: 2026-06-26
|
||||||
|
|
||||||
|
This audit records the repository state used to add the agent harness. It is a
|
||||||
|
compatibility note, not a replacement for existing rules or architecture docs.
|
||||||
|
|
||||||
|
## Existing Commands
|
||||||
|
|
||||||
|
| Area | Existing Command | Notes |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| Bootstrap | `./planet.sh init` | Syncs uv/Bun dependencies, creates missing env files, starts data services, seeds defaults. |
|
||||||
|
| Start | `./planet.sh start` | Starts backend, frontend, AI Provider, PostgreSQL/Redis, and Motion Agent when available. |
|
||||||
|
| LAN start | `./planet.sh start --allow-lan` | Opens frontend/backend/AI Provider ports and requests Windows firewall/port cleanup when needed. |
|
||||||
|
| Restart | `./planet.sh restart` | Supports scoped restart flags for backend, frontend, AI Provider, database, and Motion Agent. |
|
||||||
|
| Health | `./planet.sh health` | Checks containers, backend `/health`, AI Provider `/health`, frontend, and Motion Agent state. |
|
||||||
|
| Logs | `./planet.sh log` | Supports backend, frontend, AI Provider, and Motion Agent log views. |
|
||||||
|
| User fallback | `./planet.sh createuser` | Interactive emergency/local account creation. |
|
||||||
|
| Destructive reset | `./planet.sh destroy` | Requires confirmation and removes Planet-owned Docker/build/runtime state. Not a validation command. |
|
||||||
|
| Backend CI smoke | `cd backend && uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py tests/test_realtime_sources.py -q` | Mirrors `.gitea/workflows/ci.yaml`. |
|
||||||
|
| Frontend build | `cd frontend && bun install --frozen-lockfile && bun run build` | Bun-only workflow. |
|
||||||
|
| Root helper | `bun run mock:ais-ws` | Runs `scripts/mock-ais-ws-server.ts` from the root package. |
|
||||||
|
|
||||||
|
## Existing Agent Instructions
|
||||||
|
|
||||||
|
| File | Status | Notes |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `agents.md` | Present | Existing root agent behavior guide. It references `rules.md` and `project_context.md`. |
|
||||||
|
| `rules.md` | Present | Mandatory modular rules. Always load `core`, `security`, and `workflow`; load topic modules as needed. |
|
||||||
|
| `project_context.md` | Present | Static context. Some roadmap-era stack details are older than the current README/docs. |
|
||||||
|
| `.claude/commands/*.md` | Present | Existing command docs for cleanup, docs, goal-driven, and release workflows. |
|
||||||
|
| `.codex/skills/*.md` | Present | Existing local skills for cleanup, docs, goal-driven, and release. |
|
||||||
|
| `AGENTS.md` | Added by harness | Compatibility entry point that points to existing rules and harness docs. |
|
||||||
|
|
||||||
|
## Existing CI Gates
|
||||||
|
|
||||||
|
The repository uses `.gitea/workflows/`, not `.github/workflows/`.
|
||||||
|
|
||||||
|
| Workflow | Gate |
|
||||||
|
| --- | --- |
|
||||||
|
| `.gitea/workflows/ci.yaml` | Backend smoke tests, frontend Bun build, Docker build smoke, Helm lint/template. |
|
||||||
|
| `.gitea/workflows/release.yaml` | Builds and pushes frontend, backend, and AI Provider images on main/tag/manual release events. |
|
||||||
|
| `.gitea/workflows/deploy-staging.yaml` | Deploys Helm release to staging and runs curl smoke tests inside the cluster. |
|
||||||
|
|
||||||
|
## Existing Docs And Architecture Maps
|
||||||
|
|
||||||
|
| Area | Docs |
|
||||||
|
| --- | --- |
|
||||||
|
| Current architecture and startup | `README.md` |
|
||||||
|
| Technical docs index | `docs/technical/zh/README.md`, `docs/technical/en/README.md` |
|
||||||
|
| Documentation rules | `docs/documentation-coverage-rules.md` |
|
||||||
|
| Operations | `docs/technical/zh/ops-runbook.md`, `docs/technical/en/ops-runbook.md` |
|
||||||
|
| Startup internals | `docs/technical/zh/ops-planet-sh-startup.md`, `docs/technical/en/ops-planet-sh-startup.md` |
|
||||||
|
| AI Provider | `docs/technical/zh/agents-aiprovider.md`, `docs/technical/en/agents-aiprovider.md` |
|
||||||
|
| Frontend admin | `docs/technical/zh/frontend-admin-frontend-context.md`, `docs/technical/en/frontend-admin-frontend-context.md` |
|
||||||
|
| Earth rendering | `docs/technical/zh/earth-frontend-context.md`, `docs/technical/zh/earth-render-layer-order.md`, `docs/technical/zh/earth-layer-style-reference.md` |
|
||||||
|
| Plans and history | `docs/plans/README.md`, `docs/deprecated/README.md` |
|
||||||
|
|
||||||
|
## Release And Deploy Process
|
||||||
|
|
||||||
|
- Release workflow is documented in `.codex/skills/release/SKILL.md` and
|
||||||
|
`.claude/commands/release.md`.
|
||||||
|
- Version-bearing files include `VERSION`, `frontend/package.json`,
|
||||||
|
`pyproject.toml`, `uv.lock`, `docs/CHANGELOG.md`, and
|
||||||
|
`docs/version-history.md`.
|
||||||
|
- Delivery automation lives in `.gitea/workflows/release.yaml` and
|
||||||
|
`.gitea/workflows/deploy-staging.yaml`.
|
||||||
|
- Helm chart entry point is `deploy/helm/planet/Chart.yaml`.
|
||||||
|
|
||||||
|
## Missing Or Unclear Areas
|
||||||
|
|
||||||
|
- README previously listed `AGENTS.md` in the project tree while only lowercase
|
||||||
|
`agents.md` existed. The harness adds uppercase `AGENTS.md` as a compatibility
|
||||||
|
wrapper and preserves `agents.md`.
|
||||||
|
- `project_context.md` includes older roadmap assumptions such as Celery, Kafka,
|
||||||
|
TimescaleDB, MinIO, and UE5 as active stack elements. The current README and
|
||||||
|
technical docs describe Web Earth, React admin, FastAPI, PostgreSQL/Redis, and
|
||||||
|
`aiprovider` as the active local development shape.
|
||||||
|
- No safe automatic hook system was already configured. This phase documents
|
||||||
|
manual reminders instead of adding hooks.
|
||||||
|
- `.github/workflows/` is absent by design; CI is under `.gitea/workflows/`.
|
||||||
|
|
||||||
|
## Conflicts And Preserved Rules
|
||||||
|
|
||||||
|
| Conflict Or Tension | Resolution |
|
||||||
|
| --- | --- |
|
||||||
|
| Prompt suggested `AGENTS.md`; repository already had `agents.md`. | Added a minimal uppercase compatibility entry and preserved the existing lowercase guide. |
|
||||||
|
| Harness validation could duplicate CI. | Added wrapper scripts that call existing commands and mirror current CI gates where practical. |
|
||||||
|
| Full Docker smoke builds are expensive locally. | Kept them opt-in with `PLANET_HARNESS_DOCKER_SMOKE=1`. |
|
||||||
|
| Internal harness docs could clutter public Docs UI. | Kept `docs/HARNESS.md` and `docs/harness-audit.md` as repository docs, not product Docs entries. |
|
||||||
|
| Existing frontend toolchain is Bun-only. | Harness scripts and docs use Bun only and flag npm/pnpm/yarn lockfiles as failures. |
|
||||||
|
|
||||||
|
## Harness Files Added
|
||||||
|
|
||||||
|
| File | Purpose |
|
||||||
|
| --- | --- |
|
||||||
|
| `AGENTS.md` | Compatibility agent entry point. |
|
||||||
|
| `docs/HARNESS.md` | Harness workflow, validation tiers, conflict policy, and manual reminders. |
|
||||||
|
| `CODEMAP.md` | High-level codebase map and validation references. |
|
||||||
|
| `scripts/harness/doctor.sh` | Environment and repository-shape check. |
|
||||||
|
| `scripts/harness/quick-check.sh` | Fast deterministic local validation. |
|
||||||
|
| `scripts/harness/validate.sh` | Full local validation wrapper with optional delivery smoke. |
|
||||||
@@ -77,6 +77,8 @@ Earth settings are now grouped by `data-settings-tab` and `data-settings-tab-pan
|
|||||||
|
|
||||||
The news category selector in Display reuses the same chip-selector pattern as Cruise Modules. It only filters news categories for the current browser on the Earth frontend. It does not toggle layers, basemap, boundaries, TV, data points, BGP, vessels, satellites, or compute centers; those remain owned by the layer panel, media panel, and admin configuration. `controls.js` persists only `shared.newsCategoryFilters` and broadcasts `earth:news-category-filters-change`; `news.js` sends the selected categories to `/api/v1/news/earth-feed?categories=...&locale=zh-CN`, so Web and UE clients share the same backend category filtering path.
|
The news category selector in Display reuses the same chip-selector pattern as Cruise Modules. It only filters news categories for the current browser on the Earth frontend. It does not toggle layers, basemap, boundaries, TV, data points, BGP, vessels, satellites, or compute centers; those remain owned by the layer panel, media panel, and admin configuration. `controls.js` persists only `shared.newsCategoryFilters` and broadcasts `earth:news-category-filters-change`; `news.js` sends the selected categories to `/api/v1/news/earth-feed?categories=...&locale=zh-CN`, so Web and UE clients share the same backend category filtering path.
|
||||||
|
|
||||||
|
The news panel, ticker, and news cruise must consume `items` / `cruise_items` from the same `/api/v1/news/earth-feed` response instead of keeping separate regional caches. `news.js` builds a refresh request key from region, category, source, and limit; only concurrent requests with the same key reuse the promise, and stale responses from an older region are dropped by token. Source filtering is also region-scoped: when the user moves from Asia Pacific to Europe or another region, source IDs saved for the old region must not be appended to the next fetch. After the new payload arrives, the saved source list is intersected with the available `sources`; if the intersection is empty, the current region falls back to all available sources. This keeps the ticker, panel, and cruise cards aligned after region switches.
|
||||||
|
|
||||||
Shortcut configuration is a device-local preference owned by `controls.js`: read, capture, enable/disable, and reset all stay in the Earth frontend. It should not be written to backend user settings and should not affect other browsers. New shortcuts must provide a default key, display label, disabled/enabled state, and reset path instead of being hard-coded only in a keydown handler.
|
Shortcut configuration is a device-local preference owned by `controls.js`: read, capture, enable/disable, and reset all stay in the Earth frontend. It should not be written to backend user settings and should not affect other browsers. New shortcuts must provide a default key, display label, disabled/enabled state, and reset path instead of being hard-coded only in a keydown handler.
|
||||||
|
|
||||||
### 4. UI and Status Messages
|
### 4. UI and Status Messages
|
||||||
|
|||||||
@@ -160,6 +160,19 @@ When `sources` is omitted, the service layer prefers stories that already have d
|
|||||||
|
|
||||||
Source testing only proves that a specific RSS/Atom/XML feed can be parsed. It does not mean those items have already been written to the news table or are visible in the current region/category view. Saving or resetting news sources increments the configuration version and clears cache; if an enabled feed has no recent stored items, the next `earth-feed` request supplements from RSS so newly enabled sources such as 36Kr and Ebrun are not masked by fresh Google News rows.
|
Source testing only proves that a specific RSS/Atom/XML feed can be parsed. It does not mean those items have already been written to the news table or are visible in the current region/category view. Saving or resetting news sources increments the configuration version and clears cache; if an enabled feed has no recent stored items, the next `earth-feed` request supplements from RSS so newly enabled sources such as 36Kr and Ebrun are not masked by fresh Google News rows.
|
||||||
|
|
||||||
|
## Regional Balancing and Async Enrichment
|
||||||
|
|
||||||
|
`earth_news_items` is the current-state table for EarthFeed. When no explicit `sources` filter is present, the endpoint sorts by Breaking state, region, and publish time, then applies a regional round-robin so Asia Pacific or any other high-volume source cannot fill the entire global view and cruise queue. The default region order is Americas, Europe, Middle East / Africa, Asia Pacific, then Global; unknown regions participate after the known regions. Regional views still follow the active-region-plus-global rule and do not mix unrelated regions into the regional panel.
|
||||||
|
|
||||||
|
Both `items` and `cruise_items` are enqueued for target-location and localization enrichment, but the frontend must not wait for AI before rendering. If the requested display locale is not ready yet, Web Earth falls back to the original `title / summary` so a card does not show a "translation pending" placeholder when readable source content already exists. After translation, classification, Breaking, or target-location enrichment completes, the same `earth_news_items` row is updated and Earth receives a news reload / patch.
|
||||||
|
|
||||||
|
Target-location enrichment uses two Redis Streams queues:
|
||||||
|
|
||||||
|
- `earth_news:target_location:priority`: priority jobs for currently visible `items` and `cruise_items`, with a short dedupe TTL.
|
||||||
|
- `earth_news:target_location:jobs`: normal background enrichment jobs, with a longer dedupe TTL.
|
||||||
|
|
||||||
|
The worker always drains the priority queue before the regular queue; stale pending messages are reclaimed after the idle threshold, each AI job has a hard timeout, and failures go through retry or dead-letter handling. This prevents a large historical regular backlog from starving the Europe, Americas, or other regional news currently visible to the user.
|
||||||
|
|
||||||
## Breaking News Insertion
|
## Breaking News Insertion
|
||||||
|
|
||||||
The news system keeps three separate decisions:
|
The news system keeps three separate decisions:
|
||||||
|
|||||||
@@ -77,6 +77,8 @@ Earth 设置面板现在按 `data-settings-tab` 和 `data-settings-tab-panel`
|
|||||||
|
|
||||||
`显示` 分类里的新闻类型选择复用巡航模块的 chip 选择器形态,只控制星球端当前浏览器的新闻分类显示。它不会打开或关闭图层、底图、边界、TV、数据点、BGP、船舶、卫星或算力中心;这些仍由图层面板、媒体面板和控制台配置各自负责。`controls.js` 只持久化 `shared.newsCategoryFilters` 并广播 `earth:news-category-filters-change`,`news.js` 会把选中的类型拼到 `/api/v1/news/earth-feed?categories=...&locale=zh-CN`,让 Web 和 UE 走同一套后端类型过滤。
|
`显示` 分类里的新闻类型选择复用巡航模块的 chip 选择器形态,只控制星球端当前浏览器的新闻分类显示。它不会打开或关闭图层、底图、边界、TV、数据点、BGP、船舶、卫星或算力中心;这些仍由图层面板、媒体面板和控制台配置各自负责。`controls.js` 只持久化 `shared.newsCategoryFilters` 并广播 `earth:news-category-filters-change`,`news.js` 会把选中的类型拼到 `/api/v1/news/earth-feed?categories=...&locale=zh-CN`,让 Web 和 UE 走同一套后端类型过滤。
|
||||||
|
|
||||||
|
新闻面板、滚动条和新闻巡航必须消费同一次 `/api/v1/news/earth-feed` 响应里的 `items` / `cruise_items`,不能各自缓存区域状态。`news.js` 的刷新请求以区域、类型、来源和数量生成 request key;只有 key 相同的并发请求才复用 promise,旧区域请求返回时会被 token 丢弃。来源过滤也必须按区域作用域处理:当用户从亚太切到欧洲等其它区域时,旧区域保存的来源 ID 不允许继续拼到下一次 fetch 里;拿到新 payload 后再与 `sources` 列表做交集,若没有交集则回退到当前区域所有可用来源。这样滚动条、面板和巡航才会在区域切换后展示同一批新闻。
|
||||||
|
|
||||||
快捷键配置属于设备本地偏好,由 `controls.js` 负责读取、捕获、启用/禁用和重置。它不应写入后端用户设置,也不应影响其它浏览器。后续新增快捷键时,必须同时提供默认键、显示标签、可禁用状态和重置路径,避免只在 keydown handler 中硬编码。
|
快捷键配置属于设备本地偏好,由 `controls.js` 负责读取、捕获、启用/禁用和重置。它不应写入后端用户设置,也不应影响其它浏览器。后续新增快捷键时,必须同时提供默认键、显示标签、可禁用状态和重置路径,避免只在 keydown handler 中硬编码。
|
||||||
|
|
||||||
### 4. UI 与状态消息
|
### 4. UI 与状态消息
|
||||||
|
|||||||
@@ -160,6 +160,19 @@ Web 星球端的新闻类型按钮只保存当前浏览器的显示偏好;偏
|
|||||||
|
|
||||||
源测试只证明当前 RSS/Atom/XML 能解析到条目,不等于这些条目已经入库展示。展示链路还会检查区域、类型过滤和数据库新鲜度。保存或重置新闻源会递增配置版本并清理缓存;如果当前启用的 Feed 子项在库里没有近期条目,下一次 `earth-feed` 请求会补抓,避免新启用的 36氪、亿邦被旧 Google News 缓存挡住。
|
源测试只证明当前 RSS/Atom/XML 能解析到条目,不等于这些条目已经入库展示。展示链路还会检查区域、类型过滤和数据库新鲜度。保存或重置新闻源会递增配置版本并清理缓存;如果当前启用的 Feed 子项在库里没有近期条目,下一次 `earth-feed` 请求会补抓,避免新启用的 36氪、亿邦被旧 Google News 缓存挡住。
|
||||||
|
|
||||||
|
## 区域均衡与异步精修
|
||||||
|
|
||||||
|
`earth_news_items` 是 EarthFeed 的当前状态表。接口在没有显式 `sources` 过滤时会先按 Breaking、区域、发布时间排序,再做区域轮转,避免亚太或任一高频来源把全球视图和巡航队列全部占满。默认区域顺序是美洲、欧洲、中东与非洲、亚太、全球;未知区域只在已知区域之后参与轮转。区域视图仍遵守“当前区域 + global”的规则,不会把其它区域混进区域面板。
|
||||||
|
|
||||||
|
`items` 和 `cruise_items` 都会进入目标位置与本地化精修队列,但前端不能等 AI 完成后再展示。标题或摘要没有目标语言翻译时,Web Earth 先显示原始 `title / summary`,避免卡片出现“新闻汉化中”而实际内容已经可读。后台完成翻译、分类、Breaking 或目标坐标后,会更新同一条 `earth_news_items` 并通过 Earth news reload / patch 刷新前端。
|
||||||
|
|
||||||
|
目标位置队列使用 Redis Streams 两级队列:
|
||||||
|
|
||||||
|
- `earth_news:target_location:priority`:当前可见 `items` 与 `cruise_items` 的优先精修任务,短 TTL 去重。
|
||||||
|
- `earth_news:target_location:jobs`:普通后台精修任务,长 TTL 去重。
|
||||||
|
|
||||||
|
Worker 总是先处理优先队列,再处理普通队列;pending 消息超过空闲阈值会被 reclaim,单条 AI 任务有超时保护,失败后进入重试或 dead letter。这样即使历史普通队列有大量 backlog,当前打开的欧洲、美洲等区域新闻也不会被长队列饿死。
|
||||||
|
|
||||||
## Breaking News 插队
|
## Breaking News 插队
|
||||||
|
|
||||||
新闻体系里有三套互不替代的判断:
|
新闻体系里有三套互不替代的判断:
|
||||||
|
|||||||
@@ -16,12 +16,13 @@
|
|||||||
## Current Version
|
## Current Version
|
||||||
|
|
||||||
- `main` 当前主线历史推导到:`0.16.5`
|
- `main` 当前主线历史推导到:`0.16.5`
|
||||||
- `dev` 当前开发分支历史推导到:`0.71.0`
|
- `dev` 当前开发分支历史推导到:`0.71.1`
|
||||||
|
|
||||||
## Timeline
|
## Timeline
|
||||||
|
|
||||||
| Version | Type | Branch | Commit | Summary |
|
| Version | Type | Branch | Commit | Summary |
|
||||||
| --- | --- | --- | --- | --- |
|
| --- | --- | --- | --- | --- |
|
||||||
|
| `0.71.1` | bugfix | `dev` | `pending` | 修复 Earth 新闻区域切换、滚动条/面板/巡航一致性和新闻精修队列饿死问题,并补充 agent harness 与双语维护文档 |
|
||||||
| `0.71.0` | feature | `dev` | `pending` | Motion Agent 升级为 Web/UE 共用双向控制与真实识别服务,新增 Earth 手动新闻工作流、来源多样化,并完善启动/测试 harness 与双语文档 |
|
| `0.71.0` | feature | `dev` | `pending` | Motion Agent 升级为 Web/UE 共用双向控制与真实识别服务,新增 Earth 手动新闻工作流、来源多样化,并完善启动/测试 harness 与双语文档 |
|
||||||
| `0.70.0` | feature | `dev` | `pending` | 新增后端枚举契约治理、Earth 新闻分类/Breaking 链路和船只当前状态快照,清理错误视口刷新逻辑并同步双语文档 |
|
| `0.70.0` | feature | `dev` | `pending` | 新增后端枚举契约治理、Earth 新闻分类/Breaking 链路和船只当前状态快照,清理错误视口刷新逻辑并同步双语文档 |
|
||||||
| `0.69.0` | feature | `dev` | `pending` | 新增 Earth 新闻源治理、新闻类型服务端过滤、观测日志 fingerprint 聚合和 TV/HLS 播放恢复改进 |
|
| `0.69.0` | feature | `dev` | `pending` | 新增 Earth 新闻源治理、新闻类型服务端过滤、观测日志 fingerprint 聚合和 TV/HLS 播放恢复改进 |
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "planet-frontend",
|
"name": "planet-frontend",
|
||||||
"version": "0.71.0",
|
"version": "0.71.1",
|
||||||
"private": true,
|
"private": true,
|
||||||
"packageManager": "bun@1",
|
"packageManager": "bun@1",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
|
|||||||
@@ -107,6 +107,9 @@ export function getNewsDisplayTitle(item, locale = DEFAULT_LOCALE) {
|
|||||||
if (localized || item?.display_title) {
|
if (localized || item?.display_title) {
|
||||||
return normalizeText(item?.display_title || localized);
|
return normalizeText(item?.display_title || localized);
|
||||||
}
|
}
|
||||||
|
if (item?.title) {
|
||||||
|
return normalizeText(item.title);
|
||||||
|
}
|
||||||
return normalizeText(
|
return normalizeText(
|
||||||
TITLE_PLACEHOLDERS[item?.enrichment_status]
|
TITLE_PLACEHOLDERS[item?.enrichment_status]
|
||||||
|| "新闻汉化中",
|
|| "新闻汉化中",
|
||||||
@@ -118,6 +121,9 @@ export function getNewsDisplaySummary(item, locale = DEFAULT_LOCALE) {
|
|||||||
if (localized || item?.display_summary) {
|
if (localized || item?.display_summary) {
|
||||||
return normalizeText(item?.display_summary || localized);
|
return normalizeText(item?.display_summary || localized);
|
||||||
}
|
}
|
||||||
|
if (item?.summary) {
|
||||||
|
return normalizeText(item.summary);
|
||||||
|
}
|
||||||
return normalizeText(
|
return normalizeText(
|
||||||
SUMMARY_PLACEHOLDERS[item?.enrichment_status]
|
SUMMARY_PLACEHOLDERS[item?.enrichment_status]
|
||||||
|| "中文概要生成中,请稍后刷新。",
|
|| "中文概要生成中,请稍后刷新。",
|
||||||
@@ -126,8 +132,8 @@ export function getNewsDisplaySummary(item, locale = DEFAULT_LOCALE) {
|
|||||||
|
|
||||||
export function isNewsContentReady(item, locale = DEFAULT_LOCALE) {
|
export function isNewsContentReady(item, locale = DEFAULT_LOCALE) {
|
||||||
const localized = getLocalization(item, locale);
|
const localized = getLocalization(item, locale);
|
||||||
const title = normalizeText(item?.display_title || localized.title);
|
const title = normalizeText(item?.display_title || localized.title || item?.title);
|
||||||
const summary = normalizeText(item?.display_summary || localized.summary);
|
const summary = normalizeText(item?.display_summary || localized.summary || item?.summary);
|
||||||
return Boolean(title && summary);
|
return Boolean(title && summary);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -39,6 +39,8 @@ const NEWS_SOURCE_FILTER_STORAGE_KEY = "planet.earth.newsSourceFilters.v1";
|
|||||||
|
|
||||||
let initialized = false;
|
let initialized = false;
|
||||||
let refreshPromise = null;
|
let refreshPromise = null;
|
||||||
|
let refreshRequestKey = "";
|
||||||
|
let activeRefreshToken = 0;
|
||||||
let payload = null;
|
let payload = null;
|
||||||
let lastFocus = null;
|
let lastFocus = null;
|
||||||
let lastFetchAt = 0;
|
let lastFetchAt = 0;
|
||||||
@@ -488,7 +490,8 @@ function getEnabledNewsSourceIds(nextPayload = payload) {
|
|||||||
if (!available.length) return [];
|
if (!available.length) return [];
|
||||||
if (!Array.isArray(activeNewsSourceFilters)) return available;
|
if (!Array.isArray(activeNewsSourceFilters)) return available;
|
||||||
const allowed = new Set(activeNewsSourceFilters);
|
const allowed = new Set(activeNewsSourceFilters);
|
||||||
return available.filter((id) => allowed.has(id));
|
const enabled = available.filter((id) => allowed.has(id));
|
||||||
|
return enabled.length > 0 ? enabled : available;
|
||||||
}
|
}
|
||||||
|
|
||||||
function reconcileNewsSourceFilters(nextPayload = payload) {
|
function reconcileNewsSourceFilters(nextPayload = payload) {
|
||||||
@@ -522,6 +525,15 @@ function getNewsSourceSignature(nextPayload = payload) {
|
|||||||
return enabled.join(",");
|
return enabled.join(",");
|
||||||
}
|
}
|
||||||
|
|
||||||
|
function getNewsSourceSignatureForFetch(lat, lon) {
|
||||||
|
const currentRegion = payload?.focus?.region || null;
|
||||||
|
const nextRegion = inferRegion(lat, lon);
|
||||||
|
if (currentRegion && nextRegion !== currentRegion) {
|
||||||
|
return "";
|
||||||
|
}
|
||||||
|
return getNewsSourceSignature();
|
||||||
|
}
|
||||||
|
|
||||||
function getNewsLimit() {
|
function getNewsLimit() {
|
||||||
return newsFullListMode ? NEWS_FULL_LIMIT : NEWS_SUMMARY_LIMIT;
|
return newsFullListMode ? NEWS_FULL_LIMIT : NEWS_SUMMARY_LIMIT;
|
||||||
}
|
}
|
||||||
@@ -707,6 +719,7 @@ function renderEmptyState(message) {
|
|||||||
|
|
||||||
function renderPayload(nextPayload) {
|
function renderPayload(nextPayload) {
|
||||||
payload = nextPayload;
|
payload = nextPayload;
|
||||||
|
reconcileNewsSourceFilters(nextPayload);
|
||||||
const {
|
const {
|
||||||
board,
|
board,
|
||||||
empty,
|
empty,
|
||||||
@@ -945,9 +958,9 @@ function connectNewsRealtime() {
|
|||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
async function fetchNews(lat, lon) {
|
async function fetchNews(lat, lon, context = {}) {
|
||||||
const categorySignature = getNewsCategorySignature();
|
const categorySignature = context.categorySignature ?? getNewsCategorySignature();
|
||||||
const sourceSignature = getNewsSourceSignature();
|
const sourceSignature = context.sourceSignature ?? getNewsSourceSignatureForFetch(lat, lon);
|
||||||
if (categorySignature === "__none__" || sourceSignature === "__none__") {
|
if (categorySignature === "__none__" || sourceSignature === "__none__") {
|
||||||
return {
|
return {
|
||||||
...(payload || {}),
|
...(payload || {}),
|
||||||
@@ -990,15 +1003,30 @@ async function fetchNews(lat, lon) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
async function refreshNews(lat, lon, { silent = false } = {}) {
|
async function refreshNews(lat, lon, { silent = false } = {}) {
|
||||||
if (refreshPromise) return refreshPromise;
|
const targetRegion = inferRegion(lat, lon);
|
||||||
|
const categorySignature = getNewsCategorySignature();
|
||||||
|
const sourceSignature = getNewsSourceSignatureForFetch(lat, lon);
|
||||||
|
const requestKey = [
|
||||||
|
targetRegion,
|
||||||
|
categorySignature,
|
||||||
|
sourceSignature,
|
||||||
|
getNewsLimit(),
|
||||||
|
].join("|");
|
||||||
|
|
||||||
|
if (refreshPromise && refreshRequestKey === requestKey) return refreshPromise;
|
||||||
|
const requestToken = ++activeRefreshToken;
|
||||||
|
refreshRequestKey = requestKey;
|
||||||
|
|
||||||
const { status } = getElements();
|
const { status } = getElements();
|
||||||
if (status) {
|
if (status) {
|
||||||
status.textContent = "正在同步全球态势新闻...";
|
status.textContent = "正在同步全球态势新闻...";
|
||||||
}
|
}
|
||||||
|
|
||||||
refreshPromise = fetchNews(lat, lon)
|
refreshPromise = fetchNews(lat, lon, { categorySignature, sourceSignature })
|
||||||
.then((nextPayload) => {
|
.then((nextPayload) => {
|
||||||
|
if (requestToken !== activeRefreshToken) {
|
||||||
|
return nextPayload;
|
||||||
|
}
|
||||||
renderPayload(nextPayload);
|
renderPayload(nextPayload);
|
||||||
lastFetchAt = Date.now();
|
lastFetchAt = Date.now();
|
||||||
lastCategorySignature = getNewsCategorySignature();
|
lastCategorySignature = getNewsCategorySignature();
|
||||||
@@ -1012,6 +1040,9 @@ async function refreshNews(lat, lon, { silent = false } = {}) {
|
|||||||
return nextPayload;
|
return nextPayload;
|
||||||
})
|
})
|
||||||
.catch((error) => {
|
.catch((error) => {
|
||||||
|
if (requestToken !== activeRefreshToken) {
|
||||||
|
return null;
|
||||||
|
}
|
||||||
console.error("加载 Earth RSS 新闻失败:", error);
|
console.error("加载 Earth RSS 新闻失败:", error);
|
||||||
const message = error?.name === "AbortError"
|
const message = error?.name === "AbortError"
|
||||||
? "新闻聚合请求超时,请稍后重试"
|
? "新闻聚合请求超时,请稍后重试"
|
||||||
@@ -1024,7 +1055,10 @@ async function refreshNews(lat, lon, { silent = false } = {}) {
|
|||||||
throw error;
|
throw error;
|
||||||
})
|
})
|
||||||
.finally(() => {
|
.finally(() => {
|
||||||
|
if (requestToken === activeRefreshToken) {
|
||||||
refreshPromise = null;
|
refreshPromise = null;
|
||||||
|
refreshRequestKey = "";
|
||||||
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
return refreshPromise;
|
return refreshPromise;
|
||||||
|
|||||||
@@ -37,17 +37,14 @@ export const useAuthStore = create<AuthState>()(
|
|||||||
const { access_token, user } = response.data
|
const { access_token, user } = response.data
|
||||||
set({ token: access_token, user })
|
set({ token: access_token, user })
|
||||||
axios.defaults.headers.common['Authorization'] = `Bearer ${access_token}`
|
axios.defaults.headers.common['Authorization'] = `Bearer ${access_token}`
|
||||||
console.log('[Auth] Logged in, token:', access_token.substring(0, 30) + '...')
|
|
||||||
},
|
},
|
||||||
|
|
||||||
logout() {
|
logout() {
|
||||||
console.log('[Auth] Logging out...')
|
|
||||||
set({ token: null, user: null })
|
set({ token: null, user: null })
|
||||||
delete axios.defaults.headers.common['Authorization']
|
delete axios.defaults.headers.common['Authorization']
|
||||||
},
|
},
|
||||||
|
|
||||||
clearAuth() {
|
clearAuth() {
|
||||||
console.log('[Auth] Clearing all auth data...')
|
|
||||||
set({ token: null, user: null })
|
set({ token: null, user: null })
|
||||||
localStorage.removeItem('auth-storage')
|
localStorage.removeItem('auth-storage')
|
||||||
delete axios.defaults.headers.common['Authorization']
|
delete axios.defaults.headers.common['Authorization']
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
[project]
|
[project]
|
||||||
name = "planet"
|
name = "planet"
|
||||||
version = "0.71.0"
|
version = "0.71.1"
|
||||||
description = "智能星球计划 - 态势感知系统"
|
description = "智能星球计划 - 态势感知系统"
|
||||||
requires-python = ">=3.14"
|
requires-python = ">=3.14"
|
||||||
dependencies = [
|
dependencies = [
|
||||||
|
|||||||
104
scripts/harness/doctor.sh
Executable file
104
scripts/harness/doctor.sh
Executable file
@@ -0,0 +1,104 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
failures=0
|
||||||
|
warnings=0
|
||||||
|
|
||||||
|
ok() {
|
||||||
|
printf "ok: %s\n" "$1"
|
||||||
|
}
|
||||||
|
|
||||||
|
warn() {
|
||||||
|
warnings=$((warnings + 1))
|
||||||
|
printf "warn: %s\n" "$1"
|
||||||
|
}
|
||||||
|
|
||||||
|
fail() {
|
||||||
|
failures=$((failures + 1))
|
||||||
|
printf "fail: %s\n" "$1"
|
||||||
|
}
|
||||||
|
|
||||||
|
check_cmd() {
|
||||||
|
local cmd="$1"
|
||||||
|
if command -v "$cmd" >/dev/null 2>&1; then
|
||||||
|
ok "found command: $cmd"
|
||||||
|
else
|
||||||
|
fail "missing required command: $cmd"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
check_optional_cmd() {
|
||||||
|
local cmd="$1"
|
||||||
|
local reason="$2"
|
||||||
|
if command -v "$cmd" >/dev/null 2>&1; then
|
||||||
|
ok "found optional command: $cmd"
|
||||||
|
else
|
||||||
|
warn "missing optional command: $cmd ($reason)"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
check_file() {
|
||||||
|
local path="$1"
|
||||||
|
if [ -e "$ROOT_DIR/$path" ]; then
|
||||||
|
ok "found file: $path"
|
||||||
|
else
|
||||||
|
fail "missing required file: $path"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
check_absent() {
|
||||||
|
local path="$1"
|
||||||
|
local reason="$2"
|
||||||
|
if [ -e "$ROOT_DIR/$path" ]; then
|
||||||
|
fail "unexpected file: $path ($reason)"
|
||||||
|
else
|
||||||
|
ok "absent as expected: $path"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
|
||||||
|
check_file README.md
|
||||||
|
check_file rules.md
|
||||||
|
check_file project_context.md
|
||||||
|
check_file agents.md
|
||||||
|
check_file AGENTS.md
|
||||||
|
check_file CODEMAP.md
|
||||||
|
check_file docs/HARNESS.md
|
||||||
|
check_file docs/harness-audit.md
|
||||||
|
check_file docs/documentation-coverage-rules.md
|
||||||
|
check_file planet.sh
|
||||||
|
check_file pyproject.toml
|
||||||
|
check_file frontend/package.json
|
||||||
|
check_file .gitea/workflows/ci.yaml
|
||||||
|
|
||||||
|
check_cmd git
|
||||||
|
check_cmd zsh
|
||||||
|
check_cmd uv
|
||||||
|
check_cmd bun
|
||||||
|
|
||||||
|
check_optional_cmd docker "needed for local stack and optional image smoke builds"
|
||||||
|
check_optional_cmd helm "needed for optional Helm delivery smoke checks"
|
||||||
|
|
||||||
|
check_absent frontend/package-lock.json "frontend package management is Bun-only"
|
||||||
|
check_absent frontend/pnpm-lock.yaml "frontend package management is Bun-only"
|
||||||
|
check_absent frontend/yarn.lock "frontend package management is Bun-only"
|
||||||
|
|
||||||
|
if [ ! -x "$ROOT_DIR/planet.sh" ]; then
|
||||||
|
warn "planet.sh is not executable; run it with zsh or restore executable bit"
|
||||||
|
else
|
||||||
|
ok "planet.sh is executable"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$failures" -gt 0 ]; then
|
||||||
|
printf "harness doctor failed: %d failure(s), %d warning(s)\n" "$failures" "$warnings"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
printf "harness doctor passed: %d warning(s)\n" "$warnings"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
34
scripts/harness/quick-check.sh
Executable file
34
scripts/harness/quick-check.sh
Executable file
@@ -0,0 +1,34 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
|
||||||
|
run() {
|
||||||
|
printf "+ %s\n" "$*" >&2
|
||||||
|
"$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
|
||||||
|
run scripts/harness/doctor.sh
|
||||||
|
run git diff --check
|
||||||
|
run zsh -n planet.sh
|
||||||
|
run bash -n scripts/bootstrap-dev.sh
|
||||||
|
run bash -n scripts/harness/doctor.sh
|
||||||
|
run bash -n scripts/harness/quick-check.sh
|
||||||
|
run bash -n scripts/harness/validate.sh
|
||||||
|
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR/backend"
|
||||||
|
run uv run --frozen --group dev --project "$ROOT_DIR" python -m pytest -s \
|
||||||
|
tests/test_api.py \
|
||||||
|
tests/test_realtime_sources.py \
|
||||||
|
-q
|
||||||
|
)
|
||||||
|
|
||||||
|
printf "harness quick check passed\n"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
63
scripts/harness/validate.sh
Executable file
63
scripts/harness/validate.sh
Executable file
@@ -0,0 +1,63 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
DOCKER_SMOKE="${PLANET_HARNESS_DOCKER_SMOKE:-0}"
|
||||||
|
|
||||||
|
run() {
|
||||||
|
printf "+ %s\n" "$*" >&2
|
||||||
|
"$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_helm_smoke_if_available() {
|
||||||
|
if ! command -v helm >/dev/null 2>&1; then
|
||||||
|
printf "warn: helm not found; skipping Helm lint/template smoke\n"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
run helm lint deploy/helm/planet
|
||||||
|
run helm template planet-staging deploy/helm/planet \
|
||||||
|
--namespace planet-staging \
|
||||||
|
-f deploy/helm/planet/values.single-node.yaml \
|
||||||
|
--set image.tag=harness-smoke >/tmp/planet-harness-rendered.yaml
|
||||||
|
}
|
||||||
|
|
||||||
|
run_docker_smoke_if_requested() {
|
||||||
|
case "$DOCKER_SMOKE" in
|
||||||
|
1|true|yes|on)
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
printf "info: Docker image smoke skipped; set PLANET_HARNESS_DOCKER_SMOKE=1 to enable\n"
|
||||||
|
return 0
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
if ! command -v docker >/dev/null 2>&1; then
|
||||||
|
printf "fail: docker not found; cannot run requested image smoke builds\n"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
run docker build -t planet-harness/frontend:smoke ./frontend
|
||||||
|
run docker build -t planet-harness/backend:smoke -f backend/Dockerfile .
|
||||||
|
run docker build -t planet-harness/aiprovider:smoke -f aiprovider/Dockerfile .
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
|
||||||
|
run scripts/harness/quick-check.sh
|
||||||
|
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR/frontend"
|
||||||
|
run bun install --frozen-lockfile
|
||||||
|
run bun run build
|
||||||
|
)
|
||||||
|
|
||||||
|
run_helm_smoke_if_available
|
||||||
|
run_docker_smoke_if_requested
|
||||||
|
|
||||||
|
printf "harness validation passed\n"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Reference in New Issue
Block a user