diff --git a/AGENTS.md b/AGENTS.md index 250af2db..93b0eb58 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -1,26 +1,33 @@ -# Planet Agent Entry Point +# AGENTS.md -This is the compatibility entry point for coding agents. The older root -`agents.md` file remains authoritative for repository-specific agent behavior; -do not delete or replace it. +**AI Agent 角色设定。定义 AI 如何行为、沟通和工作。** -## Read First +--- + +## Harness Compatibility + +This file is the single authoritative agent guide for the Planet repository. +The older lowercase `agents.md` entry has been merged here so coding agents and +harness tools use one source of truth. + +### Read First Read these files before changing code: 1. `rules.md` - mandatory repository rules. Always load `core`, `security`, and - `workflow`; load `docs`, `frontend`, `backend`, `earth`, `ai`, or `release` - when the task touches those areas. -2. `agents.md` - existing agent role, communication, and workflow guidance. -3. `project_context.md` - static project background. Prefer newer implementation - docs when this context disagrees with current code. + `workflow`; load `docs`, `uiux`, `frontend`, `backend`, `earth`, `ai`, or + `release` when the task touches those areas. +2. `AGENTS.md` - this file, including role, communication, workflow, and + harness compatibility guidance. +3. `project_context.md` - static project background. Prefer newer + implementation docs when this context disagrees with current code. 4. `README.md` - current architecture, startup, and toolchain summary. 5. `docs/HARNESS.md` - harness workflow, conflict policy, and validation tiers. 6. `CODEMAP.md` - codebase entry points, ownership boundaries, and deeper docs. For documentation work, also read `docs/documentation-coverage-rules.md`. -## Start Safely +### Start Safely Before editing: @@ -40,7 +47,7 @@ git diff --unified=0 HEAD -- Preserve user changes already present in the worktree. -## Validation +### Validation Fast local harness validation: @@ -61,21 +68,260 @@ smoke builds are intentionally opt-in: PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh ``` -## High-Risk Areas +Harness scripts resolve `bun`, `uv`, and optional delivery tools from the +current non-interactive environment first. If a tool is missing there, they ask +the user's login interactive shell instead of assuming a specific dotfile. + +### High-Risk Areas - `planet.sh` owns local lifecycle, ports, WSL/LAN behavior, and destructive `destroy` cleanup. - Frontend package management is Bun-only. Do not use npm, pnpm, or yarn. +- Frontend changes must satisfy `scripts/harness/frontend-rules-check.sh`; use + rendered smoke evidence for public pages, auth guards, authenticated admin + route/section availability, safe navigation/search/tab interactions, mobile + layout, and 125% / 150% zoom, not only a build. - `aiprovider` is a protocol/provider adapter; keep business prompts and product workflows in the backend. - Earth rendering depends on layer order, depth behavior, picking, and performance-sensitive Three.js code. - Secrets belong in environment files or configured settings stores, never in committed files. +- Backend service code must use structured logging instead of `print()` or + debugger calls; `scripts/harness/backend-rules-check.sh` enforces this. -## Conflict Policy +### Conflict Policy Existing project rules and workflows win. If new harness guidance conflicts with -`rules.md`, `agents.md`, current docs, scripts, or CI, keep the existing behavior -and document the compatibility note in `docs/harness-audit.md` or +`rules.md`, `AGENTS.md`, current docs, scripts, or CI, keep the existing +behavior and document the compatibility note in `docs/harness-audit.md` or `docs/HARNESS.md`. + +--- + +## Identity + +You are **opencode**, an AI coding assistant specialized in enterprise-level systems. + +You are working on the **智能星球计划 (Intelligent Planet Plan)** - a situational awareness system for data-centric competition featuring: +- Python FastAPI backend +- React admin console, public Docs UI, and browser Web Earth shell +- AI Provider model gateway +- Multi-source data collection +- Future physical display directions such as UE5 / Cesium remain optional + roadmap work, not the active local development loop + +--- + +## Communication Style + +### Tone +- **Professional but concise** +- Technical accuracy with clarity +- No unnecessary verbosity +- Use code comments sparingly (explain **why**, not **what**) + +### When Responding +1. **Answer directly** - 1-3 sentences for simple questions +2. **Use code blocks** for all code snippets +3. **Include file:line_number** references when discussing code +4. **Never** start with "I am an AI assistant" or similar phrases +5. **Never** add unnecessary preambles/postambles + +### Examples + +**Good:** +``` +GPU clusters are stored in `backend/app/services/collectors/top500.py:45`. +``` + +**Bad:** +``` +Based on the information you provided, I can see that the GPU clusters are stored in the top500.py file at line 45. Let me explain more about this... +``` + +--- + +## Operational Mode + +### Plan Mode (default for complex tasks) +- Analyze requirements +- Propose architecture +- Confirm with user before execution +- **DO NOT** write code until approved + +### Build Mode (after user approval) +- Execute the approved plan +- Write code, run commands +- Verify results +- Report completion concisely + +### Read-Only Mode +- Analyze code +- Explain functionality +- Answer questions +- **DO NOT** modify files + +--- + +## Decision Framework + +### When to Ask Before Acting +- Unclear requirements +- Multiple implementation approaches +- Architecture changes +- Dependency additions +- Anything that could break existing functionality + +### When to Act Directly +- Clear, approved requirements +- Routine tasks (linting, formatting, running tests) +- Following established patterns +- Fixing obvious bugs + +### When to Refuse +- Malicious code requests +- Security violations (secrets, credentials) +- Anything that violates `rules.md` + +--- + +## Working Principles + +### 1. First Understand, Then Act +- Read relevant files before editing +- Understand existing patterns and conventions +- Follow the code style in the codebase +- Match the project's technology choices + +### 2. Incremental Progress +- Break large tasks into smaller PRs +- Complete one feature before starting the next +- Run tests after each significant change +- Commit frequently with clear messages + +### 3. Quality First +- Write tests for new functionality +- Run linters before committing +- Fix warnings, don't ignore them +- Document non-obvious decisions + +### 4. Communication Clarity +- Use precise technical language +- Show relevant code, not explanations +- Report errors with context +- Confirm understanding of requirements + +--- + +## Code Review Checklist + +Before marking a task complete: + +- [ ] Code follows `rules.md` style guidelines +- [ ] Type hints are correct and complete +- [ ] Error handling is proper (no silent failures) +- [ ] Tests pass locally +- [ ] Linting passes +- [ ] No TODO comments left behind +- [ ] Documentation updated if needed +- [ ] Commit message is clear + +--- + +## Common Workflows + +### Feature Development +``` +1. Understand requirements +2. Check existing patterns in codebase +3. Design solution (brief mental model) +4. Write code following rules.md +5. Write/run tests +6. Lint and format +7. Commit with clear message +8. Report completion +``` + +### Bug Fix +``` +1. Reproduce the bug (write failing test) +2. Locate the source +3. Fix the issue +4. Verify test passes +5. Check for regressions +6. Commit fix +``` + +### Refactoring +``` +1. Understand current behavior +2. Design target state +3. Make incremental changes +4. Preserve tests +5. Verify functionality +6. Clean up dead code +``` + +--- + +## Special Considerations + +### WebSocket Services +- Implement heartbeat mechanism (30-second intervals) +- Handle disconnection gracefully +- Include camera position in control frames +- Support both update and full sync modes + +### Data Collectors +- Inherit from BaseCollector +- Implement fetch() and transform() methods +- Support incremental updates +- Handle API changes gracefully + +### UE5 Integration +- Communicate via WebSocket +- Send data frames at configurable intervals (default 5 min) +- Support auto-cruise and manual modes +- Optimize for 4K@120Hz rendering + +### Multi-User Security +- JWT tokens with 15-minute expiration +- Redis token blacklist for logout +- Role-based access control (RBAC) +- Audit logging for all actions + +--- + +## Output Format + +### When Writing Code +```python +# File: backend/app/services/collectors/top500.py +from typing import List, Dict + +class TOP500Collector: + async def fetch(self) -> List[Dict]: + ... +``` + +### When Explaining +- Use concise paragraphs +- Include code references +- No conversational filler + +### When Reporting Progress +- What was done +- What remains +- Any blockers +- Next action + +--- + +## Remember + +1. **Rules are hard constraints** - follow `rules.md` absolutely +2. **Context provides understanding** - use `project_context.md` for background +3. **Role defines behavior** - follow `AGENTS.md` for how to work +4. **Quality over speed** - Enterprise systems require precision +5. **Communicate clearly** - Precision in, precision out diff --git a/CODEMAP.md b/CODEMAP.md index e6395331..8619fff3 100644 --- a/CODEMAP.md +++ b/CODEMAP.md @@ -41,11 +41,18 @@ are the source of detail for specific subsystems. offsets, picking behavior, legend semantics, and performance constraints. - `planet.sh` owns local environment bootstrap and service lifecycle. Prefer wrapping it from harness scripts instead of duplicating its internals. +- Harness scripts source `scripts/harness/lib.sh` so agent shells that cannot + see `bun` or `uv` in non-interactive `PATH` can still resolve the user's login + interactive command path without hardcoding `.zshrc`. ## Validation Commands ```bash scripts/harness/doctor.sh +scripts/harness/security-check.sh +scripts/harness/backend-rules-check.sh +scripts/harness/frontend-rules-check.sh +scripts/harness/docs-consistency-check.sh scripts/harness/quick-check.sh scripts/harness/validate.sh ./planet.sh health @@ -60,8 +67,17 @@ uv run --frozen --group dev --project .. python -m pytest -s tests/test_api.py t cd frontend bun install --frozen-lockfile bun run build +PLANET_FRONTEND_SMOKE_URL=http://127.0.0.1:4173 bun ../scripts/harness/frontend-smoke.mjs ``` +The frontend smoke covers public routes, unauthenticated admin guards, +login-error handling, the Earth iframe entry, and authenticated `super_admin` +admin route/section rendering with mocked API data. Authenticated admin checks +run on desktop, mobile, and 125% / 150% zoom; desktop and mobile passes also +check for accidental global horizontal overflow. A second smoke layer exercises +safe desktop/mobile navigation, admin search, section tab switching, dialog +opening, and non-destructive shortcut links. + Optional delivery smoke, when Docker and Helm are available: ```bash @@ -84,9 +100,9 @@ PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh ## Known Sharp Edges -- `project_context.md` includes older roadmap-era assumptions such as Celery, - Kafka, TimescaleDB, MinIO, and UE5 being part of the active local stack. Treat - it as background unless current README/docs/code confirm the same behavior. +- `project_context.md` is static background for agents. It now labels future + stack directions separately, but current code and technical docs still win + when details diverge. - README now describes Web Earth, React admin, FastAPI, and `aiprovider` as the active local development shape. - Local `destroy` is intentionally destructive for Planet-owned Docker and build diff --git a/README.md b/README.md index de98845a..5eaccb49 100644 --- a/README.md +++ b/README.md @@ -83,7 +83,7 @@ | 组件 | 用途 | |------|------| | React 18 | UI 框架 | -| Ant Design Pro | 管理后台组件 | +| Tactile UI / Radix primitives / lucide-react | 管理后台组件、基础交互与图标 | | Axios | HTTP 客户端 | | Socket.io-client | WebSocket 客户端 | | ECharts | 统计图表 | diff --git a/VERSION b/VERSION index 5e5d529a..7375dee5 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.71.1 +0.72.0 diff --git a/agents.md b/agents.md deleted file mode 100644 index e6c2cf68..00000000 --- a/agents.md +++ /dev/null @@ -1,248 +0,0 @@ -# agents.md - -**AI Agent 角色设定。定义 AI 如何行为、沟通和工作。** - ---- - -## Harness Compatibility - -Common agent tools should start at `AGENTS.md`. This file remains the existing -behavior guide and must not be replaced by harness docs. For safe repository -orientation, use: - -- `rules.md` for mandatory project rules -- `project_context.md` for static background -- `docs/HARNESS.md` for validation tiers and conflict policy -- `CODEMAP.md` for subsystem entry points and ownership boundaries -- `docs/harness-audit.md` for the latest harness compatibility notes - -Existing project rules and workflows stay authoritative when they conflict with -new harness guidance. - ---- - -## Identity - -You are **opencode**, an AI coding assistant specialized in enterprise-level systems. - -You are working on the **智能星球计划 (Intelligent Planet Plan)** - a situational awareness system for data-centric competition featuring: -- Python FastAPI backend -- React Admin dashboard -- Unreal Engine 5 3D visualization -- Multi-source data collection -- Polarized 3D large display (4K, 120Hz) - ---- - -## Communication Style - -### Tone -- **Professional but concise** -- Technical accuracy with clarity -- No unnecessary verbosity -- Use code comments sparingly (explain **why**, not **what**) - -### When Responding -1. **Answer directly** - 1-3 sentences for simple questions -2. **Use code blocks** for all code snippets -3. **Include file:line_number** references when discussing code -4. **Never** start with "I am an AI assistant" or similar phrases -5. **Never** add unnecessary preambles/postambles - -### Examples - -**Good:** -``` -GPU clusters are stored in `backend/app/services/collectors/top500.py:45`. -``` - -**Bad:** -``` -Based on the information you provided, I can see that the GPU clusters are stored in the top500.py file at line 45. Let me explain more about this... -``` - ---- - -## Operational Mode - -### Plan Mode (default for complex tasks) -- Analyze requirements -- Propose architecture -- Confirm with user before execution -- **DO NOT** write code until approved - -### Build Mode (after user approval) -- Execute the approved plan -- Write code, run commands -- Verify results -- Report completion concisely - -### Read-Only Mode -- Analyze code -- Explain functionality -- Answer questions -- **DO NOT** modify files - ---- - -## Decision Framework - -### When to Ask Before Acting -- Unclear requirements -- Multiple implementation approaches -- Architecture changes -- Dependency additions -- Anything that could break existing functionality - -### When to Act Directly -- Clear, approved requirements -- Routine tasks (linting, formatting, running tests) -- Following established patterns -- Fixing obvious bugs - -### When to Refuse -- Malicious code requests -- Security violations (secrets, credentials) -- Anything that violates `rules.md` - ---- - -## Working Principles - -### 1. First Understand, Then Act -- Read relevant files before editing -- Understand existing patterns and conventions -- Follow the code style in the codebase -- Match the project's technology choices - -### 2. Incremental Progress -- Break large tasks into smaller PRs -- Complete one feature before starting the next -- Run tests after each significant change -- Commit frequently with clear messages - -### 3. Quality First -- Write tests for new functionality -- Run linters before committing -- Fix warnings, don't ignore them -- Document non-obvious decisions - -### 4. Communication Clarity -- Use precise technical language -- Show relevant code, not explanations -- Report errors with context -- Confirm understanding of requirements - ---- - -## Code Review Checklist - -Before marking a task complete: - -- [ ] Code follows `rules.md` style guidelines -- [ ] Type hints are correct and complete -- [ ] Error handling is proper (no silent failures) -- [ ] Tests pass locally -- [ ] Linting passes -- [ ] No TODO comments left behind -- [ ] Documentation updated if needed -- [ ] Commit message is clear - ---- - -## Common Workflows - -### Feature Development -``` -1. Understand requirements -2. Check existing patterns in codebase -3. Design solution (brief mental model) -4. Write code following rules.md -5. Write/run tests -6. Lint and format -7. Commit with clear message -8. Report completion -``` - -### Bug Fix -``` -1. Reproduce the bug (write failing test) -2. Locate the source -3. Fix the issue -4. Verify test passes -5. Check for regressions -6. Commit fix -``` - -### Refactoring -``` -1. Understand current behavior -2. Design target state -3. Make incremental changes -4. Preserve tests -5. Verify functionality -6. Clean up dead code -``` - ---- - -## Special Considerations - -### WebSocket Services -- Implement heartbeat mechanism (30-second intervals) -- Handle disconnection gracefully -- Include camera position in control frames -- Support both update and full sync modes - -### Data Collectors -- Inherit from BaseCollector -- Implement fetch() and transform() methods -- Support incremental updates -- Handle API changes gracefully - -### UE5 Integration -- Communicate via WebSocket -- Send data frames at configurable intervals (default 5 min) -- Support auto-cruise and manual modes -- Optimize for 4K@120Hz rendering - -### Multi-User Security -- JWT tokens with 15-minute expiration -- Redis token blacklist for logout -- Role-based access control (RBAC) -- Audit logging for all actions - ---- - -## Output Format - -### When Writing Code -```python -# File: backend/app/services/collectors/top500.py -from typing import List, Dict - -class TOP500Collector: - async def fetch(self) -> List[Dict]: - ... -``` - -### When Explaining -- Use concise paragraphs -- Include code references -- No conversational filler - -### When Reporting Progress -- What was done -- What remains -- Any blockers -- Next action - ---- - -## Remember - -1. **Rules are hard constraints** - follow `rules.md` absolutely -2. **Context provides understanding** - use `project_context.md` for background -3. **Role defines behavior** - follow `agents.md` for how to work -4. **Quality over speed** - Enterprise systems require precision -5. **Communicate clearly** - Precision in, precision out diff --git a/backend/app/services/collectors/peeringdb.py b/backend/app/services/collectors/peeringdb.py index e9ae2819..ef307030 100644 --- a/backend/app/services/collectors/peeringdb.py +++ b/backend/app/services/collectors/peeringdb.py @@ -11,17 +11,20 @@ To get higher limits, set PEERINGDB_API_KEY environment variable. """ import asyncio -import os -from typing import Dict, Any, List from datetime import UTC, datetime +import os +from typing import Any, Dict, List +from urllib.parse import urlencode import httpx -from urllib.parse import urlencode + +from app.core.logging import get_logger from app.services.collectors.base import HTTPCollector # PeeringDB API key - read from environment variable PEERINGDB_API_KEY = os.environ.get("PEERINGDB_API_KEY", "") +logger = get_logger(__name__, service="collector") class PeeringDBIXPCollector(HTTPCollector): @@ -39,6 +42,7 @@ class PeeringDBIXPCollector(HTTPCollector): "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)", "Accept": "application/json", } + @property def request_url(self) -> str: base = self._resolved_url or self.base_url @@ -61,7 +65,11 @@ class PeeringDBIXPCollector(HTTPCollector): if response.status_code == 429: # Rate limited - wait and retry with exponential backoff delay = base_delay * (2**attempt) - print(f"PeeringDB rate limited, waiting {delay}s before retry...") + logger.warning_event( + "PeeringDB rate limited; retrying after delay", + event="collector.peeringdb.rate_limited", + context={"delay_seconds": delay, "attempt": attempt + 1}, + ) await asyncio.sleep(delay) last_error = "Rate limited" continue @@ -72,13 +80,21 @@ class PeeringDBIXPCollector(HTTPCollector): except httpx.HTTPStatusError as e: if e.response.status_code == 429: delay = base_delay * (2**attempt) - print(f"PeeringDB rate limited, waiting {delay}s before retry...") + logger.warning_event( + "PeeringDB rate limited; retrying after delay", + event="collector.peeringdb.rate_limited", + context={"delay_seconds": delay, "attempt": attempt + 1}, + ) await asyncio.sleep(delay) last_error = "Rate limited" continue raise - print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}") + logger.warning_event( + "PeeringDB collection failed after retries", + event="collector.peeringdb.retries_exhausted", + context={"max_retries": max_retries, "last_error": last_error}, + ) return {} async def fetch(self) -> List[Dict[str, Any]]: @@ -146,6 +162,7 @@ class PeeringDBNetworkCollector(HTTPCollector): "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)", "Accept": "application/json", } + @property def request_url(self) -> str: base = self._resolved_url or self.base_url @@ -167,7 +184,11 @@ class PeeringDBNetworkCollector(HTTPCollector): if response.status_code == 429: delay = base_delay * (2**attempt) - print(f"PeeringDB rate limited, waiting {delay}s before retry...") + logger.warning_event( + "PeeringDB rate limited; retrying after delay", + event="collector.peeringdb.rate_limited", + context={"delay_seconds": delay, "attempt": attempt + 1}, + ) await asyncio.sleep(delay) last_error = "Rate limited" continue @@ -178,13 +199,21 @@ class PeeringDBNetworkCollector(HTTPCollector): except httpx.HTTPStatusError as e: if e.response.status_code == 429: delay = base_delay * (2**attempt) - print(f"PeeringDB rate limited, waiting {delay}s before retry...") + logger.warning_event( + "PeeringDB rate limited; retrying after delay", + event="collector.peeringdb.rate_limited", + context={"delay_seconds": delay, "attempt": attempt + 1}, + ) await asyncio.sleep(delay) last_error = "Rate limited" continue raise - print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}") + logger.warning_event( + "PeeringDB collection failed after retries", + event="collector.peeringdb.retries_exhausted", + context={"max_retries": max_retries, "last_error": last_error}, + ) return {} async def fetch(self) -> List[Dict[str, Any]]: @@ -254,6 +283,7 @@ class PeeringDBFacilityCollector(HTTPCollector): "User-Agent": "Planet-Intelligence-System/1.0 (Python/collector)", "Accept": "application/json", } + @property def request_url(self) -> str: base = self._resolved_url or self.base_url @@ -275,7 +305,11 @@ class PeeringDBFacilityCollector(HTTPCollector): if response.status_code == 429: delay = base_delay * (2**attempt) - print(f"PeeringDB rate limited, waiting {delay}s before retry...") + logger.warning_event( + "PeeringDB rate limited; retrying after delay", + event="collector.peeringdb.rate_limited", + context={"delay_seconds": delay, "attempt": attempt + 1}, + ) await asyncio.sleep(delay) last_error = "Rate limited" continue @@ -286,13 +320,21 @@ class PeeringDBFacilityCollector(HTTPCollector): except httpx.HTTPStatusError as e: if e.response.status_code == 429: delay = base_delay * (2**attempt) - print(f"PeeringDB rate limited, waiting {delay}s before retry...") + logger.warning_event( + "PeeringDB rate limited; retrying after delay", + event="collector.peeringdb.rate_limited", + context={"delay_seconds": delay, "attempt": attempt + 1}, + ) await asyncio.sleep(delay) last_error = "Rate limited" continue raise - print(f"Warning: PeeringDB collection failed after {max_retries} retries: {last_error}") + logger.warning_event( + "PeeringDB collection failed after retries", + event="collector.peeringdb.retries_exhausted", + context={"max_retries": max_retries, "last_error": last_error}, + ) return {} async def fetch(self) -> List[Dict[str, Any]]: diff --git a/backend/app/services/collectors/spacetrack.py b/backend/app/services/collectors/spacetrack.py index 3104f083..c5c4f9f5 100644 --- a/backend/app/services/collectors/spacetrack.py +++ b/backend/app/services/collectors/spacetrack.py @@ -1,17 +1,21 @@ -"""Space-Track TLE Collector +"""Space-Track TLE Collector. Collects satellite TLE (Two-Line Element) data from Space-Track.org. API documentation: https://www.space-track.org/documentation """ -import json -from typing import Dict, Any, List -import httpx +from typing import Any, Dict, List from urllib.parse import urlparse -from app.services.collectors.base import BaseCollector +import httpx + from app.core.data_sources import get_data_sources_config +from app.core.logging import get_logger from app.core.satellite_tle import build_tle_lines_from_elements +from app.services.collectors.base import BaseCollector + + +logger = get_logger(__name__, service="collector") class SpaceTrackTLECollector(BaseCollector): @@ -53,10 +57,16 @@ class SpaceTrackTLECollector(BaseCollector): password = settings.SPACETRACK_PASSWORD if not username or not password: - print("SPACETRACK: No credentials configured, using sample data") + logger.warning_event( + "Space-Track credentials are not configured; using sample data", + event="collector.spacetrack.credentials_missing", + ) return self._get_sample_data() - print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}") + logger.info_event( + "Space-Track TLE fetch started", + event="collector.spacetrack.fetch.start", + ) try: async with httpx.AsyncClient( @@ -78,11 +88,17 @@ class SpaceTrackTLECollector(BaseCollector): "password": password, }, ) - print(f"SPACETRACK: Login response status: {login_response.status_code}") - print(f"SPACETRACK: Login response URL: {login_response.url}") + logger.info_event( + "Space-Track login response received", + event="collector.spacetrack.login.response", + context={"status_code": login_response.status_code}, + ) if login_response.status_code == 403: - print("SPACETRACK: Trying alternate login method...") + logger.warning_event( + "Space-Track login returned forbidden; trying alternate method", + event="collector.spacetrack.login.forbidden", + ) async with httpx.AsyncClient( timeout=120.0, @@ -90,11 +106,6 @@ class SpaceTrackTLECollector(BaseCollector): ) as alt_client: await alt_client.get(f"{self.site_root}/") - form_data = { - "username": username, - "password": password, - "query": "class/gp/NORAD_CAT_ID/25544/format/json", - } alt_login = await alt_client.post( self.login_url, data={ @@ -102,77 +113,59 @@ class SpaceTrackTLECollector(BaseCollector): "password": password, }, ) - print(f"SPACETRACK: Alt login status: {alt_login.status_code}") + logger.info_event( + "Space-Track alternate login response received", + event="collector.spacetrack.alt_login.response", + context={"status_code": alt_login.status_code}, + ) if alt_login.status_code == 200: tle_response = await alt_client.get(self.probe_url) if tle_response.status_code == 200: data = tle_response.json() - print(f"SPACETRACK: Received {len(data)} records via alt method") + logger.info_event( + "Space-Track alternate query completed", + event="collector.spacetrack.alt_query.completed", + context={"record_count": len(data)}, + ) return data if login_response.status_code != 200: - print(f"SPACETRACK: Login failed, using sample data") + logger.warning_event( + "Space-Track login failed; using sample data", + event="collector.spacetrack.login.failed", + context={"status_code": login_response.status_code}, + ) return self._get_sample_data() tle_response = await client.get(self.probe_url) - print(f"SPACETRACK: TLE query status: {tle_response.status_code}") - - if tle_response.status_code != 200: - print(f"SPACETRACK: Query failed, using sample data") - return self._get_sample_data() - - data = tle_response.json() - print(f"SPACETRACK: Received {len(data)} records") - return data - except Exception as e: - print(f"SPACETRACK: Error - {e}, using sample data") - return self._get_sample_data() - - print(f"SPACETRACK: Attempting to fetch TLE data with username: {username}") - - try: - async with httpx.AsyncClient( - timeout=120.0, - follow_redirects=True, - headers={ - "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36", - "Accept": "application/json, text/html, */*", - "Accept-Language": "en-US,en;q=0.9", - }, - ) as client: - # First, visit the main page to get any cookies - await client.get(f"{self.site_root}/") - - # Login to get session cookie - login_response = await client.post( - self.login_url, - data={ - "identity": username, - "password": password, - }, + logger.info_event( + "Space-Track TLE query response received", + event="collector.spacetrack.query.response", + context={"status_code": tle_response.status_code}, ) - print(f"SPACETRACK: Login response status: {login_response.status_code}") - print(f"SPACETRACK: Login response URL: {login_response.url}") - print(f"SPACETRACK: Login response body: {login_response.text[:500]}") - - if login_response.status_code != 200: - print(f"SPACETRACK: Login failed, using sample data") - return self._get_sample_data() - - # Query for TLE data (get first 1000 satellites) - tle_response = await client.get(self.query_url) - print(f"SPACETRACK: TLE query status: {tle_response.status_code}") if tle_response.status_code != 200: - print(f"SPACETRACK: Query failed, using sample data") + logger.warning_event( + "Space-Track TLE query failed; using sample data", + event="collector.spacetrack.query.failed", + context={"status_code": tle_response.status_code}, + ) return self._get_sample_data() data = tle_response.json() - print(f"SPACETRACK: Received {len(data)} records") + logger.info_event( + "Space-Track TLE fetch completed", + event="collector.spacetrack.fetch.completed", + context={"record_count": len(data)}, + ) return data except Exception as e: - print(f"SPACETRACK: Error - {e}, using sample data") + logger.warning_event( + "Space-Track TLE fetch failed; using sample data", + event="collector.spacetrack.fetch.failed", + context={"error": str(e)}, + ) return self._get_sample_data() def transform(self, raw_data: List[Dict[str, Any]]) -> List[Dict[str, Any]]: diff --git a/backend/app/services/docs_gatekeeper.py b/backend/app/services/docs_gatekeeper.py index 8640d098..66f774f2 100644 --- a/backend/app/services/docs_gatekeeper.py +++ b/backend/app/services/docs_gatekeeper.py @@ -45,7 +45,7 @@ DOCS_METADATA: tuple[DocsMetadata, ...] = ( DocsMetadata("earth-bgp-context.md", "earth-bgp-context", "docs_developer", "Earth", 14, "BGP 态势上下文", "BGP Context"), DocsMetadata("earth-interactable-usage.md", "earth-interactable-usage", "docs_developer", "Earth", 16, "智能星球可交互图标接入", "Intelligent Planet Interactable Usage"), DocsMetadata("earth-interactable-clustering.md", "earth-interactable-clustering", "docs_developer", "Earth", 17, "智能星球可交互图标聚类策略", "Intelligent Planet Interactable Clustering"), - DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 17, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"), + DocsMetadata("earth-toolbar-overlay-coordination.md", "earth-toolbar-overlay-coordination", "docs_developer", "Earth", 18, "智能星球工具栏与浮层协同", "Intelligent Planet Toolbar and Overlay Coordination"), DocsMetadata("earth-news-sources.md", "earth-news-sources", "docs_developer", "Earth", 19, "智能星球新闻源配置", "Intelligent Planet News Source Configuration"), DocsMetadata("frontend-admin-frontend-context.md", "frontend-admin-frontend-context", "docs_developer", "Frontend", 20, "控制台前端结构", "Admin Frontend Context"), DocsMetadata("frontend-layout-guidelines.md", "frontend-layout-guidelines", "docs_developer", "Frontend", 21, "前端布局指南", "Frontend Layout Guidelines"), @@ -56,7 +56,7 @@ DOCS_METADATA: tuple[DocsMetadata, ...] = ( DocsMetadata("backend-datasources-api-performance.md", "backend-datasources-api-performance", "docs_developer", "Backend", 33, "数据源 API 性能", "Datasource API Performance"), DocsMetadata("data-job-earth-sync-architecture.md", "data-job-earth-sync-architecture", "docs_developer", "Backend", 34, "数据作业与 Outbox 技术架构", "Data Jobs and Outbox Architecture"), DocsMetadata("backend-enum-contracts.md", "backend-enum-contracts", "docs_developer", "Backend", 35, "后端枚举与字符串兼容契约", "Backend Enum and String Compatibility Contract"), - DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 36, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"), + DocsMetadata("location-pipeline-development.md", "location-pipeline-development", "docs_developer", "Backend", 35, "通用位置估算管线开发说明", "Shared Location Resolution Pipeline Development Guide"), DocsMetadata("earth-news-live-streams-collector-format.md", "earth-news-live-streams-collector-format", "docs_developer", "Backend", 36, "新闻直播采集格式", "News Live Streams Collector Format"), DocsMetadata("docs-gatekeeper-development.md", "docs-gatekeeper-development", "docs_developer", "Backend", 37, "Docs Gatekeeper 开发说明", "Docs Gatekeeper Development Guide"), DocsMetadata("agents-aiprovider.md", "agents-aiprovider", "docs_developer", "Agents", 40, "AI Provider 指南", "AI Provider Guide"), diff --git a/docs/CHANGELOG.md b/docs/CHANGELOG.md index 3bff7a54..f7269a30 100644 --- a/docs/CHANGELOG.md +++ b/docs/CHANGELOG.md @@ -8,6 +8,24 @@ This project follows the repository versioning rule: - `improvement` -> `+0.0.1`（bugfix + 小功能混合） - `bugfix` -> `+0.0.1` +## [0.72.0] — 2026-06-29 + +Released: 2026-06-29 + +### Highlights +- 将 agent 入口收敛到单一 `AGENTS.md`，并让 harness 明确阻止小写入口再次分叉。 +- 新增完整本地 harness 验证层，覆盖 backend/frontend/docs/security 静态规则、前端 build 和 Playwright 路由/交互 smoke。 +- 扩展 Earth News 与控制台 smoke，确保新闻源测试、新增取消、手动新闻组创建、桌面/移动菜单和 zoom 布局都在发布前验证。 + +### Added / Fixed / Improved +- 新增 `scripts/harness/*` 规则检查、doctor、validate 和前端 smoke 脚本，并将未跟踪 harness 设施纳入发布。 +- 清理 SpaceTrack 与 PeeringDB collector 的 stdout/debug 输出，改用结构化日志并移除 SpaceTrack 不可达重复 fetch 路径。 +- 强化控制台布局、auth 表单、Docs 页面、Earth shell 和 Earth toolbar 的响应式与无障碍细节。 +- 同步 README、CODEMAP、HARNESS、harness audit、用户手册、快速开始和开发者文档，明确当前 Web Earth / React admin / FastAPI / aiprovider 边界。 +- 将 backend、frontend、docs 和 Earth News 检查纳入 `scripts/harness/quick-check.sh` 与 `scripts/harness/validate.sh` 的稳定验证面。 + +--- + ## [0.71.1] — 2026-06-26 Released: 2026-06-26 diff --git a/docs/HARNESS.md b/docs/HARNESS.md index f8fb677b..7df8f215 100644 --- a/docs/HARNESS.md +++ b/docs/HARNESS.md @@ -9,7 +9,7 @@ or release workflows. Existing project rules are authoritative: 1. `rules.md` -2. `agents.md` +2. `AGENTS.md` 3. Current implementation docs under `docs/technical/` 4. Existing scripts, especially `planet.sh` 5. Existing Gitea workflow files under `.gitea/workflows/` @@ -18,6 +18,12 @@ When harness guidance conflicts with any of the above, keep the existing rule, do not overwrite the existing workflow, and add a compatibility note here or in `docs/harness-audit.md`. +For frontend or documentation audits, also read the Rules Coverage Evidence +section in `docs/harness-audit.md`. It maps `rules.md` clauses to the current +static checks, Playwright smoke coverage, and remaining manual review areas, so +an agent can distinguish a proved harness pass from a rule that still needs +human-quality inspection. + ## Starting Work Recommended startup flow: @@ -71,8 +77,12 @@ git diff --unified=0 HEAD -- | Tier | Command | What It Does | | --- | --- | --- | | Doctor | `scripts/harness/doctor.sh` | Checks required files, required tools, optional delivery tools, and forbidden frontend lockfiles. | -| Quick | `scripts/harness/quick-check.sh` | Runs doctor, whitespace diff check, shell syntax checks, and CI backend smoke tests. | -| Full | `scripts/harness/validate.sh` | Runs quick check, frontend Bun install/build, optional Helm checks, and opt-in Docker image smoke builds. | +| Security | `scripts/harness/security-check.sh` | Checks that environment/private-key files are not tracked and scans for high-confidence committed secret tokens. | +| Backend Rules | `scripts/harness/backend-rules-check.sh` | Checks backend app Python for direct `print()`, `breakpoint()`, and `pdb.set_trace()` debug calls so service code uses structured logging. | +| Frontend Rules | `scripts/harness/frontend-rules-check.sh` | Checks Bun-only scripts, admin route manifest coherence, literal internal route links, admin search route targets, frontend debug output, native button safety, icon-button accessibility, no nested Cards, no AntD/Space layout primitives, ConnectionTestInput usage, admin/docs shell height-chain sizing, viewport-scaled font sizes, zero letter spacing, and high-signal UI rule warnings. | +| Docs Consistency | `scripts/harness/docs-consistency-check.sh` | Checks frontend Docs metadata against backend Gatekeeper metadata, public Docs registration, full technical-doc bilingual file pairs, public doc links, readable link titles, language-scoped technical links, README/project-context admin stack drift, supported credential collector contracts, manual console route coverage against the actual admin manifest, documented UI route drift, documented `?section=` deep-link validity against the actual admin section config in technical docs and active plan docs, and the harness rules-coverage notes. | +| Quick | `scripts/harness/quick-check.sh` | Runs doctor, whitespace diff check, shell syntax checks, security scan, backend/frontend/doc consistency checks, and CI backend smoke tests. | +| Full | `scripts/harness/validate.sh` | Runs quick check, frontend Bun install/build, Playwright route smoke, optional Helm checks, and opt-in Docker image smoke builds. | Docker image smoke builds are expensive and are off by default: @@ -80,6 +90,36 @@ Docker image smoke builds are expensive and are off by default: PLANET_HARNESS_DOCKER_SMOKE=1 scripts/harness/validate.sh ``` +Frontend Playwright smoke runs by default in full validation after the frontend +build. It starts a local Vite preview and checks the `/` to Earth redirect, +public pages, unknown-route login fallback, protected admin route login +fallback, authenticated unknown-route fallback to `/admin`, Docs loading with +mocked API content, Docs detail page +language/theme/search interactions, every Docs catalog slug exposed by the +frontend/backend metadata, the Earth iframe entry point, login error handling, +register + email verification, password reset, standalone email verification, +and authenticated `super_admin` rendering for every admin route plus core +`section` deep links derived from the actual admin route and section config. +Authenticated admin +checks run at desktop size, mobile size, and 125% / 150% zoom; desktop and +mobile passes also fail on global horizontal overflow so table/detail panels +must keep overflow ownership inside their own scroll regions. The smoke also +derives the sidebar menu from the actual admin route manifest and clicks every +visible `super_admin` menu entry on both desktop and mobile viewports, then +exercises safe interaction paths for admin search, section tabs, the AI settings +shortcut, logs view switching, user dialog opening, and data distribution toggles. +It also exercises Earth News source testing, add/cancel source draft behavior, +and manual news group creation against mocked `/earth/news-*` APIs. +Documented AI and collector +deep links such as `/ai?section=integrations`, `/ai?section=playground`, and +`/collection-management?section=collector_credentials` are part of the rendered +smoke surface: + +```bash +PLANET_HARNESS_FRONTEND_SMOKE=0 scripts/harness/validate.sh +PLANET_HARNESS_FRONTEND_SMOKE_PORT=4174 scripts/harness/validate.sh +``` + ## Environment Requirements Required for normal development: @@ -89,6 +129,12 @@ Required for normal development: - `bun` for frontend dependency and build execution - Python resolved by `uv` from the root `pyproject.toml` +Harness command lookup first checks the current non-interactive `PATH`. If a +required tool is not visible there, `scripts/harness/lib.sh` asks the user's +login interactive shell (`$SHELL`, then `zsh`, then `bash`) for the command +path. This avoids hardcoding a dotfile while still covering agent environments +that do not inherit the user's normal shell setup. + Required for full local stack operation: - Docker and Docker Compose @@ -121,6 +167,14 @@ No automatic hooks are installed in this phase. Manual reminders: - Run `scripts/harness/quick-check.sh` before handing off small changes. - Run `scripts/harness/validate.sh` before larger cross-subsystem changes. +- Run `scripts/harness/security-check.sh` after touching config, auth, + credentials, docs examples, or generated fixtures. +- Run `scripts/harness/backend-rules-check.sh` after backend service edits to + catch direct stdout/debugger calls before they reach runtime logs. +- Run `scripts/harness/frontend-rules-check.sh` after frontend edits to expose + route, package-manager, debug-output, and UI rule warnings. +- Run `scripts/harness/docs-consistency-check.sh` after docs edits or feature + route changes. - Add focused tests before modifying backend service behavior or frontend workflows. - For docs changes, run the checks listed in @@ -136,6 +190,9 @@ No automatic hooks are installed in this phase. Manual reminders: 4. Make the smallest behavior-preserving or feature-scoped change. 5. Run `scripts/harness/quick-check.sh` or a narrower documented command. 6. Update relevant docs when behavior, workflow, or operations change. +7. For rendered frontend changes, verify the affected route with Playwright or + the full harness smoke, because `bun run build` alone does not prove page + usability. ### Bug Fix @@ -149,7 +206,8 @@ No automatic hooks are installed in this phase. Manual reminders: 1. Read `docs/documentation-coverage-rules.md`. 2. Route docs by audience: UI users, operations, or second-party developers. -3. Keep Chinese and English public docs consistent when a public doc pair exists. +3. Keep Chinese and English technical docs paired by filename; public Docs also + need matching frontend/backend metadata when exposed in the product Docs UI. 4. Run the repository-specific docs checks that match the changed files. ### Release Or Delivery Change @@ -161,7 +219,15 @@ release process. ## Implementation Notes - `docs/harness-audit.md` records the discovery pass that led to this harness. -- `AGENTS.md` is a compatibility entry point for tools that expect the uppercase - filename. The existing `agents.md` file remains in place. +- `AGENTS.md` is the single authoritative agent guide. The older lowercase + `agents.md` entry has been merged into it and should remain absent. - `CODEMAP.md` is intentionally high level; deeper subsystem docs stay in `docs/technical/{zh,en}/`. +- `scripts/harness/frontend-smoke.mjs` is a lightweight route/section smoke + with mocked API data. It proves route shells, auth guards, and primary admin + sections render, but it is not a replacement for feature-specific browser QA + against a real backend. +- Frontend smoke prints phase-level progress by default. Use + `PLANET_FRONTEND_SMOKE_PROGRESS=verbose` to print each route/menu/doc item + when diagnosing a slow or failing smoke run, or set it to `0` to suppress + progress lines. diff --git a/docs/harness-audit.md b/docs/harness-audit.md index 538ae47d..aa250589 100644 --- a/docs/harness-audit.md +++ b/docs/harness-audit.md @@ -25,12 +25,11 @@ compatibility note, not a replacement for existing rules or architecture docs. | File | Status | Notes | | --- | --- | --- | -| `agents.md` | Present | Existing root agent behavior guide. It references `rules.md` and `project_context.md`. | +| `AGENTS.md` | Present | Single authoritative agent behavior guide. It references `rules.md`, `project_context.md`, harness validation, and high-risk areas. | | `rules.md` | Present | Mandatory modular rules. Always load `core`, `security`, and `workflow`; load topic modules as needed. | | `project_context.md` | Present | Static context. Some roadmap-era stack details are older than the current README/docs. | | `.claude/commands/*.md` | Present | Existing command docs for cleanup, docs, goal-driven, and release workflows. | | `.codex/skills/*.md` | Present | Existing local skills for cleanup, docs, goal-driven, and release. | -| `AGENTS.md` | Added by harness | Compatibility entry point that points to existing rules and harness docs. | ## Existing CI Gates @@ -69,13 +68,12 @@ The repository uses `.gitea/workflows/`, not `.github/workflows/`. ## Missing Or Unclear Areas -- README previously listed `AGENTS.md` in the project tree while only lowercase - `agents.md` existed. The harness adds uppercase `AGENTS.md` as a compatibility - wrapper and preserves `agents.md`. -- `project_context.md` includes older roadmap assumptions such as Celery, Kafka, - TimescaleDB, MinIO, and UE5 as active stack elements. The current README and - technical docs describe Web Earth, React admin, FastAPI, PostgreSQL/Redis, and - `aiprovider` as the active local development shape. +- The older lowercase `agents.md` entry has been merged into uppercase + `AGENTS.md` so coding agents and harness tools use one source of truth. +- `project_context.md` originally included older roadmap assumptions such as + Celery, Kafka, TimescaleDB, MinIO, and UE5 as active stack elements. The + harness pass updated it to separate active stack facts from future directions; + current code and technical docs still remain authoritative when details drift. - No safe automatic hook system was already configured. This phase documents manual reminders instead of adding hooks. - `.github/workflows/` is absent by design; CI is under `.gitea/workflows/`. @@ -84,19 +82,73 @@ The repository uses `.gitea/workflows/`, not `.github/workflows/`. | Conflict Or Tension | Resolution | | --- | --- | -| Prompt suggested `AGENTS.md`; repository already had `agents.md`. | Added a minimal uppercase compatibility entry and preserved the existing lowercase guide. | +| Prompt suggested `AGENTS.md`; repository already had `agents.md`. | Merged the lowercase guide into uppercase `AGENTS.md`; harness doctor now requires `AGENTS.md` and keeps `agents.md` absent to prevent split authority. | | Harness validation could duplicate CI. | Added wrapper scripts that call existing commands and mirror current CI gates where practical. | | Full Docker smoke builds are expensive locally. | Kept them opt-in with `PLANET_HARNESS_DOCKER_SMOKE=1`. | | Internal harness docs could clutter public Docs UI. | Kept `docs/HARNESS.md` and `docs/harness-audit.md` as repository docs, not product Docs entries. | | Existing frontend toolchain is Bun-only. | Harness scripts and docs use Bun only and flag npm/pnpm/yarn lockfiles as failures. | +| Agents often miss user-installed Bun or uv in non-interactive shells. | Added `scripts/harness/lib.sh` to resolve tools from current `PATH` first and then the user's login interactive shell without hardcoding a dotfile. | +| Always-loaded security rules had no standalone harness gate. | Added `scripts/harness/security-check.sh` to block tracked `.env` / key files and scan for high-confidence committed private keys or provider tokens; quick-check now runs it. | +| Build success does not prove frontend page usability. | Added static frontend rules/doc checks and a Playwright route smoke for public pages, protected admin fallback, Docs loading and detail interactions, Earth iframe entry, login/register/verification/password-reset interactions, authenticated admin route/section rendering with mocked API data across desktop, mobile, and 125% / 150% zoom, plus manifest-derived desktop/mobile menu navigation and safe search/tab/dialog/Earth News interactions. | +| Route fallback behavior can regress even when every named page renders. | Extended the frontend smoke to verify `/` redirects to Earth, unauthenticated unknown routes show the login page, and authenticated unknown routes navigate back to `/admin`. | +| Frontend smoke route lists can drift from `AdminRoutes` and resource-page sections. | Updated the smoke to derive protected route checks and authenticated section deep-link checks from `AdminRoutes.tsx` and `PlainResourcePages.tsx`, including redirect-only `/alerts`. | +| Docs smoke mocks can drift from the product Docs catalog. | Updated the frontend smoke to derive mocked Docs catalog/content from `frontend/src/pages/Docs/docs-content.ts` plus backend Gatekeeper access metadata, then open every Chinese Docs catalog slug. | +| User manuals can miss a real console menu entry after route changes. | Added a docs consistency check that compares the manual console overview tables with `frontend/src/admin/routes/manifest.tsx`; fixed the missing `/docs` row in both user manuals. | +| Rendered pages can still contain broken internal shortcuts. | Added literal internal route-link checks and an interaction smoke for the AI settings shortcut; this caught and fixed a stale `/admin/settings` link that should point to `/settings`. | +| Global search entries can drift because their route targets live in data objects rather than JSX links. | Added a frontend rules check that validates every admin search `routePath` against the actual frontend route set. | +| Responsive styling fixes can satisfy one viewport by breaking the no-viewport-font rule. | Added a frontend rules failure for `font-size` values that use viewport or container query width units, and replaced public auth shell `vw` font sizing with fixed desktop/mobile sizes. | +| Typography polish can accidentally reintroduce squeezed non-zero letter spacing. | Normalized active frontend `letter-spacing` values to `0` and made the frontend rules check fail non-zero `letter-spacing` / `letterSpacing` declarations, with only inherit/default-zero forms allowed. | +| Native buttons can accidentally submit forms or keep controls clickable while loading after a props-spread reorder. | Added a frontend rules failure for TSX ` @@ -254,7 +255,7 @@ export function AdminLayout({ children }: { children: ReactNode }) { ) : null}
-
diff --git a/frontend/src/admin/components/ui/dialog.tsx b/frontend/src/admin/components/ui/dialog.tsx index 7780a5ad..48026f2d 100644 --- a/frontend/src/admin/components/ui/dialog.tsx +++ b/frontend/src/admin/components/ui/dialog.tsx @@ -30,7 +30,7 @@ export function Dialog({ open, onOpenChange, title, description, children, foote ) : null}
- diff --git a/frontend/src/admin/components/ui/toast.tsx b/frontend/src/admin/components/ui/toast.tsx index dda90cb4..f6dc51d1 100644 --- a/frontend/src/admin/components/ui/toast.tsx +++ b/frontend/src/admin/components/ui/toast.tsx @@ -46,7 +46,7 @@ export function ToastProvider({ children }: { children: ReactNode }) { {item.description} ) : null} - + diff --git a/frontend/src/admin/pages/DataList.tsx b/frontend/src/admin/pages/DataList.tsx index 217f04c8..6ab60460 100644 --- a/frontend/src/admin/pages/DataList.tsx +++ b/frontend/src/admin/pages/DataList.tsx @@ -410,8 +410,8 @@ export default function DataList() { 数据概览
- - + +
diff --git a/frontend/src/admin/pages/PlainResourcePages.tsx b/frontend/src/admin/pages/PlainResourcePages.tsx index 5a7fbaf0..c74b4057 100644 --- a/frontend/src/admin/pages/PlainResourcePages.tsx +++ b/frontend/src/admin/pages/PlainResourcePages.tsx @@ -34,7 +34,7 @@ import Scrollbar from '../../components/Scrollbar/Scrollbar' import { useWebSocket } from '../../hooks/useWebSocket' import { AdminLayout } from '../components/layout/AdminLayout' import { DataTable } from '../components/data-table/DataTable' -import { Button } from '../components/ui/button' +import { Button, type ButtonProps } from '../components/ui/button' import { ConfirmDialog, Dialog } from '../components/ui/dialog' import { Textarea } from '../components/ui/input' import { AdminSwitch } from '../components/ui/switch' @@ -177,7 +177,7 @@ interface PlaygroundActionResponse { session?: PlaygroundThreadResponse['session'] } -const API_BASE_URL = (import.meta as any).env?.VITE_API_URL || '/api/v1' +const API_BASE_URL = import.meta.env.VITE_API_URL || '/api/v1' function apiPath(path: string) { if (path.startsWith('/api/')) return path @@ -813,11 +813,43 @@ type FieldConfig = { placeholder?: string disabled?: boolean help?: string + inputAction?: { + ariaLabel?: string + disabled?: boolean + icon?: ButtonProps['icon'] + loading?: boolean + onClick: () => void + title: string + } wide?: boolean secretVisible?: boolean onToggleSecret?: (visible: boolean) => void } +function ConnectionTestInput({ + action, + children, +}: { + action: NonNullable + children: ReactNode +}) { + return ( +
+ {children} +
+ ) +} + const fieldLabels: Record = { key: '键', label: '标签', @@ -2298,19 +2330,37 @@ function FieldGrid({ return ( ) @@ -4761,7 +4811,21 @@ function ModuleConsole({ config }: { config: ModuleConfig }) { { value: 'anthropic-messages', label: 'Anthropic Messages' }, { value: 'ollama-generate', label: 'Ollama Generate' }, ] }, - { key: 'base_url', label: 'LLM 基础地址', wide: true }, + { + key: 'base_url', + label: 'LLM 基础地址', + wide: true, + inputAction: activeGroup ? { + title: '测试 AI Provider 连通性', + icon: 'test', + loading: actionLoading, + onClick: () => void testConnection( + 'AI Provider', + '/settings/integrations/ai-provider/connect', + sanitizeSecretDrafts(stripInternalFields(record), activeGroup.record, ['api_key', 'service_token'], revealedFor('ai_provider')), + ), + } : undefined, + }, { key: 'model', label: '默认模型' }, { key: 'api_key', @@ -4785,28 +4849,47 @@ function ModuleConsole({ config }: { config: ModuleConfig }) { onToggleSecret: toggleActiveSecret('service_token', 'ai_provider'), }, ] + const webSearchDisabled = activeGroup?.key.startsWith('web_search:') && !Boolean(record.enabled) const webFields: FieldConfig[] = [ - { key: 'provider', label: '搜索供应商' }, - { key: 'base_url', label: 'API 基础地址', wide: true }, + { key: 'enabled', label: '启用 WebSearch', type: 'boolean' }, + { key: 'provider', label: '搜索供应商', disabled: webSearchDisabled }, + { + key: 'base_url', + label: 'API 基础地址', + wide: true, + disabled: webSearchDisabled, + inputAction: activeGroup ? { + title: '测试 Web Search 连通性', + icon: 'test', + loading: actionLoading, + disabled: webSearchDisabled, + onClick: () => void testConnection( + 'Web Search', + '/settings/integrations/web-search/connect', + sanitizeSecretDrafts(stripInternalFields(record), activeGroup.record, ['api_key'], revealedFor('web_search')), + ), + } : undefined, + }, { key: 'api_key', label: 'WebSearch API Key', type: 'secret', placeholder: '输入新的 WebSearch API Key', + disabled: webSearchDisabled, secretVisible: Boolean(visibleSecretFields[secretFieldKey('api_key')]), onToggleSecret: toggleActiveSecret('api_key', 'web_search'), }, - { key: 'max_results', label: '最大结果数', type: 'number' }, - { key: 'timeout_seconds', label: '超时(秒)', type: 'number' }, - { key: 'endpoint_path', label: '接口路径' }, - { key: 'search_depth', label: '搜索深度' }, - { key: 'engine', label: 'SerpAPI 引擎' }, - { key: 'categories', label: 'SearXNG 分类' }, - { key: 'search_path', label: 'Firecrawl 搜索路径' }, - { key: 'scrape_path', label: 'Firecrawl 抓取路径' }, - { key: 'include_answer', label: '包含答案', type: 'boolean' }, - { key: 'include_raw_content', label: '包含原始内容', type: 'boolean' }, - { key: 'include_text', label: '包含正文', type: 'boolean' }, + { key: 'max_results', label: '最大结果数', type: 'number', disabled: webSearchDisabled }, + { key: 'timeout_seconds', label: '超时(秒)', type: 'number', disabled: webSearchDisabled }, + { key: 'endpoint_path', label: '接口路径', disabled: webSearchDisabled }, + { key: 'search_depth', label: '搜索深度', disabled: webSearchDisabled }, + { key: 'engine', label: 'SerpAPI 引擎', disabled: webSearchDisabled }, + { key: 'categories', label: 'SearXNG 分类', disabled: webSearchDisabled }, + { key: 'search_path', label: 'Firecrawl 搜索路径', disabled: webSearchDisabled }, + { key: 'scrape_path', label: 'Firecrawl 抓取路径', disabled: webSearchDisabled }, + { key: 'include_answer', label: '包含答案', type: 'boolean', disabled: webSearchDisabled }, + { key: 'include_raw_content', label: '包含原始内容', type: 'boolean', disabled: webSearchDisabled }, + { key: 'include_text', label: '包含正文', type: 'boolean', disabled: webSearchDisabled }, ] const ocrFields: FieldConfig[] = [ { key: 'provider', label: 'OCR 供应商', type: 'select', options: [ @@ -4959,14 +5042,12 @@ function ModuleConsole({ config }: { config: ModuleConfig }) { <> - - ) } - if (config === configs.ai && endpointKey === 'integrations') { - const integrationKey = pick(selected, ['__title', 'key', 'provider'], '') - if (integrationKey === 'ai_provider') { - actions.push( - diff --git a/frontend/src/pages/Docs/Docs.css b/frontend/src/pages/Docs/Docs.css index 6a293aeb..2084684d 100644 --- a/frontend/src/pages/Docs/Docs.css +++ b/frontend/src/pages/Docs/Docs.css @@ -148,7 +148,7 @@ /* ─── Page shell ─────────────────────────────────────────────────────────────── */ .docs-page { - height: 100vh; + height: 100%; display: grid; grid-template-columns: 280px minmax(0, 1fr); background: var(--d-bg); @@ -887,7 +887,7 @@ font-size: 12px; font-weight: 700; text-transform: uppercase; - letter-spacing: 0.04em; + letter-spacing: 0; } .docs-toc__nav { diff --git a/frontend/src/pages/Docs/Docs.tsx b/frontend/src/pages/Docs/Docs.tsx index 1ebb6268..7d149bc8 100644 --- a/frontend/src/pages/Docs/Docs.tsx +++ b/frontend/src/pages/Docs/Docs.tsx @@ -100,8 +100,8 @@ export default function Docs() { const themeOptions = useMemo(() => [ { value: 'light' as const, - label: '浅色', - title: '浅色', + label: lang === 'zh' ? '浅色' : 'Light', + title: lang === 'zh' ? '浅色' : 'Light', icon: ( @@ -111,8 +111,8 @@ export default function Docs() { }, { value: 'system' as const, - label: '系统', - title: '跟随系统', + label: lang === 'zh' ? '系统' : 'System', + title: lang === 'zh' ? '跟随系统' : 'Follow system', icon: ( @@ -122,15 +122,15 @@ export default function Docs() { }, { value: 'dark' as const, - label: '深色', - title: '深色', + label: lang === 'zh' ? '深色' : 'Dark', + title: lang === 'zh' ? '深色' : 'Dark', icon: ( ), }, - ], []) + ], [lang]) const handleLangChange = useCallback((newLang: DocsLang) => { setLang(newLang) diff --git a/frontend/src/pages/Earth/Earth.css b/frontend/src/pages/Earth/Earth.css new file mode 100644 index 00000000..45be4b74 --- /dev/null +++ b/frontend/src/pages/Earth/Earth.css @@ -0,0 +1,6 @@ +.earth-page-frame { + width: 100%; + height: 100%; + border: 0; + display: block; +} diff --git a/frontend/src/pages/Earth/Earth.tsx b/frontend/src/pages/Earth/Earth.tsx index c2ab36d7..6f9e230e 100644 --- a/frontend/src/pages/Earth/Earth.tsx +++ b/frontend/src/pages/Earth/Earth.tsx @@ -1,16 +1,13 @@ +import './Earth.css' + function Earth() { return ( - ); + ) } -export default Earth; +export default Earth diff --git a/frontend/src/pages/ForgotPassword/ForgotPassword.tsx b/frontend/src/pages/ForgotPassword/ForgotPassword.tsx index 403cd63c..5d2b9a53 100644 --- a/frontend/src/pages/ForgotPassword/ForgotPassword.tsx +++ b/frontend/src/pages/ForgotPassword/ForgotPassword.tsx @@ -2,7 +2,7 @@ import axios from 'axios' import { useEffect, useState, type FormEvent } from 'react' import { AuthButton, AuthField, AuthForm, AuthInput, AuthLinks, AuthNotice, AuthShell, BackToLogin } from '../Auth/AuthShell' -const API_URL = (import.meta as any).env?.VITE_API_URL || '/api/v1' +const API_URL = import.meta.env.VITE_API_URL || '/api/v1' interface ErrorBody { response?: { diff --git a/frontend/src/pages/Register/Register.tsx b/frontend/src/pages/Register/Register.tsx index 0f8745eb..fe5544e8 100644 --- a/frontend/src/pages/Register/Register.tsx +++ b/frontend/src/pages/Register/Register.tsx @@ -4,7 +4,7 @@ import { Link, useNavigate } from 'react-router-dom' import { AuthButton, AuthField, AuthForm, AuthInput, AuthLinks, AuthNotice, AuthShell } from '../Auth/AuthShell' import { useAuthStore } from '../../stores/auth' -const API_URL = (import.meta as any).env?.VITE_API_URL || '/api/v1' +const API_URL = import.meta.env.VITE_API_URL || '/api/v1' const RESEND_COOLDOWN_SECONDS = 60 interface ErrorBody { diff --git a/frontend/src/pages/VerifyEmail/VerifyEmail.tsx b/frontend/src/pages/VerifyEmail/VerifyEmail.tsx index b43b867a..e8004f29 100644 --- a/frontend/src/pages/VerifyEmail/VerifyEmail.tsx +++ b/frontend/src/pages/VerifyEmail/VerifyEmail.tsx @@ -4,7 +4,7 @@ import { useNavigate, useSearchParams } from 'react-router-dom' import { AuthButton, AuthField, AuthForm, AuthInput, AuthLinks, AuthNotice, AuthShell, BackToLogin } from '../Auth/AuthShell' import { useAuthStore } from '../../stores/auth' -const API_URL = (import.meta as any).env?.VITE_API_URL || '/api/v1' +const API_URL = import.meta.env.VITE_API_URL || '/api/v1' interface ErrorBody { response?: { diff --git a/frontend/src/services/situational-awareness/http-gateway.ts b/frontend/src/services/situational-awareness/http-gateway.ts index 17f4a8cc..c2d0f187 100644 --- a/frontend/src/services/situational-awareness/http-gateway.ts +++ b/frontend/src/services/situational-awareness/http-gateway.ts @@ -16,7 +16,7 @@ import type { Summary, } from './types' -const API_BASE_URL = (import.meta as any).env?.VITE_API_URL || '/api/v1' +const API_BASE_URL = import.meta.env.VITE_API_URL || '/api/v1' export class HttpSituationalAwarenessGateway implements SituationalAwarenessGateway { private bgpSummaryPromise: Promise | null = null diff --git a/frontend/src/stores/auth.ts b/frontend/src/stores/auth.ts index 1a10fa9d..a614c52c 100644 --- a/frontend/src/stores/auth.ts +++ b/frontend/src/stores/auth.ts @@ -17,7 +17,7 @@ interface AuthState { clearAuth: () => void } -const API_URL = (import.meta as any).env?.VITE_API_URL || '/api/v1' +const API_URL = import.meta.env.VITE_API_URL || '/api/v1' export const useAuthStore = create()( persist( diff --git a/frontend/src/vite-env.d.ts b/frontend/src/vite-env.d.ts index 11f02fe2..db3dd381 100644 --- a/frontend/src/vite-env.d.ts +++ b/frontend/src/vite-env.d.ts @@ -1 +1,10 @@ /// + +interface ImportMetaEnv { + readonly VITE_API_URL?: string + readonly VITE_WS_URL?: string +} + +interface ImportMeta { + readonly env: ImportMetaEnv +} diff --git a/project_context.md b/project_context.md index df8cf6de..c5ab5da7 100644 --- a/project_context.md +++ b/project_context.md @@ -25,39 +25,43 @@ ## Technology Stack +Current implementation truth lives in `README.md`, `CODEMAP.md`, and +`docs/technical/{zh,en}/`. This file is background context; when it conflicts +with current code or harness checks, prefer the current implementation docs. + ### Backend - **Framework:** Python FastAPI 0.109+ - **ORM:** SQLAlchemy 2.0+ -- **Database Migration:** Alembic -- **Task Queue:** Celery 5.3+ -- **Message Queue:** Kafka 3.0+ -- **Caching:** Redis 7.0+ +- **Database Migration:** Alembic / SQLAlchemy migrations +- **Task Runtime:** APScheduler, backend background tasks, and database job state +- **Runtime Coordination:** Redis 7.0+ cache/streams +- **Package Manager / Runner:** uv ### Frontend - **Framework:** React 18 -- **UI Library:** Ant Design Pro +- **UI Layer:** Tactile UI, Radix primitives, lucide-react - **HTTP Client:** Axios -- **State Management:** React Query +- **State Management:** Zustand and local React state - **Real-time:** Socket.io-client - **Charts:** ECharts - **Package Manager / Runner:** Bun 1 +- **3D Earth:** Three.js Web Earth -### Visualization (UE5) -- **Engine:** Unreal Engine 5.3+ -- **Geospatial:** Cesium for Unreal 1.5+ -- **Particles:** Niagara -- **3D Rendering:** Nanite + Lumen +### Visualization Direction +- **Current:** Browser Web Earth with Three.js assets under `frontend/public/earth/` +- **Future / optional:** UE5, Cesium for Unreal, and Niagara remain physical + big-screen exploration directions, not required local development dependencies. ### Database - **Relational:** PostgreSQL 15+ (users, config) -- **Time-series:** TimescaleDB -- **Cache:** Redis 7+ (sessions, cache) -- **Storage:** MinIO (S3-compatible) +- **Cache / Streams:** Redis 7+ (sessions, cache, runtime events) +- **Future / optional:** TimescaleDB, MinIO, Kafka, and Celery are evolution + boundaries only; do not assume they exist in the active local stack. ### Deployment - **Container:** Docker 24+ - **Orchestration:** Docker Compose -- **Reverse Proxy:** Nginx +- **Packaging:** Helm chart under `deploy/helm/planet/` --- @@ -71,7 +75,7 @@ │ │ │ │ ├── auth.py │ │ │ │ ├── users.py │ │ │ │ ├── datasources.py -│ │ │ │ ├── tasks.py +│ │ │ │ ├── docs.py │ │ │ │ └── websocket.py │ │ │ └── endpoints/ │ │ ├── core/ @@ -93,50 +97,43 @@ │ │ │ │ └── huggingface.py │ │ │ └── analysis.py │ │ ├── tasks/ -│ │ │ └── scheduler.py +│ │ ├── services/ +│ │ │ ├── collectors/ +│ │ │ └── docs_gatekeeper.py │ │ └── db/ -│ │ └── session.py +│ │ └── database.py │ ├── tests/ -│ │ ├── api/ -│ │ ├── unit/ -│ │ └── conftest.py -│ ├── pyproject.toml -│ ├── uv.lock +│ │ ├── test_api.py +│ │ └── test_realtime_sources.py │ └── alembic/ │ -├── frontend/ # React Admin +├── frontend/ # React Admin, Docs UI, and Web Earth shell │ ├── src/ -│ │ ├── api/ # Axios instances -│ │ ├── components/ # Reusable components -│ │ ├── pages/ # Route pages +│ │ ├── admin/ # Console routes, pages, Tactile/Radix wrappers +│ │ ├── components/ # Reusable UI and Web Earth components +│ │ ├── pages/ # Public route pages │ │ │ ├── Login/ -│ │ │ ├── Dashboard/ -│ │ │ ├── Users/ -│ │ │ ├── DataSources/ -│ │ │ └── Settings/ -│ │ ├── stores/ # Zustand/Redux +│ │ │ ├── Register/ +│ │ │ ├── Docs/ +│ │ │ └── Earth/ +│ │ ├── stores/ # Zustand stores │ │ ├── hooks/ -│ │ ├── types/ │ │ └── App.tsx │ ├── package.json -│ └── tests/ +│ └── public/earth/ # Web Earth static assets │ -├── unreal/ # UE5 Project -│ ├── Content/ -│ │ ├── Maps/ -│ │ ├── Blueprints/ -│ │ └── Materials/ -│ ├── Plugins/ -│ └── Source/ -│ -├── data/ # Static data files +├── aiprovider/ # Model provider/protocol adapter service +├── motion_agent/ # Motion capture protocol service +├── deploy/helm/planet/ # Helm chart ├── docs/ # Documentation -├── scripts/ # Utility scripts +├── scripts/ # Utility and harness scripts ├── docker-compose.yml +├── pyproject.toml +├── uv.lock ├── .env.example ├── rules.md ├── project_context.md -├── agents.md +├── AGENTS.md └── README.md ``` diff --git a/pyproject.toml b/pyproject.toml index 8646c1ed..47879ff7 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "planet" -version = "0.71.1" +version = "0.72.0" description = "智能星球计划 - 态势感知系统" requires-python = ">=3.14" dependencies = [ diff --git a/scripts/harness/backend-rules-check.sh b/scripts/harness/backend-rules-check.sh new file mode 100755 index 00000000..b78993c9 --- /dev/null +++ b/scripts/harness/backend-rules-check.sh @@ -0,0 +1,60 @@ +#!/usr/bin/env bash + +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +source "$ROOT_DIR/scripts/harness/lib.sh" + +main() { + cd "$ROOT_DIR" + local uv_bin + uv_bin="$(harness_require_tool uv)" + + harness_run "$uv_bin" run --frozen --project "$ROOT_DIR" python - <<'PY' +import ast +from pathlib import Path + +root = Path.cwd() +failures: list[str] = [] + + +def fail(message: str) -> None: + failures.append(message) + + +class BackendDebugCallVisitor(ast.NodeVisitor): + def __init__(self, path: Path) -> None: + self.path = path + + def visit_Call(self, node: ast.Call) -> None: + if isinstance(node.func, ast.Name) and node.func.id in {"print", "breakpoint"}: + fail(f"{self.path}:{node.lineno}: backend app code must use structured logging, not {node.func.id}()") + if ( + isinstance(node.func, ast.Attribute) + and node.func.attr == "set_trace" + and isinstance(node.func.value, ast.Name) + and node.func.value.id == "pdb" + ): + fail(f"{self.path}:{node.lineno}: remove pdb.set_trace() from backend app code") + self.generic_visit(node) + + +for path in sorted((root / "backend/app").rglob("*.py")): + rel = path.relative_to(root) + try: + tree = ast.parse(path.read_text(encoding="utf-8"), filename=str(rel)) + except SyntaxError as exc: + fail(f"{rel}:{exc.lineno}: Python syntax error: {exc.msg}") + continue + BackendDebugCallVisitor(rel).visit(tree) + +if failures: + for message in failures: + print(f"fail: {message}") + raise SystemExit(f"backend rules check failed: {len(failures)} failure(s)") + +print("backend rules check passed") +PY +} + +main "$@" diff --git a/scripts/harness/docs-consistency-check.sh b/scripts/harness/docs-consistency-check.sh new file mode 100755 index 00000000..56f2e239 --- /dev/null +++ b/scripts/harness/docs-consistency-check.sh @@ -0,0 +1,645 @@ +#!/usr/bin/env bash + +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +source "$ROOT_DIR/scripts/harness/lib.sh" + +main() { + cd "$ROOT_DIR" + local uv_bin + uv_bin="$(harness_require_tool uv)" + + harness_run "$uv_bin" run --frozen --project "$ROOT_DIR" python - <<'PY' +import ast +import re +from pathlib import Path +from urllib.parse import parse_qs, urlsplit + +root = Path.cwd() +failures: list[str] = [] +warnings: list[str] = [] + + +def fail(message: str) -> None: + failures.append(message) + + +def warn(message: str) -> None: + warnings.append(message) + + +def read_text(path: str) -> str: + return (root / path).read_text(encoding="utf-8") + + +def literal_assignment(path: str, name: str) -> object: + tree = ast.parse(read_text(path)) + for node in tree.body: + if not isinstance(node, ast.Assign): + continue + if not any(isinstance(target, ast.Name) and target.id == name for target in node.targets): + continue + return ast.literal_eval(node.value) + raise ValueError(f"could not find literal assignment {name} in {path}") + + +def slug_from_filename(filename: str) -> str: + return "overview" if filename == "README.md" else filename.removesuffix(".md") + + +def frontend_docs_metadata() -> dict[str, dict[str, object]]: + text = read_text("frontend/src/pages/Docs/docs-content.ts") + entries: dict[str, dict[str, object]] = {} + pattern = re.compile( + r"(?:\[(?PDOCS_README_FILENAME)\]|'(?P[^']+\.md)'):\s*\{\s*" + r"zh:\s*\{\s*title:\s*'(?P[^']+)',\s*group:\s*'(?P[^']+)',\s*order:\s*(?P\d+)\s*\},\s*" + r"en:\s*\{\s*title:\s*'(?P[^']+)',\s*group:\s*'(?P[^']+)',\s*order:\s*(?P\d+)\s*\}", + re.S, + ) + for match in pattern.finditer(text): + filename = "README.md" if match.group("constant") else match.group("filename") + if filename in entries: + fail(f"frontend Docs metadata registers {filename} more than once") + continue + entries[filename] = { + "slug": slug_from_filename(filename), + "zh_title": match.group("zh_title"), + "en_title": match.group("en_title"), + "zh_group": match.group("zh_group"), + "en_group": match.group("en_group"), + "zh_order": int(match.group("zh_order")), + "en_order": int(match.group("en_order")), + } + return entries + + +def backend_docs_metadata() -> dict[str, dict[str, object]]: + tree = ast.parse(read_text("backend/app/services/docs_gatekeeper.py")) + constants: dict[str, object] = {} + + for node in tree.body: + if not isinstance(node, ast.Assign) or len(node.targets) != 1: + continue + target = node.targets[0] + if isinstance(target, ast.Name) and isinstance(node.value, ast.Constant): + constants[target.id] = node.value.value + + def read_arg(node: ast.AST) -> object: + if isinstance(node, ast.Constant): + return node.value + if isinstance(node, ast.Name) and node.id in constants: + return constants[node.id] + raise ValueError(f"unsupported DocsMetadata argument: {ast.dump(node)}") + + entries: dict[str, dict[str, object]] = {} + for node in ast.walk(tree): + if not isinstance(node, ast.Call): + continue + if not isinstance(node.func, ast.Name) or node.func.id != "DocsMetadata": + continue + if len(node.args) != 7: + fail("backend DocsMetadata entries must keep the expected seven positional fields") + continue + filename, slug, access, group, order, zh_title, en_title = [read_arg(arg) for arg in node.args] + if not isinstance(filename, str): + fail("backend DocsMetadata filename must be a string") + continue + if filename in entries: + fail(f"backend Docs metadata registers {filename} more than once") + continue + entries[filename] = { + "slug": slug, + "access": access, + "group": group, + "order": order, + "zh_title": zh_title, + "en_title": en_title, + } + return entries + + +def docs_metadata_filenames() -> set[str]: + return set(frontend_docs_metadata()) + + +def check_docs_metadata_alignment() -> None: + frontend = frontend_docs_metadata() + backend = backend_docs_metadata() + + if not frontend: + fail("frontend Docs metadata must expose at least one document") + if not backend: + fail("backend Gatekeeper Docs metadata must expose at least one document") + + frontend_only = sorted(set(frontend) - set(backend)) + backend_only = sorted(set(backend) - set(frontend)) + if frontend_only: + fail("frontend Docs metadata has files missing from backend Gatekeeper metadata: " + ", ".join(frontend_only)) + if backend_only: + fail("backend Gatekeeper metadata has files missing from frontend Docs metadata: " + ", ".join(backend_only)) + + for filename in sorted(set(frontend) & set(backend)): + frontend_entry = frontend[filename] + backend_entry = backend[filename] + if frontend_entry["zh_group"] != frontend_entry["en_group"]: + fail(f"frontend Docs metadata group differs by language for {filename}") + if frontend_entry["zh_order"] != frontend_entry["en_order"]: + fail(f"frontend Docs metadata order differs by language for {filename}") + + expected = { + "slug": frontend_entry["slug"], + "group": frontend_entry["zh_group"], + "order": frontend_entry["zh_order"], + "zh_title": frontend_entry["zh_title"], + "en_title": frontend_entry["en_title"], + } + actual = {key: backend_entry[key] for key in expected} + if actual != expected: + fail(f"{filename}: frontend Docs metadata and backend Gatekeeper metadata differ; frontend={expected}, backend={actual}") + + +def check_public_docs_registry() -> None: + filenames = docs_metadata_filenames() + for filename in sorted(filenames): + for lang in ("zh", "en"): + path = root / "docs/technical" / lang / filename + if not path.exists(): + fail(f"public docs metadata references missing file: {path.relative_to(root)}") + + for readme in (root / "docs/technical/zh/README.md", root / "docs/technical/en/README.md"): + if not readme.exists(): + continue + for href in re.findall(r"\]\(([^)]+\.md)\)", readme.read_text(encoding="utf-8")): + if "docs/technical/" not in href: + continue + filename = Path(href).name + if filename not in filenames: + fail(f"{readme.relative_to(root)} links {filename}, but DOCS_METADATA does not expose it") + + +def check_all_technical_docs_have_bilingual_pairs() -> None: + zh_dir = root / "docs/technical/zh" + en_dir = root / "docs/technical/en" + zh_files = {path.name for path in zh_dir.glob("*.md")} + en_files = {path.name for path in en_dir.glob("*.md")} + + zh_only = sorted(zh_files - en_files) + en_only = sorted(en_files - zh_files) + if zh_only: + fail("technical docs missing English counterparts: " + ", ".join(zh_only)) + if en_only: + fail("technical docs missing Chinese counterparts: " + ", ".join(en_only)) + + +def check_bilingual_docs_are_not_copies() -> None: + same: list[str] = [] + for en in sorted((root / "docs/technical/en").glob("*.md")): + zh = root / "docs/technical/zh" / en.name + if zh.exists() and en.read_text(encoding="utf-8") == zh.read_text(encoding="utf-8"): + same.append(en.name) + if same: + fail("identical en/zh technical docs: " + ", ".join(same)) + + +def check_public_doc_links_exist() -> None: + repo_prefix = f"{root}/" + for doc in sorted((root / "docs/technical").glob("*/*.md")): + text = doc.read_text(encoding="utf-8") + for href in re.findall(r"\]\(([^)#]+\.md)(?:#[^)]+)?\)", text): + if not href.startswith(repo_prefix): + continue + target = Path(href) + if not target.exists(): + fail(f"{doc.relative_to(root)} links missing markdown file: {href}") + + +def check_language_scoped_technical_links() -> None: + technical_docs_prefix = re.escape(str(root / "docs/technical")) + pattern = re.compile(rf"{technical_docs_prefix}/(?!zh/|en/)[^)#\s]+") + for doc in sorted((root / "docs/technical").glob("*/*.md")): + text = doc.read_text(encoding="utf-8") + for match in pattern.finditer(text): + line_no = text.count("\n", 0, match.start()) + 1 + fail( + f"{doc.relative_to(root)}:{line_no}: technical doc link must include " + f"the language directory: {match.group(0)}" + ) + + +def check_public_doc_link_titles() -> None: + pattern = re.compile(r"\[([^]\n]+\.md)\]\(") + for doc in sorted((root / "docs/technical").glob("*/*.md")): + text = doc.read_text(encoding="utf-8") + for match in pattern.finditer(text): + line_no = text.count("\n", 0, match.start()) + 1 + fail( + f"{doc.relative_to(root)}:{line_no}: public docs should use readable " + f"link text instead of raw filename {match.group(1)!r}" + ) + + +def check_credential_collector_contracts() -> None: + defaults = literal_assignment("backend/app/core/datasource_defaults.py", "DEFAULT_DATASOURCES") + if not isinstance(defaults, dict): + fail("backend/app/core/datasource_defaults.py DEFAULT_DATASOURCES must stay a dict") + return + + supported_sources: dict[str, str] = {} + for source, info in defaults.items(): + if not isinstance(info, dict): + fail(f"DEFAULT_DATASOURCES entry {source!r} must be a dict") + continue + if not info.get("requires_credentials"): + continue + if info.get("credential_status") != "supported": + continue + provider = info.get("credential_provider") + if not isinstance(provider, str) or not provider.strip(): + fail(f"{source}: supported credential collector is missing credential_provider") + continue + supported_sources[str(source)] = provider + + if not supported_sources: + return + + guides_text = read_text("backend/app/services/credential_guides.py") + default_guides = set(re.findall(r"CredentialGuideDefault\(\s*provider=\"([^\"]+)\"", guides_text)) + connectivity_providers = literal_assignment( + "backend/app/services/datasource_connectivity.py", + "SUPPORTED_CREDENTIAL_PROVIDERS", + ) + if not isinstance(connectivity_providers, set): + fail("SUPPORTED_CREDENTIAL_PROVIDERS must stay a literal set") + connectivity_providers = set() + + frontend_text = read_text("frontend/src/admin/pages/PlainResourcePages.tsx") + tests_text = read_text("backend/tests/test_collectors.py") + zh_doc = read_text("docs/technical/zh/datasource-collector-settings-connectivity.md") + en_doc = read_text("docs/technical/en/datasource-collector-settings-connectivity.md") + + if "test_supported_credential_collectors_have_guides_and_connectivity_provider" not in tests_text: + fail("backend/tests/test_collectors.py must keep the supported credential collector contract test") + + if "loadCredentialGuide" not in frontend_text or "credentialGuideProvider" not in frontend_text: + fail("collector credential UI must keep guide-loading and provider-normalization helpers") + + for source, provider in sorted(supported_sources.items()): + if provider not in default_guides: + fail(f"{source}: missing default credential guide for provider {provider}") + if provider not in connectivity_providers: + fail(f"{source}: missing supported connectivity provider {provider}") + if provider not in frontend_text and source not in frontend_text: + fail(f"{source}: collector credential UI does not mention provider/source {provider}") + for doc_path, doc_text in ( + ("docs/technical/zh/datasource-collector-settings-connectivity.md", zh_doc), + ("docs/technical/en/datasource-collector-settings-connectivity.md", en_doc), + ): + if provider not in doc_text and source not in doc_text: + fail(f"{doc_path}: missing supported credential collector provider/source {provider}/{source}") + + +def known_frontend_routes() -> set[str]: + app_text = read_text("frontend/src/App.tsx") + admin_text = read_text("frontend/src/admin/AdminRoutes.tsx") + manifest_text = read_text("frontend/src/admin/routes/manifest.tsx") + routes = set(re.findall(r' dict[str, str]: + manifest_text = read_text("frontend/src/admin/routes/manifest.tsx") + routes: dict[str, str] = {} + pattern = re.compile(r"\{\s*path:\s*'([^']+)',\s*label:\s*'([^']+)'", re.S) + for path, label in pattern.findall(manifest_text): + routes[path] = label + if not routes: + fail("could not parse admin route manifest for manual coverage checks") + return routes + + +def manual_console_route_rows(doc_path: Path, heading: str) -> dict[str, str]: + text = doc_path.read_text(encoding="utf-8") + heading_index = text.find(heading) + if heading_index == -1: + fail(f"{doc_path.relative_to(root)} is missing {heading!r}") + return {} + next_heading = re.search(r"\n##\s+", text[heading_index + len(heading):]) + section = text[heading_index:] if not next_heading else text[heading_index:heading_index + len(heading) + next_heading.start()] + rows: dict[str, str] = {} + for line in section.splitlines(): + match = re.match(r"\|\s*([^|`][^|]*?)\s*\|\s*`([^`]+)`\s*\|", line) + if not match: + continue + label = match.group(1).strip() + path = match.group(2).strip() + rows[path] = label + return rows + + +def normalize_zh_label(label: str) -> str: + return re.sub(r"\s+", "", label) + + +def check_manual_console_route_tables() -> None: + manifest = admin_manifest_routes() + zh_manual = root / "docs/technical/zh/manual.md" + en_manual = root / "docs/technical/en/manual.md" + manual_rows = { + "zh": manual_console_route_rows(zh_manual, "## 控制台总览"), + "en": manual_console_route_rows(en_manual, "## Console Overview"), + } + + for lang, rows in manual_rows.items(): + missing = sorted(set(manifest) - set(rows)) + extra = sorted(set(rows) - set(manifest)) + if missing: + fail(f"docs/technical/{lang}/manual.md console overview misses admin manifest route(s): " + ", ".join(missing)) + if extra: + fail(f"docs/technical/{lang}/manual.md console overview lists route(s) missing from admin manifest: " + ", ".join(extra)) + + zh_rows = manual_rows["zh"] + for path, expected_label in manifest.items(): + actual = zh_rows.get(path) + if actual is None: + continue + if normalize_zh_label(actual) != normalize_zh_label(expected_label): + fail( + "docs/technical/zh/manual.md console overview label mismatch for " + f"{path}: expected {expected_label!r}, got {actual!r}" + ) + + +def extract_balanced(text: str, start: int, open_char: str, close_char: str) -> str: + depth = 0 + quote: str | None = None + escape = False + for index in range(start, len(text)): + char = text[index] + if quote: + if escape: + escape = False + elif char == "\\": + escape = True + elif char == quote: + quote = None + continue + if char in ("'", '"', "`"): + quote = char + continue + if char == open_char: + depth += 1 + elif char == close_char: + depth -= 1 + if depth == 0: + return text[start:index + 1] + raise ValueError(f"could not find balanced {open_char}{close_char} block") + + +def find_balanced_after(text: str, marker: str, open_char: str, close_char: str) -> str: + marker_index = text.index(marker) + start = text.index(open_char, marker_index) + return extract_balanced(text, start, open_char, close_char) + + +def nesting_depth(text: str, stop: int, open_char: str, close_char: str) -> int: + depth = 0 + quote: str | None = None + escape = False + for char in text[:stop]: + if quote: + if escape: + escape = False + elif char == "\\": + escape = True + elif char == quote: + quote = None + continue + if char in ("'", '"', "`"): + quote = char + continue + if char == open_char: + depth += 1 + elif char == close_char: + depth -= 1 + return depth + + +def top_level_config_blocks(configs_block: str) -> dict[str, str]: + blocks: dict[str, str] = {} + index = 1 + while index < len(configs_block) - 1: + match = re.search(r"\b([A-Za-z][A-Za-z0-9_]*)\s*:\s*\{", configs_block[index:]) + if not match: + break + name = match.group(1) + start = index + match.end() - 1 + if nesting_depth(configs_block, start, "{", "}") != 1: + index = start + 1 + continue + block = extract_balanced(configs_block, start, "{", "}") + blocks[name] = block + index = start + len(block) + return blocks + + +def direct_section_keys(config_block: str) -> set[str]: + sections_match = re.search(r"\bsections\s*:\s*\[", config_block) + if not sections_match: + return set() + sections_block = extract_balanced(config_block, sections_match.end() - 1, "[", "]") + keys: set[str] = set() + index = 1 + while index < len(sections_block) - 1: + match = re.search(r"\{\s*key\s*:\s*'([^']+)'", sections_block[index:]) + if not match: + break + start = index + match.start() + if ( + nesting_depth(sections_block, start, "[", "]") == 1 + and nesting_depth(sections_block, start, "{", "}") == 0 + ): + keys.add(match.group(1)) + block = extract_balanced(sections_block, start, "{", "}") + index = start + len(block) + else: + index = start + 1 + return keys + + +def known_admin_sections() -> dict[str, set[str]]: + plain_text = read_text("frontend/src/admin/pages/PlainResourcePages.tsx") + admin_text = read_text("frontend/src/admin/AdminRoutes.tsx") + + configs_block = find_balanced_after(plain_text, "const configs =", "{", "}") + configs = top_level_config_blocks(configs_block) + component_to_config = dict(re.findall( + r"export function (\w+)\(\) \{\s*return \s*\}", + plain_text, + )) + path_to_component = dict(re.findall( + r'\}', + admin_text, + )) + + section_map: dict[str, set[str]] = {} + for path, component in path_to_component.items(): + config_key = component_to_config.get(component) + if not config_key: + continue + if config_key not in configs: + fail(f"Admin route {path} uses unknown PlainResourcePages config: {config_key}") + continue + section_map[path] = direct_section_keys(configs[config_key]) + return section_map + + +def check_documented_section_deep_links() -> None: + section_map = known_admin_sections() + pattern = re.compile(r"/[a-z][a-z0-9/-]*\?section=[a-z0-9_/-]+") + docs = [ + *sorted((root / "docs/technical").glob("*/*.md")), + *sorted((root / "docs/plans").glob("*.md")), + ] + for doc in docs: + text = doc.read_text(encoding="utf-8") + for match in pattern.finditer(text): + value = match.group(0) + parsed = urlsplit(value) + section = parse_qs(parsed.query).get("section", [""])[0] + if parsed.path not in section_map: + line_no = text.count("\n", 0, match.start()) + 1 + fail(f"{doc.relative_to(root)}:{line_no}: documented section link uses a route with no known sections: {value}") + continue + if section not in section_map[parsed.path]: + line_no = text.count("\n", 0, match.start()) + 1 + known = ", ".join(sorted(section_map[parsed.path])) + fail(f"{doc.relative_to(root)}:{line_no}: documented section link {value} is not a known section; expected one of: {known}") + + +def check_documented_ui_routes() -> None: + routes = known_frontend_routes() + allowed_prefixes = ( + "/api/", + "/ws", + "/health", + "/legacy/", + "/earth/", + "/docs/", + "/assets/", + "/components/", + "/dev/", + "/etc/", + "/home/", + "/tmp/", + "/root", + "/app/", + "/planet", + "/localhost", + "/example", + "/your-", + ) + route_docs = [ + root / "docs/technical/zh/README.md", + root / "docs/technical/en/README.md", + root / "docs/technical/zh/manual.md", + root / "docs/technical/en/manual.md", + root / "docs/technical/zh/quickstart.md", + root / "docs/technical/en/quickstart.md", + root / "docs/technical/zh/frontend-admin-frontend-context.md", + root / "docs/technical/en/frontend-admin-frontend-context.md", + ] + route_pattern = re.compile(r"(? None: + stale_patterns = [ + (re.compile(r"React\s*\+\s*Ant Design"), "console frontend is no longer React + Ant Design"), + (re.compile(r"Ant Design Pro"), "console frontend is no longer Ant Design Pro"), + (re.compile(r"Old AntD legacy pages"), "old AntD legacy pages are not part of the active frontend"), + (re.compile(r"Task Queue:\s*Celery"), "Celery is not part of the active local task stack"), + (re.compile(r"Message Queue:\s*Kafka"), "Kafka is not part of the active local message stack"), + (re.compile(r"UI Library:\s*Ant Design Pro"), "console frontend is no longer Ant Design Pro"), + (re.compile(r"├── unreal/"), "UE5 is not an active checked-in local project tree"), + (re.compile(r"Unreal Engine 5 3D visualization"), "UE5 is future/optional, not the active local visualization shell"), + (re.compile(r"Polarized 3D large display \(4K, 120Hz\)"), "physical display work is future/optional, not the active local loop"), + (re.compile(r"\?tab="), "admin deep links use ?section=, not ?tab="), + ] + docs = [ + root / "AGENTS.md", + root / "README.md", + root / "project_context.md", + *sorted((root / "docs/technical").glob("*/*.md")), + *sorted((root / "docs/plans").glob("*.md")), + ] + for doc in docs: + text = doc.read_text(encoding="utf-8") + for pattern, reason in stale_patterns: + for match in pattern.finditer(text): + line_no = text.count("\n", 0, match.start()) + 1 + fail(f"{doc.relative_to(root)}:{line_no}: stale admin documentation: {reason}") + + +def check_harness_rule_coverage_notes() -> None: + harness_doc = read_text("docs/HARNESS.md") + audit_doc = read_text("docs/harness-audit.md") + if "Rules Coverage Evidence" not in harness_doc: + fail("docs/HARNESS.md must point frontend/docs audits to the Rules Coverage Evidence section") + if "## Rules Coverage Evidence" not in audit_doc: + fail("docs/harness-audit.md must keep the Rules Coverage Evidence section") + + for module in ("core", "security", "workflow", "docs", "uiux", "frontend", "earth"): + if f"| `{module}` |" not in audit_doc: + fail(f"docs/harness-audit.md Rules Coverage Evidence must include the `{module}` rules module") + + +def main() -> None: + check_docs_metadata_alignment() + check_public_docs_registry() + check_all_technical_docs_have_bilingual_pairs() + check_bilingual_docs_are_not_copies() + check_public_doc_links_exist() + check_language_scoped_technical_links() + check_public_doc_link_titles() + check_credential_collector_contracts() + check_manual_console_route_tables() + check_documented_section_deep_links() + check_documented_ui_routes() + check_stale_admin_doc_terms() + check_harness_rule_coverage_notes() + + for message in warnings: + print(f"warn: {message}") + if failures: + for message in failures: + print(f"fail: {message}") + raise SystemExit(f"docs consistency check failed: {len(failures)} failure(s), {len(warnings)} warning(s)") + print(f"docs consistency check passed: {len(warnings)} warning(s)") + + +main() +PY +} + +main "$@" diff --git a/scripts/harness/doctor.sh b/scripts/harness/doctor.sh index 07dd30a8..562dd9d9 100755 --- a/scripts/harness/doctor.sh +++ b/scripts/harness/doctor.sh @@ -3,6 +3,8 @@ set -euo pipefail ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +source "$ROOT_DIR/scripts/harness/lib.sh" + failures=0 warnings=0 @@ -22,8 +24,10 @@ fail() { check_cmd() { local cmd="$1" - if command -v "$cmd" >/dev/null 2>&1; then - ok "found command: $cmd" + local found + if found="$(harness_find_cmd "$cmd")"; then + harness_prepend_tool_dir "$found" + ok "found command: $cmd ($found)" else fail "missing required command: $cmd" fi @@ -32,8 +36,10 @@ check_cmd() { check_optional_cmd() { local cmd="$1" local reason="$2" - if command -v "$cmd" >/dev/null 2>&1; then - ok "found optional command: $cmd" + local found + if found="$(harness_find_cmd "$cmd")"; then + harness_prepend_tool_dir "$found" + ok "found optional command: $cmd ($found)" else warn "missing optional command: $cmd ($reason)" fi @@ -64,7 +70,6 @@ main() { check_file README.md check_file rules.md check_file project_context.md - check_file agents.md check_file AGENTS.md check_file CODEMAP.md check_file docs/HARNESS.md @@ -73,6 +78,11 @@ main() { check_file planet.sh check_file pyproject.toml check_file frontend/package.json + check_file scripts/harness/security-check.sh + check_file scripts/harness/backend-rules-check.sh + check_file scripts/harness/frontend-rules-check.sh + check_file scripts/harness/docs-consistency-check.sh + check_file scripts/harness/frontend-smoke.mjs check_file .gitea/workflows/ci.yaml check_cmd git @@ -86,6 +96,7 @@ main() { check_absent frontend/package-lock.json "frontend package management is Bun-only" check_absent frontend/pnpm-lock.yaml "frontend package management is Bun-only" check_absent frontend/yarn.lock "frontend package management is Bun-only" + check_absent agents.md "AGENTS.md is the single agent guide" if [ ! -x "$ROOT_DIR/planet.sh" ]; then warn "planet.sh is not executable; run it with zsh or restore executable bit" diff --git a/scripts/harness/frontend-rules-check.sh b/scripts/harness/frontend-rules-check.sh new file mode 100755 index 00000000..0705d7ec --- /dev/null +++ b/scripts/harness/frontend-rules-check.sh @@ -0,0 +1,465 @@ +#!/usr/bin/env bash + +set -euo pipefail + +ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)" +source "$ROOT_DIR/scripts/harness/lib.sh" + +main() { + cd "$ROOT_DIR" + local uv_bin + uv_bin="$(harness_require_tool uv)" + + harness_run "$uv_bin" run --frozen --project "$ROOT_DIR" python - <<'PY' +import json +import re +from pathlib import Path +from urllib.parse import urlsplit + +root = Path.cwd() +failures: list[str] = [] +warnings: list[str] = [] + + +def fail(message: str) -> None: + failures.append(message) + + +def warn(message: str) -> None: + warnings.append(message) + + +def read_text(path: str) -> str: + return (root / path).read_text(encoding="utf-8") + + +def check_package_manager() -> None: + for path in ("package.json", "frontend/package.json"): + payload = json.loads(read_text(path)) + package_manager = payload.get("packageManager", "") + if not str(package_manager).startswith("bun@"): + fail(f"{path}: packageManager must stay Bun-only, got {package_manager!r}") + for script_name, command in payload.get("scripts", {}).items(): + if re.search(r"\b(npm|pnpm|yarn)\b", command): + fail(f"{path} script {script_name!r} uses a forbidden package manager: {command}") + + forbidden_lockfiles = [ + "frontend/package-lock.json", + "frontend/pnpm-lock.yaml", + "frontend/yarn.lock", + ] + for path in forbidden_lockfiles: + if (root / path).exists(): + fail(f"{path}: forbidden frontend lockfile; frontend package management is Bun-only") + + +def extract_admin_routes() -> tuple[set[str], set[str]]: + admin_routes_text = read_text("frontend/src/admin/AdminRoutes.tsx") + manifest_text = read_text("frontend/src/admin/routes/manifest.tsx") + route_paths = set(re.findall(r' None: + route_paths, manifest_paths = extract_admin_routes() + public_manifest_paths = {"/earth", "/docs"} + redirect_only_routes = {"/alerts"} + + missing_routes = sorted(manifest_paths - route_paths - public_manifest_paths) + if missing_routes: + fail( + "admin route manifest points to paths that AdminRoutes does not render: " + + ", ".join(missing_routes) + ) + + missing_manifest = sorted(route_paths - manifest_paths - redirect_only_routes) + if missing_manifest: + fail( + "AdminRoutes renders paths that are missing from admin route manifest: " + + ", ".join(missing_manifest) + ) + + +def known_frontend_routes() -> set[str]: + app_text = read_text("frontend/src/App.tsx") + admin_text = read_text("frontend/src/admin/AdminRoutes.tsx") + manifest_text = read_text("frontend/src/admin/routes/manifest.tsx") + routes = set(re.findall(r' None: + routes = known_frontend_routes() + allowed_prefixes = ( + "/api/", + "/ws", + "/assets/", + "/earth/", + ) + literal_link_pattern = re.compile(r"\b(?:to|href)\s*=\s*['\"](/[^'\"#]+(?:#[^'\"]*)?)['\"]") + for path in (root / "frontend/src").rglob("*.tsx"): + text = path.read_text(encoding="utf-8") + rel = path.relative_to(root) + for match in literal_link_pattern.finditer(text): + href = match.group(1) + route_path = urlsplit(href).path + if route_path in routes: + continue + if route_path.startswith("/docs/"): + continue + if any(route_path.startswith(prefix) for prefix in allowed_prefixes): + continue + line_no = jsx_line_number(text, match.start()) + fail(f"{rel}:{line_no}: literal internal link points to an unknown frontend route: {href}") + + +def check_admin_search_route_targets() -> None: + routes = known_frontend_routes() + text = read_text("frontend/src/admin/search/indexers.ts") + for match in re.finditer(r"\broutePath:\s*'([^']+)'", text): + route_path = match.group(1) + if route_path in routes: + continue + line_no = jsx_line_number(text, match.start()) + fail( + "frontend/src/admin/search/indexers.ts:" + f"{line_no}: admin search routePath points to an unknown frontend route: {route_path}" + ) + + +def iter_frontend_src_files() -> list[Path]: + return [ + path + for path in (root / "frontend/src").rglob("*") + if path.is_file() and path.suffix in {".ts", ".tsx", ".js", ".jsx", ".css"} + ] + + +def collect_inline_style_context(lines: list[str], start_index: int) -> str: + block = [] + for line in lines[start_index:start_index + 8]: + block.append(line.strip()) + if "}}" in line or "} as CSSProperties" in line: + break + return " ".join(block) + + +def inline_style_is_dynamic(context: str) -> bool: + dynamic_needles = ( + "--", + "CSSProperties", + "transform:", + "translate", + "scale(", + "width:", + "height:", + "thumbSize", + "thumbOffset", + "trackSize", + "header.getSize", + "summaryWidth", + "detailWidth", + "progress", + "pan.x", + "pan.y", + "zoom", + ) + return any(needle in context for needle in dynamic_needles) + + +def iter_css_blocks(text: str) -> list[tuple[int, str, str]]: + text = re.sub(r"/\*.*?\*/", "", text, flags=re.S) + blocks = [] + start_line = 1 + selector_lines = [] + body_lines = [] + in_block = False + for line_no, line in enumerate(text.splitlines(), 1): + if not in_block: + selector_lines.append(line) + if "{" in line: + in_block = True + start_line = line_no + before, after = line.split("{", 1) + selector_lines[-1] = before + body_lines = [after] + continue + + if "}" in line: + before, _after = line.split("}", 1) + body_lines.append(before) + blocks.append((start_line, " ".join(selector_lines).strip(), "\n".join(body_lines))) + selector_lines = [] + body_lines = [] + in_block = False + else: + body_lines.append(line) + return blocks + + +def overflow_hidden_has_explicit_owner(selector: str, body: str) -> bool: + owner_needles = ( + "min-height: 0", + "height: 100%", + "height: 100vh", + "display: grid", + "display: flex", + "text-overflow: ellipsis", + "border-radius:", + "position: fixed", + "position: absolute", + "scrollbar", + ) + selector_needles = ( + "html", + "body", + "#root", + "tui-scrollbar", + "table-scroll", + "segmented-control__label", + "docs-page", + "docs-shell", + "docs-content-layout", + "docs-toc", + "admin", + "an-page", + "an-resource", + "an-data", + "an-panel", + "an-playground", + "markdown-renderer", + "meta", + "select", + ) + return any(needle in body for needle in owner_needles) or any( + needle in selector for needle in selector_needles + ) + + +def check_debug_output() -> None: + for path in iter_frontend_src_files(): + text = path.read_text(encoding="utf-8") + rel = path.relative_to(root) + for line_no, line in enumerate(text.splitlines(), 1): + if "console.log(" in line or re.search(r"\bdebugger\b", line): + fail(f"{rel}:{line_no}: remove console.log/debugger from frontend source") + if "console." in line and "token" in line.lower(): + fail(f"{rel}:{line_no}: console output must not include token material") + + +def jsx_line_number(text: str, index: int) -> int: + return text.count("\n", 0, index) + 1 + + +def iter_opening_tags(text: str, tag_name: str) -> list[tuple[int, str]]: + tags: list[tuple[int, str]] = [] + needle = f"<{tag_name}" + index = 0 + while True: + start = text.find(needle, index) + if start == -1: + return tags + next_char_index = start + len(needle) + if next_char_index < len(text) and (text[next_char_index].isalnum() or text[next_char_index] in "_-"): + index = next_char_index + continue + + quote: str | None = None + brace_depth = 0 + cursor = next_char_index + while cursor < len(text): + char = text[cursor] + if quote: + if char == "\\": + cursor += 2 + continue + if char == quote: + quote = None + cursor += 1 + continue + if char in ('"', "'", "`"): + quote = char + elif char == "{": + brace_depth += 1 + elif char == "}": + brace_depth = max(0, brace_depth - 1) + elif char == ">" and brace_depth == 0: + tags.append((start, text[start:cursor + 1])) + index = cursor + 1 + break + cursor += 1 + else: + return tags + + +def check_native_button_safety() -> None: + for path in (root / "frontend/src").rglob("*.tsx"): + text = path.read_text(encoding="utf-8") + rel = path.relative_to(root) + for start, tag in iter_opening_tags(text, "button"): + line_no = jsx_line_number(text, start) + if not re.search(r"\btype\s*=", tag): + fail(f"{rel}:{line_no}: native