358 lines
9.2 KiB
Markdown
358 lines
9.2 KiB
Markdown
# project_context.md
|
||
|
||
**AI 需要知道的静态事实。用于理解项目，非约束。**
|
||
|
||
---
|
||
|
||
## Project Overview
|
||
|
||
**智能星球计划 (Intelligent Planet Plan)** - A situational awareness system for data-centric competition.
|
||
|
||
### Core Vision
|
||
|
||
构建人类智能空间的"实时全景图" - 在智能时代，同时监测"现实层宇宙"和"认知层宇宙"。
|
||
|
||
### Four Core Elements
|
||
|
||
| Layer | Element | Description |
|
||
|-------|---------|-------------|
|
||
| L1 | 新兴技术支撑 | AI算力、模型生态、云基础设施 |
|
||
| L2 | 关键基础设施 | 卫星、海底光缆、IXP、路由 |
|
||
| L3 | 组织制度资源 | 规则制定权、顶层设计 |
|
||
| L4 | 文化内容供给 | 新闻、社交视频、舆论情绪 |
|
||
|
||
---
|
||
|
||
## Technology Stack
|
||
|
||
### Backend
|
||
- **Framework:** Python FastAPI 0.109+
|
||
- **ORM:** SQLAlchemy 2.0+
|
||
- **Database Migration:** Alembic
|
||
- **Task Queue:** Celery 5.3+
|
||
- **Message Queue:** Kafka 3.0+
|
||
- **Caching:** Redis 7.0+
|
||
|
||
### Frontend
|
||
- **Framework:** React 18
|
||
- **UI Library:** Ant Design Pro
|
||
- **HTTP Client:** Axios
|
||
- **State Management:** React Query
|
||
- **Real-time:** Socket.io-client
|
||
- **Charts:** ECharts
|
||
|
||
### Visualization (UE5)
|
||
- **Engine:** Unreal Engine 5.3+
|
||
- **Geospatial:** Cesium for Unreal 1.5+
|
||
- **Particles:** Niagara
|
||
- **3D Rendering:** Nanite + Lumen
|
||
|
||
### Database
|
||
- **Relational:** PostgreSQL 15+ (users, config)
|
||
- **Time-series:** TimescaleDB
|
||
- **Cache:** Redis 7+ (sessions, cache)
|
||
- **Storage:** MinIO (S3-compatible)
|
||
|
||
### Deployment
|
||
- **Container:** Docker 24+
|
||
- **Orchestration:** Docker Compose
|
||
- **Reverse Proxy:** Nginx
|
||
|
||
---
|
||
|
||
## Project Structure
|
||
|
||
```
|
||
├── backend/ # FastAPI Backend
|
||
│ ├── app/
|
||
│ │ ├── api/
|
||
│ │ │ ├── v1/
|
||
│ │ │ │ ├── auth.py
|
||
│ │ │ │ ├── users.py
|
||
│ │ │ │ ├── datasources.py
|
||
│ │ │ │ ├── tasks.py
|
||
│ │ │ │ └── websocket.py
|
||
│ │ │ └── endpoints/
|
||
│ │ ├── core/
|
||
│ │ │ ├── config.py
|
||
│ │ │ ├── security.py
|
||
│ │ │ └── websocket_manager.py
|
||
│ │ ├── models/
|
||
│ │ │ ├── user.py
|
||
│ │ │ ├── datasource.py
|
||
│ │ │ └── task.py
|
||
│ │ ├── schemas/
|
||
│ │ │ ├── user.py
|
||
│ │ │ └── datasource.py
|
||
│ │ ├── services/
|
||
│ │ │ ├── collectors/
|
||
│ │ │ │ ├── base.py
|
||
│ │ │ │ ├── top500.py
|
||
│ │ │ │ ├── epoch_ai.py
|
||
│ │ │ │ └── huggingface.py
|
||
│ │ │ └── analysis.py
|
||
│ │ ├── tasks/
|
||
│ │ │ └── scheduler.py
|
||
│ │ └── db/
|
||
│ │ └── session.py
|
||
│ ├── tests/
|
||
│ │ ├── api/
|
||
│ │ ├── unit/
|
||
│ │ └── conftest.py
|
||
│ ├── requirements.txt
|
||
│ └── alembic/
|
||
│
|
||
├── frontend/ # React Admin
|
||
│ ├── src/
|
||
│ │ ├── api/ # Axios instances
|
||
│ │ ├── components/ # Reusable components
|
||
│ │ ├── pages/ # Route pages
|
||
│ │ │ ├── Login/
|
||
│ │ │ ├── Dashboard/
|
||
│ │ │ ├── Users/
|
||
│ │ │ ├── DataSources/
|
||
│ │ │ └── Settings/
|
||
│ │ ├── stores/ # Zustand/Redux
|
||
│ │ ├── hooks/
|
||
│ │ ├── types/
|
||
│ │ └── App.tsx
|
||
│ ├── package.json
|
||
│ └── tests/
|
||
│
|
||
├── unreal/ # UE5 Project
|
||
│ ├── Content/
|
||
│ │ ├── Maps/
|
||
│ │ ├── Blueprints/
|
||
│ │ └── Materials/
|
||
│ ├── Plugins/
|
||
│ └── Source/
|
||
│
|
||
├── data/ # Static data files
|
||
├── docs/ # Documentation
|
||
├── scripts/ # Utility scripts
|
||
├── docker-compose.yml
|
||
├── .env.example
|
||
├── rules.md
|
||
├── project_context.md
|
||
├── agents.md
|
||
└── README.md
|
||
```
|
||
|
||
---
|
||
|
||
## Role-Based Access Control
|
||
|
||
| Role | Permissions | Description |
|
||
|------|-------------|-------------|
|
||
| **super_admin** | Full access | System init, **不可删除** |
|
||
| **admin** | All except user management | Daily operations |
|
||
| **operator** | View + manual operations | Manual refresh, alert confirm |
|
||
| **viewer** | Read-only | View dashboard only |
|
||
|
||
### Default Users (MVP)
|
||
- `admin` / `admin123` (super_admin)
|
||
- `operator` / `operator123` (operator)
|
||
- `viewer` / `viewer123` (viewer)
|
||
|
||
---
|
||
|
||
## Data Sources
|
||
|
||
| Priority | Source | Frequency | Module | Description |
|
||
|----------|--------|-----------|--------|-------------|
|
||
| P0 | TOP500 | 4 hours | L1 | Supercomputer rankings |
|
||
| P0 | Epoch AI | 1 hour | L1 | GPU cluster data |
|
||
| P0 | Hugging Face | 2 hours | L1 | Model ecosystem |
|
||
| P0 | GitHub | 4 hours | L1 | Code contributions |
|
||
| P0 | TeleGeography | Daily | L2 | Submarine cables, cloud infra |
|
||
| P0 | PeeringDB | 2 hours | L2 | IXP data |
|
||
| P1 | OECD.AI | 6 hours | L1 | Policy/investment |
|
||
| P1 | Cloudflare Radar | 1 hour | L2 | Traffic/blocking |
|
||
| P1 | CAIDA BGPStream | 15 minutes | L2 | Route monitoring |
|
||
|
||
---
|
||
|
||
## 3D Display Specifications
|
||
|
||
- **Display Type:** Polarized 3D (glasses-style)
|
||
- **Screen Size:** 2m × 3m
|
||
- **Resolution:** 4K (3840×2160)
|
||
- **Refresh Rate:** 120Hz
|
||
- **Engine:** Unreal Engine 5.3+
|
||
- **Plugin:** Cesium for Unreal
|
||
|
||
### Communication
|
||
- WebSocket for real-time data
|
||
- 30-second heartbeat
|
||
- Data frame updates every 5 minutes (configurable)
|
||
|
||
---
|
||
|
||
## Build Commands Reference
|
||
|
||
### Backend
|
||
```bash
|
||
cd backend
|
||
pip install -r requirements.txt
|
||
python -m uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||
docker-compose up -d backend
|
||
ruff check . && black --check .
|
||
pytest -v && pytest tests/api/test_auth.py::test_login -v
|
||
```
|
||
|
||
### Frontend
|
||
```bash
|
||
cd frontend
|
||
npm install
|
||
npm run dev
|
||
npm run build
|
||
npm run lint && npm run lint:fix
|
||
npm test -- --testPathPattern="api" --testNamePattern="login"
|
||
```
|
||
|
||
### Docker
|
||
```bash
|
||
docker-compose up -d
|
||
docker-compose logs -f backend
|
||
docker-compose up -d --build
|
||
docker-compose down
|
||
```
|
||
|
||
---
|
||
|
||
## API Documentation
|
||
|
||
After starting the backend, access:
|
||
- **Swagger UI:** `http://localhost:8000/docs`
|
||
- **ReDoc:** `http://localhost:8000/redoc`
|
||
|
||
---
|
||
|
||
## Common Tasks Workflows
|
||
|
||
### Adding a New Data Collector
|
||
1. Create collector class in `backend/app/services/collectors/`
|
||
2. Inherit from `BaseCollector`
|
||
3. Implement `fetch()` and `transform()` methods
|
||
4. Register in `collector_registry.py`
|
||
5. Add configuration in database
|
||
6. Write unit tests
|
||
|
||
### Adding a New API Endpoint
|
||
1. Create schema in `backend/app/schemas/`
|
||
2. Create endpoint in `backend/app/api/v1/`
|
||
3. Add to router in `backend/app/api/main.py`
|
||
4. Write integration tests
|
||
5. Update API documentation
|
||
|
||
### Fixing a Bug
|
||
1. Write failing test first
|
||
2. Fix the code to pass test
|
||
3. Verify no regressions
|
||
4. Document in commit message
|
||
|
||
---
|
||
|
||
## Database Schema
|
||
|
||
### Users Table
|
||
```sql
|
||
CREATE TABLE users (
|
||
id SERIAL PRIMARY KEY,
|
||
username VARCHAR(50) UNIQUE NOT NULL,
|
||
password_hash VARCHAR(255) NOT NULL,
|
||
role VARCHAR(20) NOT NULL,
|
||
is_active BOOLEAN DEFAULT true,
|
||
created_at TIMESTAMP DEFAULT NOW()
|
||
);
|
||
```
|
||
|
||
### Permissions Table
|
||
```sql
|
||
CREATE TABLE permissions (
|
||
id SERIAL PRIMARY KEY,
|
||
resource VARCHAR(50) NOT NULL,
|
||
action VARCHAR(20) NOT NULL,
|
||
role VARCHAR(20) NOT NULL
|
||
);
|
||
```
|
||
|
||
### Audit Logs Table
|
||
```sql
|
||
CREATE TABLE audit_logs (
|
||
id SERIAL PRIMARY KEY,
|
||
user_id INTEGER REFERENCES users(id),
|
||
action VARCHAR(100),
|
||
resource VARCHAR(50),
|
||
detail JSONB,
|
||
created_at TIMESTAMP DEFAULT NOW()
|
||
);
|
||
```
|
||
|
||
---
|
||
|
||
## Data Models
|
||
|
||
### GPU Cluster
|
||
```json
|
||
{
|
||
"id": "epoch-gpu-001",
|
||
"name": "Frontier",
|
||
"location": {
|
||
"country": "US",
|
||
"city": "Oak Ridge, TN",
|
||
"lat": 35.9327,
|
||
"lng": -84.3107
|
||
},
|
||
"organization": "Oak Ridge National Laboratory",
|
||
"specs": {
|
||
"gpu_count": 37888,
|
||
"gpu_type": "AMD MI250X",
|
||
"total_flops": 1.54e9,
|
||
"rank": 1
|
||
},
|
||
"last_updated": "2024-01-15T00:00:00Z"
|
||
}
|
||
```
|
||
|
||
### Submarine Cable
|
||
```json
|
||
{
|
||
"id": "cable-001",
|
||
"name": "FASTER",
|
||
"length": 11600,
|
||
"owners": ["Google", "中国移动", "NEC"],
|
||
"points": [
|
||
{"lat": 37.7749, "lng": -122.4194, "city": "San Francisco"},
|
||
{"lat": 35.6762, "lng": 139.6503, "city": "Tokyo"}
|
||
],
|
||
"capacity": "60 Tbps",
|
||
"status": "active"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## MVP Timeline
|
||
|
||
| Phase | Duration | Deliverables |
|
||
|-------|----------|--------------|
|
||
| Phase 1 | Week 1-2 | Project scaffolding, auth, base framework |
|
||
| Phase 2 | Week 3-4 | Data collectors (TOP500, Epoch AI, Hugging Face) |
|
||
| Phase 3 | Week 5 | WebSocket real-time service |
|
||
| Phase 4 | Week 6 | Admin dashboard |
|
||
| Phase 5 | Week 7-8 | UE5 3D display integration |
|
||
| Phase 6 | Week 9 | Testing, deployment, documentation |
|
||
|
||
---
|
||
|
||
## Risk Mitigation
|
||
|
||
| Risk | Likelihood | Impact | Mitigation |
|
||
|------|------------|--------|------------|
|
||
| Data source API changes | Medium | High | Abstract collector layer, graceful degradation |
|
||
| WebSocket performance | Medium | Medium | Rate limiting, heartbeat optimization |
|
||
| UE5 development overrun | Medium | Medium | Phased delivery (2D → 3D) |
|
||
| Data compliance | Low | High | Legal review |
|
||
| 3D compatibility | Low | Medium | Early hardware testing |
|