first commit
This commit is contained in:
rayd1o
357
project_context.md
Normal file
357
project_context.md
Normal file
@@ -0,0 +1,357 @@
|
||||
# project_context.md
|
||||
|
||||
**AI 需要知道的静态事实。用于理解项目，非约束。**
|
||||
|
||||
---
|
||||
|
||||
## Project Overview
|
||||
|
||||
**智能星球计划 (Intelligent Planet Plan)** - A situational awareness system for data-centric competition.
|
||||
|
||||
### Core Vision
|
||||
|
||||
构建人类智能空间的"实时全景图" - 在智能时代，同时监测"现实层宇宙"和"认知层宇宙"。
|
||||
|
||||
### Four Core Elements
|
||||
|
||||
| Layer | Element | Description |
|
||||
|-------|---------|-------------|
|
||||
| L1 | 新兴技术支撑 | AI算力、模型生态、云基础设施 |
|
||||
| L2 | 关键基础设施 | 卫星、海底光缆、IXP、路由 |
|
||||
| L3 | 组织制度资源 | 规则制定权、顶层设计 |
|
||||
| L4 | 文化内容供给 | 新闻、社交视频、舆论情绪 |
|
||||
|
||||
---
|
||||
|
||||
## Technology Stack
|
||||
|
||||
### Backend
|
||||
- **Framework:** Python FastAPI 0.109+
|
||||
- **ORM:** SQLAlchemy 2.0+
|
||||
- **Database Migration:** Alembic
|
||||
- **Task Queue:** Celery 5.3+
|
||||
- **Message Queue:** Kafka 3.0+
|
||||
- **Caching:** Redis 7.0+
|
||||
|
||||
### Frontend
|
||||
- **Framework:** React 18
|
||||
- **UI Library:** Ant Design Pro
|
||||
- **HTTP Client:** Axios
|
||||
- **State Management:** React Query
|
||||
- **Real-time:** Socket.io-client
|
||||
- **Charts:** ECharts
|
||||
|
||||
### Visualization (UE5)
|
||||
- **Engine:** Unreal Engine 5.3+
|
||||
- **Geospatial:** Cesium for Unreal 1.5+
|
||||
- **Particles:** Niagara
|
||||
- **3D Rendering:** Nanite + Lumen
|
||||
|
||||
### Database
|
||||
- **Relational:** PostgreSQL 15+ (users, config)
|
||||
- **Time-series:** TimescaleDB
|
||||
- **Cache:** Redis 7+ (sessions, cache)
|
||||
- **Storage:** MinIO (S3-compatible)
|
||||
|
||||
### Deployment
|
||||
- **Container:** Docker 24+
|
||||
- **Orchestration:** Docker Compose
|
||||
- **Reverse Proxy:** Nginx
|
||||
|
||||
---
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
├── backend/ # FastAPI Backend
|
||||
│ ├── app/
|
||||
│ │ ├── api/
|
||||
│ │ │ ├── v1/
|
||||
│ │ │ │ ├── auth.py
|
||||
│ │ │ │ ├── users.py
|
||||
│ │ │ │ ├── datasources.py
|
||||
│ │ │ │ ├── tasks.py
|
||||
│ │ │ │ └── websocket.py
|
||||
│ │ │ └── endpoints/
|
||||
│ │ ├── core/
|
||||
│ │ │ ├── config.py
|
||||
│ │ │ ├── security.py
|
||||
│ │ │ └── websocket_manager.py
|
||||
│ │ ├── models/
|
||||
│ │ │ ├── user.py
|
||||
│ │ │ ├── datasource.py
|
||||
│ │ │ └── task.py
|
||||
│ │ ├── schemas/
|
||||
│ │ │ ├── user.py
|
||||
│ │ │ └── datasource.py
|
||||
│ │ ├── services/
|
||||
│ │ │ ├── collectors/
|
||||
│ │ │ │ ├── base.py
|
||||
│ │ │ │ ├── top500.py
|
||||
│ │ │ │ ├── epoch_ai.py
|
||||
│ │ │ │ └── huggingface.py
|
||||
│ │ │ └── analysis.py
|
||||
│ │ ├── tasks/
|
||||
│ │ │ └── scheduler.py
|
||||
│ │ └── db/
|
||||
│ │ └── session.py
|
||||
│ ├── tests/
|
||||
│ │ ├── api/
|
||||
│ │ ├── unit/
|
||||
│ │ └── conftest.py
|
||||
│ ├── requirements.txt
|
||||
│ └── alembic/
|
||||
│
|
||||
├── frontend/ # React Admin
|
||||
│ ├── src/
|
||||
│ │ ├── api/ # Axios instances
|
||||
│ │ ├── components/ # Reusable components
|
||||
│ │ ├── pages/ # Route pages
|
||||
│ │ │ ├── Login/
|
||||
│ │ │ ├── Dashboard/
|
||||
│ │ │ ├── Users/
|
||||
│ │ │ ├── DataSources/
|
||||
│ │ │ └── Settings/
|
||||
│ │ ├── stores/ # Zustand/Redux
|
||||
│ │ ├── hooks/
|
||||
│ │ ├── types/
|
||||
│ │ └── App.tsx
|
||||
│ ├── package.json
|
||||
│ └── tests/
|
||||
│
|
||||
├── unreal/ # UE5 Project
|
||||
│ ├── Content/
|
||||
│ │ ├── Maps/
|
||||
│ │ ├── Blueprints/
|
||||
│ │ └── Materials/
|
||||
│ ├── Plugins/
|
||||
│ └── Source/
|
||||
│
|
||||
├── data/ # Static data files
|
||||
├── docs/ # Documentation
|
||||
├── scripts/ # Utility scripts
|
||||
├── docker-compose.yml
|
||||
├── .env.example
|
||||
├── rules.md
|
||||
├── project_context.md
|
||||
├── agents.md
|
||||
└── README.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Role-Based Access Control
|
||||
|
||||
| Role | Permissions | Description |
|
||||
|------|-------------|-------------|
|
||||
| **super_admin** | Full access | System init, **不可删除** |
|
||||
| **admin** | All except user management | Daily operations |
|
||||
| **operator** | View + manual operations | Manual refresh, alert confirm |
|
||||
| **viewer** | Read-only | View dashboard only |
|
||||
|
||||
### Default Users (MVP)
|
||||
- `admin` / `admin123` (super_admin)
|
||||
- `operator` / `operator123` (operator)
|
||||
- `viewer` / `viewer123` (viewer)
|
||||
|
||||
---
|
||||
|
||||
## Data Sources
|
||||
|
||||
| Priority | Source | Frequency | Module | Description |
|
||||
|----------|--------|-----------|--------|-------------|
|
||||
| P0 | TOP500 | 4 hours | L1 | Supercomputer rankings |
|
||||
| P0 | Epoch AI | 1 hour | L1 | GPU cluster data |
|
||||
| P0 | Hugging Face | 2 hours | L1 | Model ecosystem |
|
||||
| P0 | GitHub | 4 hours | L1 | Code contributions |
|
||||
| P0 | TeleGeography | Daily | L2 | Submarine cables, cloud infra |
|
||||
| P0 | PeeringDB | 2 hours | L2 | IXP data |
|
||||
| P1 | OECD.AI | 6 hours | L1 | Policy/investment |
|
||||
| P1 | Cloudflare Radar | 1 hour | L2 | Traffic/blocking |
|
||||
| P1 | CAIDA BGPStream | 15 minutes | L2 | Route monitoring |
|
||||
|
||||
---
|
||||
|
||||
## 3D Display Specifications
|
||||
|
||||
- **Display Type:** Polarized 3D (glasses-style)
|
||||
- **Screen Size:** 2m × 3m
|
||||
- **Resolution:** 4K (3840×2160)
|
||||
- **Refresh Rate:** 120Hz
|
||||
- **Engine:** Unreal Engine 5.3+
|
||||
- **Plugin:** Cesium for Unreal
|
||||
|
||||
### Communication
|
||||
- WebSocket for real-time data
|
||||
- 30-second heartbeat
|
||||
- Data frame updates every 5 minutes (configurable)
|
||||
|
||||
---
|
||||
|
||||
## Build Commands Reference
|
||||
|
||||
### Backend
|
||||
```bash
|
||||
cd backend
|
||||
pip install -r requirements.txt
|
||||
python -m uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||||
docker-compose up -d backend
|
||||
ruff check . && black --check .
|
||||
pytest -v && pytest tests/api/test_auth.py::test_login -v
|
||||
```
|
||||
|
||||
### Frontend
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev
|
||||
npm run build
|
||||
npm run lint && npm run lint:fix
|
||||
npm test -- --testPathPattern="api" --testNamePattern="login"
|
||||
```
|
||||
|
||||
### Docker
|
||||
```bash
|
||||
docker-compose up -d
|
||||
docker-compose logs -f backend
|
||||
docker-compose up -d --build
|
||||
docker-compose down
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Documentation
|
||||
|
||||
After starting the backend, access:
|
||||
- **Swagger UI:** `http://localhost:8000/docs`
|
||||
- **ReDoc:** `http://localhost:8000/redoc`
|
||||
|
||||
---
|
||||
|
||||
## Common Tasks Workflows
|
||||
|
||||
### Adding a New Data Collector
|
||||
1. Create collector class in `backend/app/services/collectors/`
|
||||
2. Inherit from `BaseCollector`
|
||||
3. Implement `fetch()` and `transform()` methods
|
||||
4. Register in `collector_registry.py`
|
||||
5. Add configuration in database
|
||||
6. Write unit tests
|
||||
|
||||
### Adding a New API Endpoint
|
||||
1. Create schema in `backend/app/schemas/`
|
||||
2. Create endpoint in `backend/app/api/v1/`
|
||||
3. Add to router in `backend/app/api/main.py`
|
||||
4. Write integration tests
|
||||
5. Update API documentation
|
||||
|
||||
### Fixing a Bug
|
||||
1. Write failing test first
|
||||
2. Fix the code to pass test
|
||||
3. Verify no regressions
|
||||
4. Document in commit message
|
||||
|
||||
---
|
||||
|
||||
## Database Schema
|
||||
|
||||
### Users Table
|
||||
```sql
|
||||
CREATE TABLE users (
|
||||
id SERIAL PRIMARY KEY,
|
||||
username VARCHAR(50) UNIQUE NOT NULL,
|
||||
password_hash VARCHAR(255) NOT NULL,
|
||||
role VARCHAR(20) NOT NULL,
|
||||
is_active BOOLEAN DEFAULT true,
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
### Permissions Table
|
||||
```sql
|
||||
CREATE TABLE permissions (
|
||||
id SERIAL PRIMARY KEY,
|
||||
resource VARCHAR(50) NOT NULL,
|
||||
action VARCHAR(20) NOT NULL,
|
||||
role VARCHAR(20) NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
### Audit Logs Table
|
||||
```sql
|
||||
CREATE TABLE audit_logs (
|
||||
id SERIAL PRIMARY KEY,
|
||||
user_id INTEGER REFERENCES users(id),
|
||||
action VARCHAR(100),
|
||||
resource VARCHAR(50),
|
||||
detail JSONB,
|
||||
created_at TIMESTAMP DEFAULT NOW()
|
||||
);
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Data Models
|
||||
|
||||
### GPU Cluster
|
||||
```json
|
||||
{
|
||||
"id": "epoch-gpu-001",
|
||||
"name": "Frontier",
|
||||
"location": {
|
||||
"country": "US",
|
||||
"city": "Oak Ridge, TN",
|
||||
"lat": 35.9327,
|
||||
"lng": -84.3107
|
||||
},
|
||||
"organization": "Oak Ridge National Laboratory",
|
||||
"specs": {
|
||||
"gpu_count": 37888,
|
||||
"gpu_type": "AMD MI250X",
|
||||
"total_flops": 1.54e9,
|
||||
"rank": 1
|
||||
},
|
||||
"last_updated": "2024-01-15T00:00:00Z"
|
||||
}
|
||||
```
|
||||
|
||||
### Submarine Cable
|
||||
```json
|
||||
{
|
||||
"id": "cable-001",
|
||||
"name": "FASTER",
|
||||
"length": 11600,
|
||||
"owners": ["Google", "中国移动", "NEC"],
|
||||
"points": [
|
||||
{"lat": 37.7749, "lng": -122.4194, "city": "San Francisco"},
|
||||
{"lat": 35.6762, "lng": 139.6503, "city": "Tokyo"}
|
||||
],
|
||||
"capacity": "60 Tbps",
|
||||
"status": "active"
|
||||
}
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## MVP Timeline
|
||||
|
||||
| Phase | Duration | Deliverables |
|
||||
|-------|----------|--------------|
|
||||
| Phase 1 | Week 1-2 | Project scaffolding, auth, base framework |
|
||||
| Phase 2 | Week 3-4 | Data collectors (TOP500, Epoch AI, Hugging Face) |
|
||||
| Phase 3 | Week 5 | WebSocket real-time service |
|
||||
| Phase 4 | Week 6 | Admin dashboard |
|
||||
| Phase 5 | Week 7-8 | UE5 3D display integration |
|
||||
| Phase 6 | Week 9 | Testing, deployment, documentation |
|
||||
|
||||
---
|
||||
|
||||
## Risk Mitigation
|
||||
|
||||
| Risk | Likelihood | Impact | Mitigation |
|
||||
|------|------------|--------|------------|
|
||||
| Data source API changes | Medium | High | Abstract collector layer, graceful degradation |
|
||||
| WebSocket performance | Medium | Medium | Rate limiting, heartbeat optimization |
|
||||
| UE5 development overrun | Medium | Medium | Phased delivery (2D → 3D) |
|
||||
| Data compliance | Low | High | Legal review |
|
||||
| 3D compatibility | Low | Medium | Early hardware testing |
|
||||
Reference in New Issue
Block a user