72 lines
4.1 KiB
Markdown
72 lines
4.1 KiB
Markdown
# Earth 图层 Redis 缓存与 OOM 防护完整计划
|
||
|
||
## Summary
|
||
|
||
目标不是单纯“加缓存”，而是把 Earth 图层读路径改成可控、可观测、可降级的缓存架构，避免演示前高并发、重图层、船只数据膨胀再次把后端打到 OOM 无限重启。
|
||
|
||
前端继续请求原 API，response body 保持兼容。后端新增 Redis 读穿缓存、防击穿锁、stale 兜底、payload budget、主动失效、观测 header 和日志。
|
||
|
||
更新架构图：
|
||
|
||
```mermaid
|
||
flowchart TD
|
||
Earth["Earth 前端<br/>原 API 不变"] --> API["FastAPI Visualization / Layers APIs"]
|
||
|
||
API --> Guard["Request Guard<br/>limit clamp / bbox required / payload budget"]
|
||
Guard --> Cache["Layer Cache Adapter<br/>key / TTL / lock / stale"]
|
||
Cache -->|fresh hit| Redis["Redis<br/>earth:layer:v1:*<br/>fresh + stale payloads"]
|
||
Cache -->|miss or refresh| Builder["Layer Builders<br/>DB query + GeoJSON conversion"]
|
||
Builder --> DB["PostgreSQL / Timescale<br/>authoritative data"]
|
||
Builder --> Budget["Response Budget Check<br/>feature cap / byte cap / diagnostics"]
|
||
Budget --> Cache
|
||
Cache --> API
|
||
API --> Earth
|
||
|
||
Collectors["Collectors / Data writes"] --> DB
|
||
Collectors --> Invalidate["Source-scoped invalidation"]
|
||
Invalidate --> Redis
|
||
|
||
Cache --> Metrics["Structured logs / headers<br/>hit miss stale bypass refresh<br/>bytes features duration"]
|
||
```
|
||
|
||
## Implementation Changes
|
||
|
||
- Add an Earth layer cache adapter that owns Redis keys, TTLs, stale fallback, single-flight locks, JSON serialization, response headers, and graceful Redis bypass.
|
||
- Use `earth:layer:v1:{layer}:{params}` for fresh cache, `earth:layer:v1:{layer}:{params}:stale` for stale fallback, and `earth:layer:lock:v1:{hash}` for rebuild locks.
|
||
- Cache policy:
|
||
- `cables`, `landing-points`: fresh `6h`, stale `24h`
|
||
- `satellites`: fresh `15m`, stale `2h`
|
||
- `compute-centers`: fresh `10m`, stale `1h`
|
||
- `bgp-collectors`, `bgp-anomalies`, `bgp-incidents`, `geo/summary`: fresh `30-60s`, stale `10m`
|
||
- `vessels snapshot`: fresh `5s`, stale `30s`, with bbox rounded to `0.1` degrees and key including `zoom/type/limit/since_minutes`
|
||
- Prevent cache stampedes with `SET NX EX` locks. The lock holder refreshes; other requests prefer stale, wait briefly, then fall back to the guarded DB path.
|
||
- Enforce payload budgets on every cached layer: maximum features, maximum serialized bytes, and diagnostics when truncation happens.
|
||
- Keep vessel snapshot viewport-first: require bbox, clamp low-zoom limits, never build an unbounded all-vessel GeoJSON for Earth startup.
|
||
- Add cache observability headers: `X-Planet-Cache`, `X-Planet-Cache-Features`, `X-Planet-Cache-Bytes`, and development-only `X-Planet-Cache-Key`.
|
||
- Add super-admin system endpoints for Earth layer cache status and clearing.
|
||
|
||
## Public Interfaces
|
||
|
||
- Frontend request URLs stay unchanged.
|
||
- Response bodies stay compatible.
|
||
- New optional response headers report cache state.
|
||
- New system endpoints:
|
||
- `GET /api/v1/system/cache/earth-layers`
|
||
- `DELETE /api/v1/system/cache/earth-layers`
|
||
- Redis key contract: `earth:layer:v1:*`. Existing news keys remain `earth_news:target_location:*`.
|
||
|
||
## Test Plan
|
||
|
||
- Unit-test key generation, bbox rounding, payload budget truncation, Redis miss/hit, stale fallback, Redis bypass, and single-flight lock behavior.
|
||
- API-test repeated requests for cache headers, super-admin cache status/clear endpoints, and vessel snapshot bbox/limit safeguards.
|
||
- Regression-test existing layer guard behavior and vessel type forwarding.
|
||
- Verify Redis outage does not break Earth API responses.
|
||
- Verify large vessel requests return bounded payload diagnostics instead of exhausting memory.
|
||
|
||
## Assumptions
|
||
|
||
- PostgreSQL remains the authoritative data source; Redis is disposable read-through cache.
|
||
- First phase does not change frontend rendering. If Three.js rendering becomes the bottleneck, that is a separate frontend performance task.
|
||
- When safety conflicts with completeness, vessel responses prefer bounded/truncated data plus diagnostics over risking backend OOM.
|
||
- GeoJSON schema changes should bump the Redis key version from `v1` to `v2`.
|