planet/docs/plans/earth-layer-redis-cache-oom-guard-plan.md

Earth 图层 Redis 缓存与 OOM 防护完整计划

Summary

目标不是单纯“加缓存”，而是把 Earth 图层读路径改成可控、可观测、可降级的缓存架构，避免演示前高并发、重图层、船只数据膨胀再次把后端打到 OOM 无限重启。

前端继续请求原 API，response body 保持兼容。后端新增 Redis 读穿缓存、防击穿锁、stale 兜底、payload budget、主动失效、观测 header 和日志。

更新架构图：

flowchart TD
  Earth["Earth 前端<br/>原 API 不变"] --> API["FastAPI Visualization / Layers APIs"]

  API --> Guard["Request Guard<br/>limit clamp / bbox required / payload budget"]
  Guard --> Cache["Layer Cache Adapter<br/>key / TTL / lock / stale"]
  Cache -->|fresh hit| Redis["Redis<br/>earth:layer:v1:*<br/>fresh + stale payloads"]
  Cache -->|miss or refresh| Builder["Layer Builders<br/>DB query + GeoJSON conversion"]
  Builder --> DB["PostgreSQL / Timescale<br/>authoritative data"]
  Builder --> Budget["Response Budget Check<br/>feature cap / byte cap / diagnostics"]
  Budget --> Cache
  Cache --> API
  API --> Earth

  Collectors["Collectors / Data writes"] --> DB
  Collectors --> Invalidate["Source-scoped invalidation"]
  Invalidate --> Redis

  Cache --> Metrics["Structured logs / headers<br/>hit miss stale bypass refresh<br/>bytes features duration"]

Implementation Changes

• Add an Earth layer cache adapter that owns Redis keys, TTLs, stale fallback, single-flight locks, JSON serialization, response headers, and graceful Redis bypass.
• Use earth:layer:v1:{layer}:{params} for fresh cache, earth:layer:v1:{layer}:{params}:stale for stale fallback, and earth:layer:lock:v1:{hash} for rebuild locks.
• Cache policy:
  • cables, landing-points: fresh 6h, stale 24h
  • satellites: fresh 15m, stale 2h
  • compute-centers: fresh 10m, stale 1h
  • bgp-collectors, bgp-anomalies, bgp-incidents, geo/summary: fresh 30-60s, stale 10m
  • vessels snapshot: fresh 5s, stale 30s, with bbox rounded to 0.1 degrees and key including zoom/type/limit/since_minutes
• Prevent cache stampedes with SET NX EX locks. The lock holder refreshes; other requests prefer stale, wait briefly, then fall back to the guarded DB path.
• Enforce payload budgets on every cached layer: maximum features, maximum serialized bytes, and diagnostics when truncation happens.
• Keep vessel snapshot viewport-first: require bbox, clamp low-zoom limits, never build an unbounded all-vessel GeoJSON for Earth startup.
• Add cache observability headers: X-Planet-Cache, X-Planet-Cache-Features, X-Planet-Cache-Bytes, and development-only X-Planet-Cache-Key.
• Add super-admin system endpoints for Earth layer cache status and clearing.

Public Interfaces

• Frontend request URLs stay unchanged.
• Response bodies stay compatible.
• New optional response headers report cache state.
• New system endpoints:
  • GET /api/v1/system/cache/earth-layers
  • DELETE /api/v1/system/cache/earth-layers
• Redis key contract: earth:layer:v1:*. Existing news keys remain earth_news:target_location:*.

Test Plan

• Unit-test key generation, bbox rounding, payload budget truncation, Redis miss/hit, stale fallback, Redis bypass, and single-flight lock behavior.
• API-test repeated requests for cache headers, super-admin cache status/clear endpoints, and vessel snapshot bbox/limit safeguards.
• Regression-test existing layer guard behavior and vessel type forwarding.
• Verify Redis outage does not break Earth API responses.
• Verify large vessel requests return bounded payload diagnostics instead of exhausting memory.

Assumptions

• PostgreSQL remains the authoritative data source; Redis is disposable read-through cache.
• First phase does not change frontend rendering. If Three.js rendering becomes the bottleneck, that is a separate frontend performance task.
• When safety conflicts with completeness, vessel responses prefer bounded/truncated data plus diagnostics over risking backend OOM.
• GeoJSON schema changes should bump the Redis key version from v1 to v2.