planet/docs/plans/agents-datasource-health-stage2-tasks.md

Datasource Health Stage 2 Tasks

Goal

Stage 2 focuses on the first practical operational layer:

• deterministic datasource health checks
• persisted health results
• health visibility through API and UI
• no agent-assisted repair yet

This stage should make Planet capable of answering:

• which collectors are healthy
• which collectors are degraded
• which collectors are failing
• why they are failing at a basic deterministic level

Scope

Included:

• datasource health data model
• deterministic health check service
• manual and scheduled health check triggers
• health result APIs
• frontend visibility

Excluded:

• LLM reasoning
• web-search-based repair proposals
• automatic endpoint rewriting
• runtime override application

Delivery Target

At the end of Stage 2, an operator should be able to:

1. see health status for each collector
2. trigger a health check manually
3. inspect the latest failure reason
4. inspect the last checked endpoint
5. understand whether the problem is:
   • unreachable
   • auth-related
   • rate-limit-related
   • schema-related
   • empty-data-related

Work Breakdown

A. Data Model

A1. Add datasource health record table

Create a new model, for example:

• backend/app/models/datasource_health_check.py

Suggested fields:

• id
• datasource_id
• collector_name
• endpoint_checked
• status
• http_status
• content_type
• latency_ms
• sample_count
• error_message
• details
• checked_at

Suggested status enum values:

• healthy
• degraded
• failed
• schema_changed
• rate_limited
• auth_required
• empty_result

A2. Add datasource health summary fields

Option A:

• keep summary only in the health check table

Option B:

• also add summary fields on data_sources

Recommended first step:

• do not mutate data_sources schema yet
• derive summary from the latest health record

A3. Migration task

Add migration for the health table.

Deliverables:

• migration file
• model registration

B. Health Check Engine

B1. Define health check service

Add a new service module, for example:

• backend/app/services/datasource_health.py

Responsibilities:

• resolve effective endpoint
• execute deterministic check
• classify result
• persist health record

B2. Define shared result schema

Create a typed result object, for example:

• HealthCheckResult

Suggested fields:

• status
• endpoint_checked
• http_status
• content_type
• latency_ms
• sample_count
• error_message
• details

B3. Implement base deterministic checks

Every datasource should go through a minimal baseline check:

1. resolve endpoint
2. perform request
3. measure latency
4. inspect status code
5. inspect content type
6. inspect body shape

Classification rules:

• network error -> failed
• HTTP 401/403 -> auth_required
• HTTP 429 -> rate_limited
• HTTP 404/410 -> failed
• parse failure -> schema_changed
• zero or suspiciously empty results -> empty_result or degraded
• valid parse -> healthy

B4. Add collector-aware adapters

Some collectors do not use the same fetch semantics.

Add adapter profiles such as:

• http_json
• http_csv
• html_scrape
• stream_probe
• auth_session_http

Initial mapping suggestion:

• huggingface, peeringdb, cloudflare -> http_json
• fao -> http_csv
• top500, epoch_ai, telegeography live_map -> html_scrape
• ris_live -> stream_probe
• spacetrack -> auth_session_http

B5. Add sample validation hooks

For each adapter, add a lightweight validation rule.

Examples:

• JSON array length > 0
• CSV rows > 1
• HTML page contains expected table or script patterns
• stream source yields at least one valid event within timeout

C. Persistence and Query Layer

C1. Save every check run

Each health check should insert a record.

Do not overwrite history in Stage 2.

C2. Add latest-health query helpers

Add helper functions to fetch:

• latest health record by datasource
• latest failed health record
• recent health history

C3. Optional retention policy

For Stage 2, retention can be deferred.

If desired, keep only:

• last N records per datasource

D. API Layer

D1. Add health list endpoint

Suggested endpoint:

• GET /api/v1/datasources/health

Returns:

• datasource id
• collector name
• current endpoint
• latest health status
• last checked time
• short reason

D2. Add per-datasource health detail endpoint

Suggested endpoint:

• GET /api/v1/datasources/{id}/health

Returns:

• latest record
• recent history
• detailed classification fields

D3. Add manual health trigger endpoint

Suggested endpoint:

• POST /api/v1/datasources/{id}/health-check

Behavior:

• run a health check now
• persist the result
• return the new record

D4. Add bulk health trigger endpoint

Suggested endpoint:

• POST /api/v1/datasources/health-check-all

Behavior:

• enqueue or run health checks for all active datasources

E. Scheduling

E1. Add health scheduler task

Decide scheduling strategy.

Recommended first version:

• run collector jobs and health checks separately
• health checks run on a lower frequency

Suggested frequency:

• every 6h or 12h for most datasources
• optionally on-demand only in the very first cut

E2. Prevent health check collision with collection

Rules:

• health checks should not disrupt active collection
• they should use light requests
• if a collector is currently running, health check may:
  • skip
  • or use a lightweight endpoint probe only

F. Frontend

F1. Add health columns to datasource list

Update:

• frontend/src/pages/DataSources/DataSources.tsx

Suggested new columns:

• health status
• last checked
• reason summary

F2. Add manual health check action

Per datasource:

• button or dropdown action:
  • 健康检查

F3. Add health detail drawer or modal

Show:

• endpoint checked
• status
• HTTP status
• content type
• sample count
• error message
• last few results

F4. Add basic visual language

Suggested colors:

• green -> healthy
• yellow -> degraded
• orange -> rate-limited / auth-required
• red -> failed / schema-changed

G. Observability

G1. Structured logging

Every health check should log:

• datasource id
• collector name
• endpoint
• status
• latency
• failure class

G2. Optional metrics

If metrics are added later, useful counters include:

• health checks total
• health checks failed
• schema changes detected
• rate limited checks

H. Tests

H1. Unit tests

Add tests for:

• status classification
• content type classification
• adapter behavior
• latest-health query helpers

H2. API tests

Add tests for:

• health endpoints require auth
• manual trigger endpoint works
• list endpoint returns latest status

H3. Failure-path tests

Add coverage for:

• HTTP 404
• HTTP 429
• invalid JSON
• empty response
• parse mismatch

Suggested File Plan

Possible implementation files:

• backend/app/models/datasource_health_check.py
• backend/app/services/datasource_health.py
• backend/app/schemas/datasource_health.py
• backend/app/api/v1/datasource_health.py
• migration file under the project migration system

Likely touched existing files:

• backend/app/api/main.py
• frontend/src/pages/DataSources/DataSources.tsx
• backend/tests/test_api.py

Suggested Execution Order

1. Add model and migration
2. Add service and result schema
3. Add deterministic adapters
4. Add manual trigger API
5. Add list/detail API
6. Add frontend visibility
7. Add scheduled checks
8. Expand tests

Minimal First Milestone

If we want the fastest useful slice, do this first:

1. health table
2. deterministic check service
3. manual per-datasource health check API
4. latest health list API
5. frontend status badge column

That is enough to start operating the system and will provide the input layer for Stage 3.

Dependency On Later Stages

Stage 2 outputs become direct inputs for Stage 3.

Specifically:

• failed or schema-changed health records become agent triggers
• health history becomes repair context
• endpoint_checked becomes proposal baseline

Success Criteria

Stage 2 is done when:

• every active datasource can be health-checked deterministically
• the latest health state is visible in API and UI
• operators can manually trigger checks
• failures are categorized into stable machine-readable statuses
• no LLM is required for core health visibility