release: bump version to 0.59.0
This commit is contained in:
rayd1o
@@ -95,9 +95,18 @@ The AI settings page uses:
|
||||
- `POST /api/v1/settings/integrations/ai-provider/connect`
|
||||
- `GET /api/v1/settings/integrations/ai-provider/secrets`
|
||||
- `GET /api/v1/settings/integrations/ai-provider/presets`
|
||||
- `GET /api/v1/settings/ai-prompts`
|
||||
- `PUT /api/v1/settings/ai-prompts/{task_key}`
|
||||
- `POST /api/v1/settings/ai-prompts/{task_key}/reset`
|
||||
|
||||
These endpoints require an authenticated user. The `secrets` endpoint is only used when the settings page reveals a key or token; hiding the field restores the masked preview.
|
||||
|
||||
The `ai-prompts` endpoints back the Prompts tab in AI settings. Shipped defaults come from versioned backend resources, while business code references stable task keys. The API stores only operator overrides. Resetting a prompt removes the override and falls back to the current shipped default.
|
||||
|
||||
### Prompt Boundary
|
||||
|
||||
`aiprovider` is a pure model adapter and does not inject a global business system prompt. News localization, alert briefing, BGP briefing, location factcheck, datasource mapping, and credential guide generation each resolve their own effective prompt by task key. Alert-analysis system prompts are only sent by alert-related tasks and do not leak into other LLM calls.
|
||||
|
||||
### AI provider internal API
|
||||
|
||||
Internal-only endpoints:
|
||||
@@ -287,7 +296,6 @@ SERVICE_VERSION=0.1.0
|
||||
AI_PROVIDER_SERVICE_TOKEN=change_me
|
||||
AI_TIMEOUT_SECONDS=60
|
||||
AI_HTTP_RETRY_ATTEMPTS=2
|
||||
AI_ANALYSIS_SYSTEM_PROMPT=你是态势感知分析助手。请基于输入的上下文、观测与约束，输出结构化、克制、可执行的分析。
|
||||
```
|
||||
|
||||
Optional provider-specific keys:
|
||||
|
||||
@@ -86,16 +86,10 @@ async def run(self, db):
|
||||
| TeleGeography | submarine_cable | Submarine cable information | 7 days |
|
||||
| BarentsWatch AIS | vessel | AIS vessel positions, speed, heading, MMSI, and related fields | Collector settings |
|
||||
| AISStream Vessels | vessel_ais | AIS WebSocket realtime stream, written to the raw observation layer and displayed through aggregation | Collector settings |
|
||||
| Earth Admin-0 Boundaries | earth_admin0_boundaries | Downloads the configured country-boundary source, saves an artifact, and writes an `earth_boundary_source` manifest record | Collector settings |
|
||||
| Earth Coastline | earth_coastline | Downloads the configured coastline source, saves an artifact, and writes an `earth_boundary_source` manifest record | Collector settings |
|
||||
| Earth Claim Lines | earth_claim_lines | Downloads the configured claim-line source, saves an artifact, and writes an `earth_boundary_source` manifest record | Collector settings |
|
||||
| Earth PMTiles Builder | earth_boundary_tiles | Reads the three Earth boundary source records and builds / registers the PMTiles artifact | Collector settings |
|
||||
|
||||
AIS vessel collectors use a different persistence path from regular `CollectedData` collectors. BarentsWatch, AISStream, and custom `vessel_ais` sources write into the AIS raw observation layer first, then the aggregation service merges those observations into the GeoJSON and detail payloads used by the Earth vessel layer. This preserves source, transport, field conflicts, and observation time instead of letting one realtime source overwrite the final display table.
|
||||
|
||||
Earth boundaries are now split into three real source collectors plus one downstream builder. `earth_admin0_boundaries`, `earth_coastline`, and `earth_claim_lines` read endpoint, headers, auth, and `config.target_schema=earth_boundary_source` from Collector Settings. Triggering them requests the configured endpoint, writes the full response to `data/earth-boundary-sources/<collector>/<sha256>.*`, and stores sha256, feature count, license, artifact path, sample properties, and mapping metadata in `CollectedData`.
|
||||
|
||||
`earth_boundary_tiles` no longer means source-data collection. It reads the latest successful records from those three source collectors; if any source is missing, the task fails as "not ready" and does not register "4 high-precision tile" records. Once all sources exist, it uses `tippecanoe` / `pmtiles` to build `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`; if those tools are missing, the task fails with the missing-tool message. There is no legacy low-precision fallback for country boundaries.
|
||||
Earth boundaries are no longer data collectors. They are Earth static rendering assets: the Earth Assets settings panel owns source configuration, and `/api/v1/earth/boundaries/*` builds `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`. When no high-precision PMTiles artifact is available locally, the frontend uses the bundled low-precision GeoJSON fallback and does not write boundary records to `CollectedData`.
|
||||
|
||||
TOP500 and Epoch AI compute sources do not always provide usable coordinates. The unified Earth compute-center endpoint uses only valid source-provided coordinates or `compute_center_locations` dimension-table coordinates during the main map startup path; records without coordinates are returned as `unresolved` instead of being rendered from a local registry, country centroid, or guessed city. When users manually collect candidates, the backend queries ROR and Nominatim/OpenStreetMap from source fields; accepted candidates are saved into `compute_center_locations` and rendered from that table on the next layer refresh.
|
||||
|
||||
@@ -306,7 +300,7 @@ State semantics:
|
||||
- `reconnecting`: upstream or network interruption; the collector records `AISSourceHealth` and waits before reconnecting.
|
||||
- `stopped` / `cancelled`: stopped by a test limit or user action.
|
||||
|
||||
AISStream connectivity validation reads the saved collector configuration, environment variables, and `AISSTREAM_API_KEY` in `~/.zshrc` through `datasource_connectivity.py`. For actual collection, the most reliable path is saving the API key in `Settings -> Collector Settings -> AISStream Vessels`; if the key only lives in `~/.zshrc`, confirm that the backend process inherited it.
|
||||
AISStream connectivity validation reads the saved collector configuration, environment variables, and `AISSTREAM_API_KEY` in `~/.zshrc` through `datasource_connectivity.py`. For actual collection, the most reliable path is saving the API key in `Collection Management -> Collectors -> AISStream Vessels`; if the key only lives in `~/.zshrc`, confirm that the backend process inherited it.
|
||||
|
||||
The console manages AISStream from `/datasources -> Realtime Streams`, not from the normal finite collection progress bar. The realtime stream API aggregates runtime state, health, configuration preview, and raw observation counters:
|
||||
|
||||
@@ -374,9 +368,9 @@ GET /api/v1/layers/bgp/collectors?bbox=lon_min,lat_min,lon_max,lat_max&zoom=12&l
|
||||
|
||||
`/api/v1/data-products/*` is for aggregate panels and keeps a global statistics scope independent of the map bbox. `/api/v1/layers/*` is for map rendering, requires `bbox` and `zoom`, defaults to `limit=1000`, and caps `limit` at `5000`; low zoom falls back to a smaller response cap and reports `degraded`, `truncated`, `limit_clamped`, and `stats_scope=viewport` in `diagnostics`. Non-vessel layers currently reuse the existing GeoJSON converters before the guard layer; future product-specific queries can push bbox filtering deeper.
|
||||
|
||||
## X. Collector Settings And Connectivity Validation
|
||||
## X. Collectors And Connectivity Validation
|
||||
|
||||
The console "Collector Settings" page owns endpoint, headers, timeouts, retries, and credentials for all built-in collectors. Connectivity is derived by the backend checksum rather than by frontend button styling:
|
||||
The console "Collectors" page owns endpoint, headers, timeouts, retries, and credentials for all built-in collectors. Connectivity is derived by the backend checksum rather than by frontend button styling:
|
||||
|
||||
- endpoint
|
||||
- auth type
|
||||
@@ -397,7 +391,7 @@ POST /api/v1/settings/credential-guides/{provider}/generate
|
||||
POST /api/v1/settings/credential-guides/{provider}/reset
|
||||
```
|
||||
|
||||
See [Collector Settings and Connectivity Validation](/home/ray/dev/linkong/planet/docs/technical/en/datasource-collector-settings-connectivity.md) for the full flow.
|
||||
See [Collectors and Connectivity Validation](/home/ray/dev/linkong/planet/docs/technical/en/datasource-collector-settings-connectivity.md) for the full flow.
|
||||
|
||||
## XI. Data Usage
|
||||
|
||||
|
||||
@@ -8,12 +8,12 @@ The console now separates the "data source catalog" from "collector configuratio
|
||||
- Lists all data sources, including built-in and custom sources.
|
||||
- Clicking a name only opens an information drawer.
|
||||
- Focuses on status, manual collection, and running collection tasks.
|
||||
- `/settings?tab=collector_credentials`
|
||||
- Displays as "Collector Settings".
|
||||
- `/collection-management?tab=collector_credentials`
|
||||
- Displays as "Collectors".
|
||||
- Owns endpoint, headers, base parameters, and credentials.
|
||||
- Every collector exposes a connection button for health checks.
|
||||
|
||||
This reduces first-use confusion: API endpoints, headers, credentials, and custom source configuration all belong to collector settings instead of being scattered across the data source list and system settings.
|
||||
This reduces first-use confusion: API endpoints, headers, credentials, and custom source configuration all belong to Collectors instead of being scattered across the data source list and system settings.
|
||||
|
||||
## User-Facing Rules
|
||||
|
||||
@@ -53,7 +53,7 @@ Current behavior:
|
||||
|
||||
`data-source-bulk-toolbar__running-pill` is the styling entry point for the "Collecting" pill. It is aligned with other status tags, while hover treatment, arrow affordance, and blue outline indicate interactivity.
|
||||
|
||||
### Collector Settings
|
||||
### Collectors
|
||||
|
||||
File:
|
||||
|
||||
@@ -61,7 +61,7 @@ File:
|
||||
|
||||
Current behavior:
|
||||
|
||||
- The `collector_credentials` tab is displayed as "Collector Settings".
|
||||
- The `collector_credentials` tab is displayed as "Collectors" under `/collection-management`.
|
||||
- A select lists built-in collectors and supports maintaining custom supplemental sources that merge into built-in data.
|
||||
- The only button beside the select is a plug icon for health checks.
|
||||
- Status tags below the select show:
|
||||
@@ -307,7 +307,7 @@ Files:
|
||||
|
||||
Custom sources are supplemental inputs for existing target schemas, not isolated data islands. The most complete target today is `vessel_ais`: a custom REST or WebSocket source is mapped deterministically, written into AIS raw observations, and then pushed to Earth through the `vessels` WebSocket channel.
|
||||
|
||||
Earth high-precision boundaries use the same target-schema mechanism. `earth_boundary_source` receives mapped records for `earth_admin0_boundaries`, `earth_coastline`, and `earth_claim_lines`; full GeoJSON / JSON payloads are stored as artifacts, while the database only keeps source kind, sha256, feature count, license, artifact path, and sample properties so large geometries do not land in a single row.
|
||||
Earth high-precision boundaries no longer use custom-source target schemas. Boundaries are Earth static assets: the Earth Assets settings panel saves local source configuration and triggers PMTiles builds without writing records to `CollectedData`.
|
||||
|
||||
### Configuration Semantics
|
||||
|
||||
@@ -318,7 +318,7 @@ Important fields:
|
||||
- `auth_type`: `none`, `bearer`, `api_key`, or `basic`.
|
||||
- `headers`: static request headers.
|
||||
- `auth_config`: token, API key, or basic username/password; API keys can be sent by header or query.
|
||||
- `config.target_schema`: for example `vessel_ais` or `earth_boundary_source`.
|
||||
- `config.target_schema`: for example `vessel_ais`, `geo_points`, or `generic_records`.
|
||||
- `config.delivery_mode`: REST defaults to `polling`; WebSocket defaults to `realtime_stream`.
|
||||
- `config.merge_target_source`: records which built-in source this custom source supplements, such as `barentswatch_vessels`.
|
||||
|
||||
|
||||
@@ -216,6 +216,8 @@ The real satellite altitude preference is persisted by `controls.js`, while the
|
||||
|
||||
SGP4 propagation returns an inertial-frame position, so it must not be drawn directly as Earth-fixed longitude / latitude. `satellites.js` uses `gstime` to convert ECI/TEME positions to ECF, then maps that result into the same Three.js axes as `latLonToVector3()`. Satellite dots and short trails use Earth-fixed coordinates for each sample time, representing the object's current position relative to the globe surface. The locked predicted orbit uses the `gstime` from the lock moment for the whole future orbit, projecting the inertial orbit plane onto the current globe pose; that keeps the line closed and keeps the visual orbit inclination aligned with the details card. Fallback predicted orbits must also use a real RAAN + inclination orbital-plane formula, not treat inclination as a constant latitude.
|
||||
|
||||
Boundary precision is stored separately by `country-boundaries.js` under `planet.earth.boundaries.highPrecisionEnabled`. When high precision is off, Earth keeps using the bundled low-precision `countries-admin0.min.geojson` fallback even if high-precision manifest/PMTiles files exist locally. When high precision is on but the artifact is missing, the Earth toolbar settings call `/api/v1/earth/boundaries/build` and poll progress. After success, `reloadCountryBoundaries()` hot-swaps the boundary layer without refreshing the page. Boundary hover is independent from interactable hover: a country polygon remains highlighted whenever the surface coordinate is inside it, while the tooltip can still prioritize a satellite, vessel, BGP marker, or other interactable.
|
||||
|
||||
## Current Terrain Pipeline
|
||||
|
||||
1. `terrain.js` creates a sphere geometry with enough segments
|
||||
|
||||
@@ -78,9 +78,10 @@ The land/ocean base is an Earth base-map asset and preloads at startup; the "Bor
|
||||
|
||||
| Name | Variable | Current Value | Location / Notes |
|
||||
| --- | --- | --- | --- |
|
||||
| Boundary tile manifest | `COUNTRY_BOUNDARY_CONFIG.tileManifestPath` | `"/earth/data/boundaries/v1/manifest.json"` | Required production PMTiles manifest; missing manifest is an error |
|
||||
| Boundary tile provider | `COUNTRY_BOUNDARY_CONFIG.tileProvider` | `"pmtiles-mvt"` | Only PMTiles/MVT is accepted for country boundaries |
|
||||
| Boundary tile manifest | `COUNTRY_BOUNDARY_CONFIG.tileManifestPath` | `"/earth/data/boundaries/v1/manifest.json"` | High-precision PMTiles manifest; missing manifest uses the low-precision fallback |
|
||||
| Boundary tile provider | `COUNTRY_BOUNDARY_CONFIG.tileProvider` | `"auto"` | Prefer high-precision PMTiles/MVT, then fall back to legacy GeoJSON |
|
||||
| PMTiles artifact path | `COUNTRY_BOUNDARY_CONFIG.pmtilesPath` | `"/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles"` | Production single-file PMTiles/MVT artifact |
|
||||
| Low-precision fallback | `COUNTRY_BOUNDARY_CONFIG.legacyFallbackPath` | `"/earth/data/countries-admin0.min.geojson"` | Default land/ocean base and hover data when no high-precision boundary asset has been built locally |
|
||||
| MVT layer names | `COUNTRY_BOUNDARY_CONFIG.mvtLayerNames` | `boundary_admin0 / boundary_disputed_internal / coastline / claim_line` | Fixed layer names decoded by the PMTiles provider |
|
||||
| Boundary tile base path | `COUNTRY_BOUNDARY_CONFIG.tileBasePath` | `"/earth/data/boundaries/v1/"` | PMTiles manifest base path |
|
||||
| Boundary tile zoom thresholds | `COUNTRY_BOUNDARY_CONFIG.tileZoomThresholds` | `1.6 -> z5`, `2.8 -> z6`, `3.4 -> z7`, `4.0 -> z8`, `4.6 -> z9`, `5.2 -> z10` | Production PMTiles zoom selection |
|
||||
|
||||
@@ -103,7 +103,7 @@ Goals:
|
||||
|
||||
## Collector Configuration
|
||||
|
||||
`news_live_streams` does not need a separate new page; it reuses Collector Settings under `/settings`:
|
||||
`news_live_streams` does not need a separate new page; it reuses Collectors under `/collection-management`:
|
||||
|
||||
- `endpoint`
|
||||
- Channel directory JSON API URL
|
||||
|
||||
@@ -294,7 +294,7 @@ If only AI Provider is unhealthy, restart just that service:
|
||||
|
||||
### Connectivity validation passes, but collection cannot read credentials. Why?
|
||||
|
||||
Connectivity validation can read saved console settings, environment variables, and some credentials from `~/.zshrc`. For actual collection, prefer saving credentials in Settings -> Collector Settings, especially for AISStream's long-lived WebSocket collector.
|
||||
Connectivity validation can read saved console settings, environment variables, and some credentials from `~/.zshrc`. For actual collection, prefer saving credentials in Collection Management -> Collectors, especially for AISStream's long-lived WebSocket collector.
|
||||
|
||||
If `AISSTREAM_API_KEY` only lives in `~/.zshrc`, confirm the backend process actually inherited it. Otherwise validation may pass while the collector runtime has no key.
|
||||
|
||||
@@ -308,7 +308,7 @@ export BARENTSWATCH_CLIENT_ID="..."
|
||||
export BARENTSWATCH_CLIENT_SECRET="..."
|
||||
```
|
||||
|
||||
For stable operation, save credentials in Collector Settings so connectivity validation, collection jobs, and Earth realtime aggregation use the same configuration.
|
||||
For stable operation, save credentials in Collectors so connectivity validation, collection jobs, and Earth realtime aggregation use the same configuration.
|
||||
|
||||
## Docs / Permissions
|
||||
|
||||
@@ -323,6 +323,17 @@ Docs visibility is controlled by Gatekeeper groups:
|
||||
|
||||
## Earth Common Tasks
|
||||
|
||||
### Why does Earth say the boundary endpoint is not configured, or only show low precision boundaries?
|
||||
|
||||
Country boundaries have moved out of the collector system. They are no longer generated by datasource collection tasks. The low-precision boundary file is bundled with the frontend and is the expected fallback when no local high-precision PMTiles artifact exists.
|
||||
|
||||
There are two high-precision entry points:
|
||||
|
||||
- Earth page settings gear -> Boundary Precision: switching to High Precision starts the first background download/build, shows percentage progress, and applies the result automatically.
|
||||
- Console `Operations and Configuration -> Earth Content -> Boundary Precision`: use this to inspect provider, manifest, PMTiles, fallback state, edit source JSON, or rebuild manually.
|
||||
|
||||
If the UI says the update source is incomplete, save the source configuration from `Earth Content -> Boundary Precision`. The private local config is written to `config/earth-boundary-sources.local.json`; do not commit it. Falling back to low precision is normal when no high-precision artifact has been built.
|
||||
|
||||
### Why did collecting a location candidate not write anything?
|
||||
|
||||
Collecting and saving are two separate actions. Candidates can be previewed on Earth first. A candidate is written only after clicking Save or using the unresolved list's one-click adopt flow.
|
||||
|
||||
@@ -33,6 +33,8 @@ Current admin-related routes:
|
||||
- `/alerts/situational`
|
||||
- `/bgp`
|
||||
- `/ai`
|
||||
- `/earth-content`
|
||||
- `/collection-management`
|
||||
- `/settings`
|
||||
|
||||
`/earth` is a standalone display page and is not part of the console shell.
|
||||
@@ -277,6 +279,42 @@ Constraints:
|
||||
- Do not let tables blow out the full page
|
||||
- New table areas should reuse `TableScrollRegion` / `ScrollbarOverlay`
|
||||
|
||||
### Datasource Directory Page
|
||||
|
||||
[DataSources.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/DataSources/DataSources.tsx) is the datasource directory and collection operation page. It should not grow back into a configuration editor.
|
||||
|
||||
Current page boundary:
|
||||
|
||||
- Built-in and custom sources are merged as `UnifiedDataSource`.
|
||||
- The list shows type, state, last run, collection progress, and actions.
|
||||
- Clicking a name opens a read-only drawer.
|
||||
- Endpoint, headers, and config are displayed here, not edited.
|
||||
- Credential-bearing collectors point users to `Collection Management -> Collectors`.
|
||||
|
||||
Keep this boundary: do not put custom datasource editing, built-in endpoint overrides, or credential forms back into `/datasources`. Those configuration entry points live at `/collection-management?tab=collector_credentials`.
|
||||
|
||||
### Collectors Page
|
||||
|
||||
[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx) has three route modes: `/settings` for System Settings, `/earth-content` for Earth Content, and `/collection-management` for Collection Management. The `collector_credentials` tab is shown as `Collectors` under `/collection-management`.
|
||||
|
||||
Current boundary:
|
||||
|
||||
- The dropdown selects built-in collectors.
|
||||
- The plug icon beside the dropdown runs the health check.
|
||||
- Credential-bearing collectors place credential forms above base config.
|
||||
- Free collectors show endpoint, default endpoint, headers, timeout, and retry.
|
||||
- BarentsWatch AIS keeps its dedicated credential form.
|
||||
|
||||
### Earth Content Page
|
||||
|
||||
`/earth-content` reuses the same single-screen tab container from [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx), but its ownership is separate from System Settings:
|
||||
|
||||
- `TV Livestream` owns the Earth media-panel source configuration.
|
||||
- `Boundary Precision` owns the Earth static boundary asset state: provider, low-precision fallback, high-precision manifest/PMTiles, source JSON, and build action.
|
||||
- `Base Map`, `Layer Resources`, `3D Assets`, and `News Anchor Strategy` are placeholders only. They show module status and do not invent fake APIs or fake data.
|
||||
|
||||
Do not add Earth experience resources or collection-lifecycle tabs back into `/settings`; collection belongs to `/collection-management`, and Earth display resources belong to `/earth-content`.
|
||||
|
||||
### 3. Complex Workspace Pages
|
||||
|
||||
Examples:
|
||||
|
||||
@@ -76,15 +76,17 @@ The console at `http://localhost:3000/admin` is built with React + Ant Design. T
|
||||
| BGP Alerts | `/alerts/bgp` | BGP-related alerts |
|
||||
| Situational Alerts | `/alerts/situational` | Situational analysis alerts |
|
||||
| AI | `/ai` | Model providers, tools, testbench |
|
||||
| Earth Content | `/earth-content` | TV livestreams, boundary precision, base-map and layer-resource entry points |
|
||||
| Collection Management | `/collection-management` | Collectors, scheduling, collection history entry points |
|
||||
| Logs | `/logs` | Usually visible only to super admin |
|
||||
| Users | `/users` | Create/delete users, change roles/groups |
|
||||
| Settings | `/settings` | System, SMTP, TV, collectors |
|
||||
| System Settings | `/settings` | Display, notification, security, SMTP |
|
||||
|
||||
Menu items hide automatically when you lack permission. If a menu is missing, check your role and Gatekeeper groups.
|
||||
|
||||
## Configure Data Collectors
|
||||
|
||||
`/settings?tab=collector_credentials` is the "Collector Settings" page. It manages connection configuration for every collector, not just credentials.
|
||||
`/collection-management?tab=collector_credentials` is the "Collectors" page. It manages connection configuration for every collector, not just credentials. Legacy `/settings?tab=collector_credentials` redirects here; the datasource directory remains at `/datasources`.
|
||||
|
||||
Steps:
|
||||
|
||||
@@ -125,7 +127,7 @@ The default guide follows the BarentsWatch official tutorial and reminds you to
|
||||
|
||||
Steps:
|
||||
|
||||
1. Open `/settings?tab=collector_credentials` and select `AISStream Realtime Vessels : aisstream_vessels`
|
||||
1. Open `/collection-management?tab=collector_credentials` and select `AISStream Realtime Vessels : aisstream_vessels`
|
||||
2. Fill the AISStream API Key
|
||||
3. Keep the default endpoint `wss://stream.aisstream.io/v0/stream`
|
||||
4. Click the plug icon to test; confirm it reports `Reachable`
|
||||
@@ -139,10 +141,11 @@ Steps:
|
||||
|
||||
## Configure AI Credentials
|
||||
|
||||
`/ai?tab=providers` is the AI management entry. Two key sub-tabs:
|
||||
`/ai?tab=providers` is the AI management entry. Three key sub-tabs:
|
||||
|
||||
- `Model Providers`: default LLM provider, model, base URL, API key, local `aiprovider` proxy, connection test
|
||||
- `Tools`: a dropdown for specific tools — currently WebSearch and OCR
|
||||
- `Prompts`: a task dropdown for news localization, alert analysis, BGP briefs, and other LLM tasks. Operators can edit the prompt or reset it to the default
|
||||
|
||||
### Model Providers
|
||||
|
||||
@@ -163,6 +166,10 @@ The plug icon at the end of the Base URL input runs a connection test. A passing
|
||||
- **WebSearch**: provider, API key, base URL, max results, timeout, advanced provider parameters. While disabled, all fields except the enable switch are greyed out
|
||||
- **OCR**: provider, base URL, API key, model/engine, recognition languages, timeout, max file size, output format
|
||||
|
||||
### Prompts
|
||||
|
||||
After selecting a task, the page shows the effective prompt, whether it is customized, the shipped default version, and a reset button. Saving affects only that task. Reset restores the default prompt from the current release package. Business facts, context, and output schemas are still assembled by the backend for each task.
|
||||
|
||||
The legacy link `/settings?tab=ai` redirects to `/ai?tab=providers`.
|
||||
|
||||
## System Settings
|
||||
@@ -173,8 +180,26 @@ The legacy link `/settings?tab=ai` redirects to `/ai?tab=providers`.
|
||||
- **Notifications**: alert email switch, recipient, critical/warning/daily summary
|
||||
- **Security**: session timeout, max login attempts, password policy
|
||||
- **SMTP Email**: outgoing email used by registration and password reset (visible to `admin` / `super_admin` only)
|
||||
- **TV Livestream**: TV source management
|
||||
- **AI / WebSearch / OCR**: see above
|
||||
|
||||
TV livestreams and boundary precision moved to `/earth-content`; collectors and scheduling moved to `/collection-management`; AI Provider / WebSearch / OCR live at `/ai`.
|
||||
|
||||
### Earth Content
|
||||
|
||||
`/earth-content` is under the console's Operations and Configuration group and owns resources used by the Earth frontend:
|
||||
|
||||
- **TV Livestream**: manages sources shown in the Earth media panel.
|
||||
- **Boundary Precision**: shows the current provider, low-precision fallback, high-precision PMTiles/manifest status, local source JSON, and manual build action.
|
||||
- **Base Map**, **Layer Resources**, **3D Assets**, and **News Anchor Strategy**: placeholder tabs for future configuration. They do not display fake data.
|
||||
|
||||
The Earth page settings gear also includes Boundary Precision. Switching to High Precision starts a local background download/build, like a game update package, when no high-precision asset exists yet. Progress is shown as a percentage, and the result applies automatically after success without a page reload. Switching back to Low Precision only changes the local display preference.
|
||||
|
||||
### Collection Management
|
||||
|
||||
`/collection-management` is also under Operations and Configuration and owns the collection lifecycle:
|
||||
|
||||
- **Collectors**: endpoint, headers, credentials, timeout, retry, and connection checks.
|
||||
- **Collection Scheduling**: the existing scheduling configuration.
|
||||
- **Collection History / Snapshots**: a placeholder for future collection task, snapshot, and collected-data browsing.
|
||||
|
||||
### SMTP Email Settings
|
||||
|
||||
@@ -204,7 +229,7 @@ To let a regular user read developer or operations docs, add `docs_developer` or
|
||||
|
||||
## Data Exploration
|
||||
|
||||
- `/datasources`: source directory. The `Collection Tasks` tab is for one-shot, scheduled, and finite collectors; it can be filtered by product domain, layer/module, enabled state, last run status, whether collected records exist, and search text. Selecting rows triggers only those sources; with no selected rows, `Collect current filter` triggers the filtered scope. The `Realtime Streams` tab is for AISStream / WebSocket long connections and shows connection health, stored totals, time-window counters, and Start / Stop / Reconnect actions. Clicking a name opens an info drawer showing endpoint, headers, base config, and built-in flag; endpoint/credentials editing happens at `/settings -> Collector Settings`. The `Collecting N` tag under the overall progress can be clicked to expand the current collection task list
|
||||
- `/datasources`: source directory. The `Collection Tasks` tab is for one-shot, scheduled, and finite collectors; it can be filtered by product domain, layer/module, enabled state, last run status, whether collected records exist, and search text. Selecting rows triggers only those sources; with no selected rows, `Collect current filter` triggers the filtered scope. The `Realtime Streams` tab is for AISStream / WebSocket long connections and shows connection health, stored totals, time-window counters, and Start / Stop / Reconnect actions. Clicking a name opens an info drawer showing endpoint, headers, base config, and built-in flag; endpoint/credentials editing happens at `/collection-management -> Collectors`. The `Collecting N` tag under the overall progress can be clicked to expand the current collection task list
|
||||
- `/data`: collected data table — used to verify "did data arrive", "is the freshness right", "does a source emit valid records"
|
||||
- `/bgp`: BGP detail page with list + detail + analysis; complements the BGP layer on Earth
|
||||
- `/alerts/system`, `/alerts/bgp`, `/alerts/situational`: system, BGP, and situational alerts
|
||||
|
||||
@@ -243,13 +243,12 @@ uv run pytest backend/tests/test_otp_service.py
|
||||
|
||||
## Earth Boundary PMTiles Operations
|
||||
|
||||
1. In Collector Settings, configure endpoints, headers/auth, `config.target_schema=earth_boundary_source`, license, and mapping for `Earth Admin-0 Boundaries`, `Earth Coastline`, and `Earth Claim Lines`.
|
||||
2. In the data-source console, collect those three sources first. Each successful source writes the full artifact to `data/earth-boundary-sources/<collector>/<sha256>.*` and stores sha256, feature count, artifact path, and sample properties.
|
||||
3. After all three sources succeed, collect `Earth PMTiles Builder`. If any source is missing, it fails as "not ready" and does not update Earth boundaries.
|
||||
4. The builder requires `tippecanoe` and `pmtiles` on PATH. Missing tools fail the task with a clear message.
|
||||
5. A successful production build outputs `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles` and its manifest.
|
||||
6. After deployment, open Earth, enable "Border Lines", and inspect China's southeast coast, Taiwan, Hainan, the South China Sea, Zangnan, Kosovo, and Gaza for hover behavior and boundary policy.
|
||||
7. If PMTiles loading fails, Earth reports a boundary-layer error and does not draw legacy low-precision borders. Troubleshoot in this order: browser Network range requests for PMTiles, manifest `tileProvider: "pmtiles-mvt"`, Nginx static serving for `.pmtiles`, and artifact path / sha256 consistency with the manifest.
|
||||
1. In the console, open `Operations and Configuration -> Earth Content -> Boundary Precision` to save boundary source configuration. The local config is written to `config/earth-boundary-sources.local.json`; do not commit it.
|
||||
2. Click "Build high precision boundaries", or switch the Earth toolbar settings gear to High Precision for the first build. The backend downloads the three source packages to `data/earth-boundary-sources/`, writes the source manifest, and invokes the PMTiles build script.
|
||||
3. The builder requires `tippecanoe` and `pmtiles` on PATH. Missing tools return a clear API error and do not write data-source collection records.
|
||||
4. A successful production build outputs `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles` and its manifest.
|
||||
5. After deployment, open Earth, enable "Border Lines", and inspect China's southeast coast, Taiwan, Hainan, the South China Sea, Zangnan, Kosovo, and Gaza for hover behavior and boundary policy.
|
||||
6. If no high-precision manifest/PMTiles exists locally, Earth uses the bundled `frontend/public/earth/data/countries-admin0.min.geojson` fallback. If high-precision assets exist but tile requests fail, troubleshoot PMTiles range requests, manifest provider, Nginx `.pmtiles` static serving, and sha256 consistency.
|
||||
|
||||
## Related Docs
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@ Open the URL your administrator gave you, e.g. `http://planet.example.com`. A lo
|
||||
Entry points are split in two:
|
||||
|
||||
- Public: `/earth` (3D situational view), `/docs` (public documentation)
|
||||
- Login required: `/admin` (console), `/ai` (AI), `/settings` (system configuration)
|
||||
- Login required: `/admin` (console), `/ai` (AI), `/settings` (system settings), `/earth-content` (Earth content), `/collection-management` (collection management)
|
||||
|
||||
## 2. Register
|
||||
|
||||
@@ -32,7 +32,7 @@ The default role is `viewer`: you can sign in but only see public pages. For col
|
||||
|
||||
After landing on the `/admin` dashboard, here's a recommended walk-through:
|
||||
|
||||
1. `/settings?tab=collector_credentials`: pick a collector and click the plug icon to test connectivity. Free collectors (e.g. open BGP) usually work right away; credential-bearing ones like `AISStream` or `BarentsWatch` need an API key / client secret first
|
||||
1. `/collection-management?tab=collector_credentials`: pick a collector and click the plug icon to test connectivity. Free collectors (e.g. open BGP) usually work right away; credential-bearing ones like `AISStream` or `BarentsWatch` need an API key / client secret first
|
||||
2. `/ai?tab=providers`: fill an LLM provider (e.g. `minimax` / `openai`), model, base URL, API key, and click the plug at the end of the base URL to test. WebSearch / OCR tools are optional
|
||||
3. `/datasources` or `/data`: check whether collectors have produced data. Use `/datasources -> Collection Tasks` for finite collectors, and `/datasources -> Realtime Streams` for AISStream / WebSocket health and counters
|
||||
4. `/alerts/system`: verify system alerts look right
|
||||
|
||||
@@ -95,9 +95,18 @@ AI 配置页使用的接口：
|
||||
- `POST /api/v1/settings/integrations/ai-provider/connect`
|
||||
- `GET /api/v1/settings/integrations/ai-provider/secrets`
|
||||
- `GET /api/v1/settings/integrations/ai-provider/presets`
|
||||
- `GET /api/v1/settings/ai-prompts`
|
||||
- `PUT /api/v1/settings/ai-prompts/{task_key}`
|
||||
- `POST /api/v1/settings/ai-prompts/{task_key}/reset`
|
||||
|
||||
这些接口都需要用户登录。`secrets` 接口只用于配置页点击显示 key/token 时取回明文，隐藏时前端恢复为脱敏预览。
|
||||
|
||||
`ai-prompts` 接口用于运维配置页的“提示词”Tab。默认提示词来自后端随发布包携带的版本化资源，业务代码只引用稳定 task key；接口只保存运维覆盖值。重置时删除覆盖值并恢复当前发布包中的缺省提示词。
|
||||
|
||||
### 提示词边界
|
||||
|
||||
`aiprovider` 是纯模型适配器，不注入通用业务 system prompt。新闻汉化、告警研判、BGP 简报、位置 factcheck、数据源映射和凭据教程等入口各自通过 task key 解析有效提示词。告警研判 prompt 只会在告警相关 task 中作为 system prompt 传入，不会污染其它 LLM 调用。
|
||||
|
||||
### AI Provider 内部 API
|
||||
|
||||
仅供内部调用的接口：
|
||||
@@ -287,7 +296,6 @@ SERVICE_VERSION=0.1.0
|
||||
AI_PROVIDER_SERVICE_TOKEN=change_me
|
||||
AI_TIMEOUT_SECONDS=60
|
||||
AI_HTTP_RETRY_ATTEMPTS=2
|
||||
AI_ANALYSIS_SYSTEM_PROMPT=你是态势感知分析助手。请基于输入的上下文、观测与约束，输出结构化、克制、可执行的分析。
|
||||
```
|
||||
|
||||
可选 provider 专属 key：
|
||||
|
||||
@@ -87,16 +87,10 @@ async def run(self, db):
|
||||
| Space-Track TLE | satellite_tle | 卫星轨道 TLE 数据 | 依采集器配置 |
|
||||
| BarentsWatch AIS | vessel | 船只位置、航速、航向、MMSI 等 AIS 数据 | 依采集器配置 |
|
||||
| AISStream Vessels | vessel_ais | AIS WebSocket 实时流，写入原始观测层并由聚合接口展示 | 依采集器配置 |
|
||||
| Earth Admin-0 Boundaries | earth_admin0_boundaries | 从配置 endpoint 下载国家级边界源，保存 artifact 并写入 `earth_boundary_source` manifest 记录 | 依采集器配置 |
|
||||
| Earth Coastline | earth_coastline | 从配置 endpoint 下载海岸线源，保存 artifact 并写入 `earth_boundary_source` manifest 记录 | 依采集器配置 |
|
||||
| Earth Claim Lines | earth_claim_lines | 从配置 endpoint 下载主张线源，保存 artifact 并写入 `earth_boundary_source` manifest 记录 | 依采集器配置 |
|
||||
| Earth PMTiles Builder | earth_boundary_tiles | 读取三类 Earth 边界源采集结果并构建 / 登记 PMTiles 产物 | 依采集器配置 |
|
||||
|
||||
AIS 船只类采集器和其它 `CollectedData` 采集器的落库路径不同。BarentsWatch、AISStream 和自定义 `vessel_ais` 源都会进入 AIS 原始观测层，随后由聚合服务合并成 Earth 船只图层使用的 GeoJSON 和详情数据。这样做可以保留来源、传输方式、字段冲突和观测时间，避免某个实时源直接覆盖最终展示表。
|
||||
|
||||
Earth 边界现在拆成三个真实源采集器和一个下游构建器。`earth_admin0_boundaries`、`earth_coastline`、`earth_claim_lines` 都读取后台 Collector Settings 里的 endpoint、headers、auth 和 `config.target_schema=earth_boundary_source`，点击采集时会真实请求 endpoint，保存完整响应到 `data/earth-boundary-sources/<collector>/<sha256>.*`，并在 `CollectedData` 中写入 sha256、feature count、license、artifact path、sample properties 和 mapping 信息。
|
||||
|
||||
`earth_boundary_tiles` 不再代表源数据采集。它只读取上述三类源的最新成功记录；缺任一源时任务失败并显示“未就绪”，不会登记“4 条高精度瓦片”。三类源齐全后，它会调用 `tippecanoe` / `pmtiles` 生成 `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`；本机缺少这些工具时任务失败并说明缺失工具。国界不再有旧低精度兜底。
|
||||
Earth 国界不再属于采集器体系。它是 Earth 静态渲染资产，由控制台 `运维与配置 -> Earth 内容 -> 国界精度` 维护源配置，并由 `/api/v1/earth/boundaries/*` 构建 `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles`。本地没有高精 PMTiles 时，前端会使用仓库内置的低精度 GeoJSON 作为 fallback，不会向 `CollectedData` 写入国界记录。
|
||||
|
||||
TOP500 和 Epoch AI 算力数据的公开源不总是提供可用经纬度。Earth 统一算力中心接口在主地图启动链路中只使用源数据自带坐标或 `compute_center_locations` 维表坐标；缺少坐标的记录会进入 `unresolved`，不会通过本地注册表、国家质心或猜测城市自动渲染。用户手动采集候选时，后端会用源字段调用 ROR 组织注册 API 和 Nominatim/OpenStreetMap 在线搜索；候选经前端保存后写入 `compute_center_locations`，后续地图刷新再从维表渲染。
|
||||
|
||||
@@ -232,7 +226,7 @@ if datasource.last_status == "success":
|
||||
)
|
||||
```
|
||||
|
||||
这个记录用于控制台“采集器设置”中的连接状态判断：如果当前配置和成功采集时的 checksum 一致，就视为已连接，不要求用户再手动点击连接按钮。只有 endpoint、请求头、基础配置或凭证指纹变化时，才需要重新验证。
|
||||
这个记录用于控制台“采集管理 -> 采集器”中的连接状态判断：如果当前配置和成功采集时的 checksum 一致，就视为已连接，不要求用户再手动点击连接按钮。只有 endpoint、请求头、基础配置或凭证指纹变化时，才需要重新验证。
|
||||
|
||||
相关实现见：
|
||||
|
||||
@@ -273,8 +267,8 @@ backend/app/models/
|
||||
|
||||
| 采集器 | credential provider | 凭证来源 |
|
||||
| --- | --- | --- |
|
||||
| `barentswatch_vessels` | `barentswatch` | 控制台采集器设置、环境变量、`~/.zshrc` |
|
||||
| `aisstream_vessels` | `aisstream` | 控制台采集器设置、环境变量、`~/.zshrc`（连接验证可读；正式采集建议保存到采集器设置或注入后端环境） |
|
||||
| `barentswatch_vessels` | `barentswatch` | 控制台“采集管理 -> 采集器”、环境变量、`~/.zshrc` |
|
||||
| `aisstream_vessels` | `aisstream` | 控制台“采集管理 -> 采集器”、环境变量、`~/.zshrc`（连接验证可读；正式采集建议保存到“采集管理 -> 采集器”或注入后端环境） |
|
||||
| `spacetrack_tle` | `spacetrack` | 环境变量、`~/.zshrc` |
|
||||
|
||||
### BarentsWatch AIS
|
||||
@@ -333,7 +327,7 @@ AISStream 使用 `wss://stream.aisstream.io/v0/stream` WebSocket endpoint。默
|
||||
- `reconnecting`：上游断开或网络异常，采集器记录 `AISSourceHealth` 后等待重连。
|
||||
- `stopped` / `cancelled`：任务被测试上限或用户停止。
|
||||
|
||||
AISStream 连接验证会通过 `datasource_connectivity.py` 读取保存的采集器配置、环境变量和 `~/.zshrc` 中的 `AISSTREAM_API_KEY`。正式采集时，最稳妥的方式是把 API Key 保存到“设置 -> 采集器设置 -> AISStream 实时船舶”；如果只放在 `~/.zshrc`，需要确认后端进程实际继承到了该环境变量。
|
||||
AISStream 连接验证会通过 `datasource_connectivity.py` 读取保存的采集器配置、环境变量和 `~/.zshrc` 中的 `AISSTREAM_API_KEY`。正式采集时，最稳妥的方式是把 API Key 保存到“采集管理 -> 采集器 -> AISStream 实时船舶”；如果只放在 `~/.zshrc`，需要确认后端进程实际继承到了该环境变量。
|
||||
|
||||
控制台通过 `/datasources -> 实时流` 管理 AISStream，而不是把它放进普通有限采集任务的进度条。实时流 API 会聚合运行态、健康状态、配置摘要和 raw observation 计数：
|
||||
|
||||
@@ -403,7 +397,7 @@ GET /api/v1/layers/bgp/collectors?bbox=lon_min,lat_min,lon_max,lat_max&zoom=12&l
|
||||
|
||||
## 十、采集器设置与连接验证
|
||||
|
||||
控制台的“采集器设置”页提供所有内置采集器的 endpoint、请求头、超时、重试和凭证配置。连接验证不是只看前端按钮状态，而是由后端计算 checksum：
|
||||
控制台的“采集管理 -> 采集器”页提供所有内置采集器的 endpoint、请求头、超时、重试和凭证配置。连接验证不是只看前端按钮状态，而是由后端计算 checksum：
|
||||
|
||||
- endpoint
|
||||
- auth type
|
||||
|
||||
@@ -8,12 +8,12 @@
|
||||
- 展示所有数据源，包括内置和自定义。
|
||||
- 点击名称只打开信息抽屉。
|
||||
- 负责查看状态、触发采集和查看采集中任务。
|
||||
- `/settings?tab=collector_credentials`
|
||||
- 显示为“采集器设置”。
|
||||
- `/collection-management?tab=collector_credentials`
|
||||
- 显示为“采集器”。
|
||||
- 负责 endpoint、请求头、基础参数和凭证配置。
|
||||
- 所有采集器都提供连接按钮，用于健康检查。
|
||||
|
||||
这样做是为了减少首次使用时的认知分裂：接口地址、请求头、凭证和自定义源配置都属于“采集器设置”，而不是散落在数据源列表和系统设置多个入口里。
|
||||
这样做是为了减少首次使用时的认知分裂：接口地址、请求头、凭证和自定义源配置都属于“采集器”，而不是散落在数据源列表和系统设置多个入口里。
|
||||
|
||||
## 用户侧规则
|
||||
|
||||
@@ -53,7 +53,7 @@
|
||||
|
||||
`data-source-bulk-toolbar__running-pill` 是“采集中”标签的样式入口。它和其他状态标签同排，但通过 hover、箭头和蓝色描边表达可交互性。
|
||||
|
||||
### 采集器设置
|
||||
### 采集器
|
||||
|
||||
文件：
|
||||
|
||||
@@ -61,7 +61,7 @@
|
||||
|
||||
当前行为：
|
||||
|
||||
- `collector_credentials` tab 展示为“采集器设置”。
|
||||
- `collector_credentials` tab 在 `/collection-management` 下展示为“采集器”。
|
||||
- 下拉框列出内置采集器，并支持维护合并到内置数据的自定义补充源。
|
||||
- 下拉框右侧只有一个插头图标按钮，用于健康检查。
|
||||
- 下拉框下方用状态标签展示：
|
||||
@@ -309,7 +309,7 @@ GET /api/v1/vessels/snapshot?bbox=lon_min,lat_min,lon_max,lat_max&zoom=12&limit=
|
||||
|
||||
自定义源现在不是独立的新数据孤岛，而是作为内置数据源的补充源写入目标 schema。当前最完整的目标是 `vessel_ais`：自定义 REST 或 WebSocket 源经过确定性 mapping 后写入 AIS raw observations，再通过 `vessels` WebSocket channel 推送给 Earth。
|
||||
|
||||
Earth 高精度边界使用同一套目标 schema 机制。`earth_boundary_source` 承接 `earth_admin0_boundaries`、`earth_coastline`、`earth_claim_lines` 三类源的映射结果；完整 GeoJSON / JSON 原文保存为 artifact，数据库只保存 source kind、sha256、feature count、license、artifact path 和 sample properties，避免把大型几何塞进单行记录。
|
||||
Earth 高精度边界不再使用自定义源目标 schema。国界是 Earth 静态资产，由控制台 `运维与配置 -> Earth 内容 -> 国界精度` 保存本机源配置并触发 PMTiles 构建，不写入 `CollectedData`。
|
||||
|
||||
### 配置语义
|
||||
|
||||
@@ -320,7 +320,7 @@ Earth 高精度边界使用同一套目标 schema 机制。`earth_boundary_sourc
|
||||
- `auth_type`：`none`、`bearer`、`api_key`、`basic`。
|
||||
- `headers`：静态请求头。
|
||||
- `auth_config`：token、API key、basic 用户名密码，API key 支持 header 或 query。
|
||||
- `config.target_schema`：例如 `vessel_ais` 或 `earth_boundary_source`。
|
||||
- `config.target_schema`：例如 `vessel_ais`、`geo_points` 或 `generic_records`。
|
||||
- `config.delivery_mode`：REST 默认 `polling`，WebSocket 默认 `realtime_stream`。
|
||||
- `config.merge_target_source`：记录该自定义源补充哪个内置数据，例如 `barentswatch_vessels`。
|
||||
|
||||
|
||||
@@ -426,6 +426,7 @@ Earth 设置面板当前由 [controls.js](/home/ray/dev/linkong/planet/frontend/
|
||||
- 图层控制开关：`地形 / 卫星 / 海缆 / BGP`
|
||||
- 卫星显示偏好：`卫星显示风格 / 卫星呼吸闪烁 / 真实卫星高度 / 轨迹显示`
|
||||
- 地表 hover 提示偏好：`国家 / 位置 / 完整`
|
||||
- 国界精度偏好：低精 fallback / 高精 PMTiles
|
||||
- 地形透明度
|
||||
|
||||
也就是说，Earth 设置不是一次性 UI 状态了，而是本地设备级偏好。后续如果再加入新的设置项，应优先接入同一条持久化链，而不是各自散着写 `localStorage`。
|
||||
@@ -436,6 +437,8 @@ Earth 设置面板当前由 [controls.js](/home/ray/dev/linkong/planet/frontend/
|
||||
|
||||
SGP4 传播输出是惯性系位置，不能直接当成 Earth 的经纬度固定坐标使用。`satellites.js` 会用当前时间的 `gstime` 把 ECI/TEME 位置转换到 ECF，再映射到 `latLonToVector3()` 使用的 Three.js 坐标轴。卫星点和短尾迹使用随采样时间变化的地固坐标，表示相对当前地球表面的实际位置；锁定后的预测轨道线使用锁定时刻固定的 `gstime`，把未来一圈惯性轨道投到当前地球姿态上显示，因此会闭合，并且轨道面倾角应与详情卡一致。fallback 预测轨道也必须使用真正的 RAAN + inclination 轨道平面公式，不能把 inclination 当成恒定纬度。
|
||||
|
||||
国界精度偏好独立存储在 `country-boundaries.js` 的 `planet.earth.boundaries.highPrecisionEnabled`。未开启高精时，即使本机已经有高精 manifest/PMTiles，也继续加载低精 `countries-admin0.min.geojson` fallback；开启高精但高精产物缺失时，Earth 工具栏设置会调用 `/api/v1/earth/boundaries/build` 启动后台构建并轮询进度。构建成功后调用 `reloadCountryBoundaries()` 热切换，不再刷新整个页面。国界 hover 与 tooltip 解耦：只要地表坐标落在国界 polygon 内就保持高亮；如果鼠标同时命中卫星、船只、BGP 等 interactable，tooltip 显示 interactable 信息，但国界高亮不应闪烁。
|
||||
|
||||
## 当前地形链路
|
||||
|
||||
真实地形首次启用会慢，原因不只是一个：
|
||||
|
||||
@@ -84,9 +84,10 @@
|
||||
|
||||
| 正式名称 | 变量名 | 当前值 | 使用位置 / 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| 国界瓦片 manifest | `COUNTRY_BOUNDARY_CONFIG.tileManifestPath` | `"/earth/data/boundaries/v1/manifest.json"` | 必需的生产 PMTiles manifest；缺失即报错 |
|
||||
| 国界瓦片 provider | `COUNTRY_BOUNDARY_CONFIG.tileProvider` | `"pmtiles-mvt"` | 国界只接受 PMTiles/MVT |
|
||||
| 国界瓦片 manifest | `COUNTRY_BOUNDARY_CONFIG.tileManifestPath` | `"/earth/data/boundaries/v1/manifest.json"` | 高精 PMTiles manifest；缺失时使用低精度 fallback |
|
||||
| 国界瓦片 provider | `COUNTRY_BOUNDARY_CONFIG.tileProvider` | `"auto"` | 优先高精 PMTiles/MVT，缺失时降级到 legacy GeoJSON |
|
||||
| PMTiles 产物路径 | `COUNTRY_BOUNDARY_CONFIG.pmtilesPath` | `"/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles"` | 生产单文件 PMTiles/MVT artifact |
|
||||
| 低精度 fallback | `COUNTRY_BOUNDARY_CONFIG.legacyFallbackPath` | `"/earth/data/countries-admin0.min.geojson"` | 本地未构建高精国界时的缺省海陆基座和 hover 数据 |
|
||||
| MVT 图层名 | `COUNTRY_BOUNDARY_CONFIG.mvtLayerNames` | `boundary_admin0 / boundary_disputed_internal / coastline / claim_line` | PMTiles provider 解码时读取的固定 layer 名 |
|
||||
| 国界瓦片基础路径 | `COUNTRY_BOUNDARY_CONFIG.tileBasePath` | `"/earth/data/boundaries/v1/"` | PMTiles manifest 基础路径 |
|
||||
| 国界瓦片缩放阈值 | `COUNTRY_BOUNDARY_CONFIG.tileZoomThresholds` | `1.6 -> z5`, `2.8 -> z6`, `3.4 -> z7`, `4.0 -> z8`, `4.6 -> z9`, `5.2 -> z10` | 生产 PMTiles zoom 选择 |
|
||||
|
||||
@@ -103,7 +103,7 @@
|
||||
|
||||
## 采集器配置方式
|
||||
|
||||
`news_live_streams` 不需要单独新页面，直接复用控制台 `/settings` 的“采集器设置”：
|
||||
`news_live_streams` 不需要单独新页面，直接复用控制台 `/collection-management` 的“采集器”配置：
|
||||
|
||||
- `endpoint`
|
||||
- 频道目录 JSON API 地址
|
||||
|
||||
@@ -296,7 +296,7 @@ USB 摄像头透传到 WSL 属于高级路径；脚本不会默认把无摄像
|
||||
|
||||
### 采集器连接验证通过，但正式采集拿不到凭证怎么办？
|
||||
|
||||
连接验证会读取控制台保存配置、环境变量和部分 `~/.zshrc` 凭证。正式采集更推荐把凭证保存到“设置 -> 采集器设置”，尤其是 AISStream 这类长连接 collector。
|
||||
连接验证会读取控制台保存配置、环境变量和部分 `~/.zshrc` 凭证。正式采集更推荐把凭证保存到“采集管理 -> 采集器”，尤其是 AISStream 这类长连接 collector。
|
||||
|
||||
如果只把 `AISSTREAM_API_KEY` 放在 `~/.zshrc`，需要确认后端进程实际继承了该变量。否则可能出现连接验证可用，但 collector 运行时没有 key 的情况。
|
||||
|
||||
@@ -310,7 +310,7 @@ export BARENTSWATCH_CLIENT_ID="..."
|
||||
export BARENTSWATCH_CLIENT_SECRET="..."
|
||||
```
|
||||
|
||||
稳定运行时，优先在控制台采集器设置中保存凭证，保证连接验证、采集任务和 Earth 实时聚合使用同一份配置。
|
||||
稳定运行时，优先在控制台“采集管理 -> 采集器”中保存凭证，保证连接验证、采集任务和 Earth 实时聚合使用同一份配置。
|
||||
|
||||
## Docs / 权限
|
||||
|
||||
@@ -325,6 +325,17 @@ Docs 按 Gatekeeper 权限组控制可见性：
|
||||
|
||||
## Earth 常见操作
|
||||
|
||||
### 为什么国界提示“未配置 endpoint”或只能看到低精度？
|
||||
|
||||
国界已经从采集器体系移出，不再通过数据源采集任务生成。低精度国界是随前端打包的兜底资产，本地未构建高精 PMTiles 时会自动使用它。
|
||||
|
||||
要启用高精国界，有两条入口：
|
||||
|
||||
- Earth 页面齿轮设置里的“国界精度”：切到“高精”会启动首次后台下载/构建，并显示百分比，完成后自动应用。
|
||||
- 控制台 `运维与配置 -> Earth 内容 -> 国界精度`：适合查看 provider、manifest、PMTiles、fallback 状态，编辑源配置 JSON，或手动重建。
|
||||
|
||||
如果看到“更新源未配置完整”，先到 `Earth 内容 -> 国界精度` 保存源配置；本机私有配置写入 `config/earth-boundary-sources.local.json`，不要提交到仓库。没有高精产物时，使用低精 fallback 是正常行为。
|
||||
|
||||
### Earth 位置候选采集后没有写入怎么办？
|
||||
|
||||
“采集候选”和“保存候选”是两步。候选可以先在 Earth 上预览，只有点击保存或使用待定位列表中的“一键采用”后，才会写入维表并刷新图层。
|
||||
|
||||
@@ -33,6 +33,8 @@
|
||||
- `/alerts/situational`
|
||||
- `/bgp`
|
||||
- `/ai`
|
||||
- `/earth-content`
|
||||
- `/collection-management`
|
||||
- `/settings`
|
||||
|
||||
`/earth` 是独立展示页，不属于控制台骨架。
|
||||
@@ -288,9 +290,9 @@
|
||||
- 点击名称打开只读抽屉。
|
||||
- 抽屉中明确显示“内置数据源”或“自定义数据源”。
|
||||
- endpoint、headers、config 只展示，不在这里编辑。
|
||||
- 需要凭证的采集器提示用户到“设置 -> 采集器设置”维护。
|
||||
- 需要凭证的采集器提示用户到“采集管理 -> 采集器”维护。
|
||||
|
||||
这个边界很重要：后续不要把自定义数据源编辑、内置 endpoint 覆盖或凭证表单再塞回 `/datasources`。这些配置入口统一放在 `/settings?tab=collector_credentials`。
|
||||
这个边界很重要：后续不要把自定义数据源编辑、内置 endpoint 覆盖或凭证表单再塞回 `/datasources`。这些配置入口统一放在 `/collection-management?tab=collector_credentials`。
|
||||
|
||||
页面顶部的总进度区域新增 `采集中 N` 标签：
|
||||
|
||||
@@ -303,7 +305,7 @@
|
||||
|
||||
### 采集器设置页
|
||||
|
||||
[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx) 中的 `collector_credentials` tab 当前显示为“采集器设置”。
|
||||
[Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx) 会按路由进入三种模式：`/settings` 是系统设置，`/earth-content` 是 Earth 内容，`/collection-management` 是采集管理。`collector_credentials` tab 当前在 `/collection-management` 下显示为“采集器”。
|
||||
|
||||
当前页面边界：
|
||||
|
||||
@@ -334,6 +336,16 @@
|
||||
|
||||
- [数据源、采集器设置与连接验证](/home/ray/dev/linkong/planet/docs/technical/zh/datasource-collector-settings-connectivity.md)
|
||||
|
||||
### Earth 内容页
|
||||
|
||||
`/earth-content` 复用 [Settings.tsx](/home/ray/dev/linkong/planet/frontend/src/pages/Settings/Settings.tsx) 的单屏 tab 容器，但页面责任与系统设置分离：
|
||||
|
||||
- `电视直播` 迁移原直播源配置，继续管理 Earth 媒体面板内容源。
|
||||
- `国界精度` 管理 Earth 静态国界资产：provider 状态、低精 fallback、高精 manifest/PMTiles、源配置 JSON 和构建动作。
|
||||
- `地球底图`、`图层资源`、`三维素材`、`新闻锚点策略` 是占位页，只显示模块待接入，不造假接口或假数据。
|
||||
|
||||
系统级 `/settings` 不应再新增 Earth 体验资源或采集生命周期 tab；采集相关入口属于 `/collection-management`，Earth 展示资源属于 `/earth-content`。
|
||||
|
||||
### 3. 复杂工作区页面
|
||||
|
||||
例如：
|
||||
|
||||
@@ -76,15 +76,17 @@
|
||||
| BGP 告警 | `/alerts/bgp` | BGP 相关告警 |
|
||||
| 态势告警 | `/alerts/situational` | 态势研判告警 |
|
||||
| AI | `/ai` | 模型供应商、工具、测试台 |
|
||||
| Earth 内容 | `/earth-content` | 电视直播、国界精度、底图和图层资源入口 |
|
||||
| 采集管理 | `/collection-management` | 采集器、采集调度、采集历史入口 |
|
||||
| 系统日志 | `/logs` | 通常仅 super admin 可见 |
|
||||
| 用户管理 | `/users` | 创建/删除/改角色/调权限组 |
|
||||
| 系统配置 | `/settings` | 系统、SMTP、TV、采集器设置 |
|
||||
| 系统设置 | `/settings` | 系统显示、通知、安全、SMTP |
|
||||
|
||||
权限不足时菜单项会自动隐藏。如果发现某个菜单看不到，先确认自己的角色和 Gatekeeper 权限组。
|
||||
|
||||
## 配置数据采集器
|
||||
|
||||
`/settings?tab=collector_credentials` 是"采集器设置"页。这里统一维护所有采集器的连接配置，不仅是凭证。
|
||||
`/collection-management?tab=collector_credentials` 是"采集器"页。这里统一维护所有采集器的连接配置，不仅是凭证。旧链接 `/settings?tab=collector_credentials` 会自动跳转到这个入口；数据源目录仍保留在 `/datasources`。
|
||||
|
||||
操作步骤：
|
||||
|
||||
@@ -128,7 +130,7 @@
|
||||
|
||||
操作步骤：
|
||||
|
||||
1. `/settings?tab=collector_credentials` 选择 `AISStream 实时船舶 : aisstream_vessels`
|
||||
1. `/collection-management?tab=collector_credentials` 选择 `AISStream 实时船舶 : aisstream_vessels`
|
||||
2. 在 `AISStream 凭证` 填入 API Key
|
||||
3. Endpoint 保持默认 `wss://stream.aisstream.io/v0/stream`
|
||||
4. 点击插头图标进行连接测试，确认显示 `可用`
|
||||
@@ -142,10 +144,11 @@
|
||||
|
||||
## 配置 AI 凭证
|
||||
|
||||
`/ai?tab=providers` 是 AI 模型管理入口。包含两个核心子 tab：
|
||||
`/ai?tab=providers` 是 AI 模型管理入口。包含三个核心子 tab：
|
||||
|
||||
- `模型供应商`：默认 LLM provider、模型、Base URL、API Key、本地 `aiprovider` 代理和连接测试
|
||||
- `工具`：通过下拉菜单选择具体工具，当前支持 WebSearch 和 OCR
|
||||
- `提示词`：通过功能入口下拉菜单选择新闻汉化、告警研判、BGP 简报等 LLM 任务，手动调整提示词或重置为缺省
|
||||
|
||||
### 模型供应商
|
||||
|
||||
@@ -166,6 +169,10 @@ Base URL 输入框尾端的插头图标会触发连接测试。测试通过会
|
||||
- **WebSearch**：provider、API Key、Base URL、最大结果数、超时、高级 provider 参数。未启用时除"启用"开关外其它配置项和连接测试都会置灰
|
||||
- **OCR**：provider、Base URL、API Key、模型/engine、识别语言、超时、最大文件大小、输出格式
|
||||
|
||||
### 提示词
|
||||
|
||||
选择功能入口后，页面会显示当前提示词、是否已自定义、缺省版本和重置按钮。保存只影响该功能入口；重置会恢复当前发布包中的缺省提示词。业务事实、上下文和输出 schema 仍由后端按功能入口自动传入。
|
||||
|
||||
旧链接 `/settings?tab=ai` 会跳到 `/ai?tab=providers`。
|
||||
|
||||
## 系统设置
|
||||
@@ -176,8 +183,26 @@ Base URL 输入框尾端的插头图标会触发连接测试。测试通过会
|
||||
- **通知策略**：告警邮件开关、收件邮箱、严重/警告/每日摘要通知
|
||||
- **安全策略**：会话超时、最大登录尝试、密码策略
|
||||
- **SMTP 邮件**：注册和找回密码所需的发件配置（仅 `admin` / `super_admin` 可见）
|
||||
- **电视直播**：电视直播源管理
|
||||
- **AI / WebSearch / OCR**：见上节
|
||||
|
||||
电视直播和国界精度已经移到 `/earth-content`，采集器和采集调度已经移到 `/collection-management`，AI Provider / WebSearch / OCR 在 `/ai`。
|
||||
|
||||
### Earth 内容
|
||||
|
||||
`/earth-content` 位于控制台“运维与配置”下，面向 Earth 前端体验资源：
|
||||
|
||||
- **电视直播**：维护 Earth 媒体面板里的直播源。
|
||||
- **国界精度**：查看当前国界 provider、低精 fallback、高精 PMTiles/manifest 状态，编辑本机源配置并手动构建。
|
||||
- **地球底图**、**图层资源**、**三维素材**、**新闻锚点策略**：目前是待接入占位页，不展示假数据。
|
||||
|
||||
Earth 页面工具栏齿轮中也有“国界精度”。切到“高精”时，如果本机尚未构建高精资产，会像游戏更新包一样启动后台下载/构建并显示百分比；构建成功后自动应用，无需刷新。切回“低精”只切换本机显示偏好，不重新下载。
|
||||
|
||||
### 采集管理
|
||||
|
||||
`/collection-management` 位于控制台“运维与配置”下，面向采集生命周期：
|
||||
|
||||
- **采集器**：维护 endpoint、请求头、凭证、timeout、retry，并运行连接检查。
|
||||
- **采集调度**：维护原有调度相关设置。
|
||||
- **采集历史 / 快照**：当前是待接入占位页，后续承载 collection task、snapshot、collected data 浏览能力。
|
||||
|
||||
### SMTP 邮件设置
|
||||
|
||||
@@ -207,7 +232,7 @@ Base URL 输入框尾端的插头图标会触发连接测试。测试通过会
|
||||
|
||||
## 数据探索
|
||||
|
||||
- `/datasources`：数据源目录。`采集任务` tab 面向一次性/定时采集器，可以按产品域、层级、启用状态、最近执行状态、是否已有采集数据和关键词筛选；勾选多行后可批量采集选中项，未勾选时“一键采集”触发当前筛选范围。`实时流` tab 面向 AISStream / WebSocket 长连接，展示连接健康、累计入库、时间窗统计和启动 / 停止 / 重连操作。点击名称打开信息抽屉查看 endpoint、请求头、基础配置和是否内置；接口、凭证、请求头的编辑统一在 `/settings` 的"采集器设置"。总体进度下方的 `采集中 N` 标签可点击，展开当前采集任务列表
|
||||
- `/datasources`：数据源目录。`采集任务` tab 面向一次性/定时采集器，可以按产品域、层级、启用状态、最近执行状态、是否已有采集数据和关键词筛选；勾选多行后可批量采集选中项，未勾选时“一键采集”触发当前筛选范围。`实时流` tab 面向 AISStream / WebSocket 长连接，展示连接健康、累计入库、时间窗统计和启动 / 停止 / 重连操作。点击名称打开信息抽屉查看 endpoint、请求头、基础配置和是否内置；接口、凭证、请求头的编辑统一在 `/collection-management` 的"采集器"。总体进度下方的 `采集中 N` 标签可点击，展开当前采集任务列表
|
||||
- `/data`：采集后数据表，适合排查"数据是否已经进入系统"、"更新时间是否符合预期"、"某个数据源是否产出有效记录"
|
||||
- `/bgp`：BGP 专题页面，列表 + 详情 + 研判，与 Earth 的 BGP 图层互补
|
||||
- `/alerts/system`、`/alerts/bgp`、`/alerts/situational`：系统、BGP、态势告警
|
||||
|
||||
@@ -243,13 +243,12 @@ uv run pytest backend/tests/test_otp_service.py
|
||||
|
||||
## Earth 国界 PMTiles 操作步骤
|
||||
|
||||
1. 在后台 Collector Settings 中分别配置 `Earth Admin-0 国界源`、`Earth 海岸线源`、`Earth 主张线源` 的 endpoint、headers/auth、`config.target_schema=earth_boundary_source`、license 和 mapping。
|
||||
2. 在数据源页依次采集这三个源。每个源成功后会保存完整 artifact 到 `data/earth-boundary-sources/<collector>/<sha256>.*`，并写入 sha256、feature count、artifact path、sample properties。
|
||||
3. 三个源都成功后，再采集 `Earth PMTiles 构建器`。缺任一源时它会失败为“未就绪”，不会更新 Earth 国界。
|
||||
4. 构建器需要本机 PATH 里有 `tippecanoe` 和 `pmtiles`。缺工具时任务失败并显示缺失工具。
|
||||
5. 生产构建成功后应输出 `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles` 和对应 manifest。
|
||||
6. 部署后打开 Earth，开启“国界线”，放大中国东南海岸、台湾、海南、南海、藏南、科索沃、加沙等区域验证 hover 和边界口径。
|
||||
7. 如果 PMTiles 加载失败，Earth 会报国界图层错误并且不绘制旧低精度国界。排查顺序是：浏览器 Network 是否有 PMTiles range 请求、manifest 的 `tileProvider` 是否为 `pmtiles-mvt`、Nginx 是否能静态返回 `.pmtiles`、artifact 路径和 sha256 是否与 manifest 一致。
|
||||
1. 在控制台 `运维与配置 -> Earth 内容 -> 国界精度` 保存国界源配置；本机配置写入 `config/earth-boundary-sources.local.json`，不要提交。
|
||||
2. 点击“构建高精国界”，或在 Earth 页面工具栏齿轮中切到“高精”触发首次构建。后端会下载三类源到 `data/earth-boundary-sources/`，生成 source manifest，并调用 PMTiles 构建脚本。
|
||||
3. 构建器需要本机 PATH 里有 `tippecanoe` 和 `pmtiles`。缺工具时接口返回明确错误，不会写入数据源采集记录。
|
||||
4. 构建成功后应输出 `frontend/public/earth/data/boundaries/earth-boundaries-china-pov-v1.pmtiles` 和对应 manifest。
|
||||
5. 部署后打开 Earth，开启“国界线”，放大中国东南海岸、台湾、海南、南海、藏南、科索沃、加沙等区域验证 hover 和边界口径。
|
||||
6. 如果本地没有高精 manifest/PMTiles，Earth 会使用 `frontend/public/earth/data/countries-admin0.min.geojson` 低精度 fallback；如果高精产物存在但瓦片请求失败，按 PMTiles range 请求、manifest provider、Nginx `.pmtiles` 静态返回和 sha256 一致性排查。
|
||||
|
||||
## 相关文档
|
||||
|
||||
|
||||
@@ -11,7 +11,7 @@
|
||||
入口分两类：
|
||||
|
||||
- 公开页：`/earth`（3D 态势）、`/docs`（公共文档）
|
||||
- 登录后：`/admin`（控制台）、`/ai`（AI）、`/settings`（系统配置）
|
||||
- 登录后：`/admin`（控制台）、`/ai`（AI）、`/settings`（系统设置）、`/earth-content`（Earth 内容）、`/collection-management`（采集管理）
|
||||
|
||||
## 2. 注册账号
|
||||
|
||||
@@ -32,7 +32,7 @@
|
||||
|
||||
进入 `/admin` 仪表盘后，建议按这个顺序熟悉控制台：
|
||||
|
||||
1. `/settings?tab=collector_credentials`：选一个采集器，点插头图标做连接测试。免费 collector（开源 BGP 等）通常直接可用；像 `AISStream`、`BarentsWatch` 这类需要凭证的，需要先填 API Key/Client Secret
|
||||
1. `/collection-management?tab=collector_credentials`：选一个采集器，点插头图标做连接测试。免费 collector（开源 BGP 等）通常直接可用；像 `AISStream`、`BarentsWatch` 这类需要凭证的，需要先填 API Key/Client Secret
|
||||
2. `/ai?tab=providers`：填一个 LLM provider（例如 `minimax` / `openai`）、模型名、Base URL、API Key，点 Base URL 末端的插头测试连接。WebSearch / OCR 工具可选
|
||||
3. `/datasources` 或 `/data`：看采集器是否已经产出数据。有限采集器看 `/datasources -> 采集任务`，AISStream / WebSocket 长连接看 `/datasources -> 实时流` 的健康状态和计数
|
||||
4. `/alerts/system`：看系统告警是否正常
|
||||
|
||||
Reference in New Issue
Block a user