release: bump version to 0.69.0
This commit is contained in:
linkong
@@ -23,6 +23,7 @@ This is the current Intelligent Planet documentation entry point. Docs are organ
|
||||
- [Earth Interactable Usage](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-usage.md): `Interactable` API, lifecycle, and integration examples
|
||||
- [Earth Interactable Clustering](/home/ray/dev/linkong/planet/docs/technical/en/earth-interactable-clustering.md): pluggable cluster strategies, stable spherical clustering, and dynamic screen clustering boundaries
|
||||
- [Earth Toolbar and Overlay Coordination](/home/ray/dev/linkong/planet/docs/technical/en/earth-toolbar-overlay-coordination.md): close matrix for toolbar buttons, search, settings, news, and layer overlays
|
||||
- [Intelligent Planet News Source Configuration](/home/ray/dev/linkong/planet/docs/technical/en/earth-news-sources.md): default sources, feed children, source property tags, content categories, importance rules, and configuration APIs
|
||||
|
||||
## Frontend Implementation
|
||||
|
||||
|
||||
@@ -75,6 +75,8 @@ This is currently the most critical UI control entry point for the Earth fronten
|
||||
|
||||
Earth settings are now grouped by `data-settings-tab` and `data-settings-tab-panel`. Desktop and mobile share the same category semantics: Runtime, Display, Panels, Motion, Shortcuts, and System. When adding a setting, first choose its category, then add the DOM, persistence field, and restore logic; do not keep growing one long undifferentiated panel.
|
||||
|
||||
The news category selector in Display reuses the same chip-selector pattern as Cruise Modules. It only filters news categories for the current browser on the Earth frontend. It does not toggle layers, basemap, boundaries, TV, data points, BGP, vessels, satellites, or compute centers; those remain owned by the layer panel, media panel, and admin configuration. `controls.js` persists only `shared.newsCategoryFilters` and broadcasts `earth:news-category-filters-change`; `news.js` sends the selected categories to `/api/v1/news/earth-feed?categories=...&locale=zh-CN`, so Web and UE clients share the same backend category filtering path.
|
||||
|
||||
Shortcut configuration is a device-local preference owned by `controls.js`: read, capture, enable/disable, and reset all stay in the Earth frontend. It should not be written to backend user settings and should not affect other browsers. New shortcuts must provide a default key, display label, disabled/enabled state, and reset path instead of being hard-coded only in a keydown handler.
|
||||
|
||||
### 4. UI and Status Messages
|
||||
|
||||
161
docs/technical/en/earth-news-sources.md
Normal file
161
docs/technical/en/earth-news-sources.md
Normal file
@@ -0,0 +1,161 @@
|
||||
# Earth News Source Configuration
|
||||
|
||||
Earth situational news is served through `/api/v1/news/earth-feed`. News source configuration lives in `SystemSetting.category = "earth_news_sources"`; when no database configuration exists, the backend uses the built-in default sources as the fallback seed.
|
||||
|
||||
## Default Sources
|
||||
|
||||
The default set contains four groups:
|
||||
|
||||
- **News feeds**: BBC World, DW Top Stories, CNBC Business, BBC Business, Guardian Business, NPR Business, MarketWatch, TechCrunch, Retail Dive, PR Newswire Retail, 36Kr, Ebrun, and China NBS data releases.
|
||||
- **Industry insight sources**: McKinsey Retail and Deloitte Retail.
|
||||
- **Official data sources**: China NBS data releases, US Census Retail / E-Commerce, MOFCOM Data, MOFCOM e-commerce updates, and China e-commerce logistics index.
|
||||
- **Lead sources**: BusinessWire Electronic Commerce; Google News is one aggregated source with feed children for global, Americas, Europe, Middle East / Africa, and Asia Pacific.
|
||||
|
||||
Config data sources are visible in Admin by default. If a source is not a stable RSS/Atom feed, it is kept disabled for automatic fetching until an administrator replaces it with a fetchable URL and enables it.
|
||||
|
||||
| Source | Type | Default state | Default category | Main tags | Purpose |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| BBC World | RSS | Enabled | Politics | `official_media`, `global` | Global public-news baseline |
|
||||
| DW Top Stories | RSS | Enabled | Politics | `official_media`, `europe` | Europe and international baseline |
|
||||
| CNBC Business | RSS | Enabled | Business | `business_news`, `us`, `global` | International business news |
|
||||
| BBC Business / Guardian Business / NPR Business / MarketWatch | RSS | Enabled | Business / Finance | `business_news`, `finance` | UK / US business and finance baseline |
|
||||
| TechCrunch / Retail Dive / PR Newswire Retail | RSS | Enabled | Technology / Business | `business_news`, `ecommerce`, `retail`, `press_release` | Technology, e-commerce, retail, and company announcements |
|
||||
| 36Kr | RSS | Enabled | Business | `business_news`, `ecommerce`, `china` | China business, venture, and newsflash feeds; homepage is `https://www.36kr.com/`, the Feed Directory is `https://www.36kr.com/rss-center`, and feed children are the general, article, newsflash, and moment feeds |
|
||||
| Ebrun | RSS | Enabled | E-commerce | `ecommerce`, `business_news`, `china`, `retail` | China e-commerce industry news; homepage is `https://www.ebrun.com/`, the Feed Directory is `https://www.ebrun.com/rss/`, and feed children are B2C, B2B, retail, O2O, service, data, and policy XML feeds |
|
||||
| China NBS data releases | RSS | Enabled | E-commerce | `official_data`, `ecommerce`, `retail`, `china` | Official data release RSS; retail and online retail items are identified by category and importance rules |
|
||||
| Google News | Aggregated | Enabled | Politics | `aggregated`, `low_stability` | One aggregated source with global, Americas, Europe, Middle East / Africa, and Asia Pacific feed children; lower priority than real RSS |
|
||||
| BusinessWire Electronic Commerce | Reference | Disabled | E-commerce | `press_release`, `ecommerce`, `low_stability` | Corporate announcement leads |
|
||||
| McKinsey Retail Insights | Reference | Disabled | Business | `industry_insight`, `retail` | Retail industry insight |
|
||||
| Deloitte Retail | Reference | Disabled | Business | `industry_insight`, `retail` | Retail industry insight |
|
||||
| US Census Retail / E-Commerce | Reference | Disabled | E-commerce | `official_data`, `ecommerce`, `retail`, `us` | US retail and e-commerce official data |
|
||||
| MOFCOM Data | Reference | Disabled | Business | `official_data`, `china` | China commerce data |
|
||||
| MOFCOM e-commerce updates | Reference | Disabled | E-commerce | `official_data`, `ecommerce`, `china` | China e-commerce policy and updates |
|
||||
| China e-commerce logistics index | Reference | Disabled | E-commerce | `official_data`, `ecommerce`, `logistics`, `china` | Logistics fulfillment and e-commerce activity |
|
||||
|
||||
`Reference` means a reference link or future collector lead. It records a homepage, report page, or data page and does not participate in RSS/Atom fetching. This lets commercial and official sources enter Admin governance without letting non-feed pages break the live news feed.
|
||||
|
||||
The news source model has two levels:
|
||||
|
||||
- `source` is the brand or aggregator, such as 36Kr, Ebrun, Google News, or BBC.
|
||||
- `homepage_url` is the source homepage, section page, or report page.
|
||||
- `feed_directory_url` is the Feed Directory page, such as an RSS subscription center or feed index. It is for human inspection and is not fetched.
|
||||
- `feeds` are the actual RSS, Atom, or Aggregated child entries under that source. Each feed child has `id / name / url / type / enabled / default_category / tags / priority`.
|
||||
|
||||
The backend iterates over every enabled feed child under the same source, fetches them independently, merges and deduplicates items, and writes per-feed diagnostics into `health.feed_results`. This is not a backup URL model: all four 36Kr subscription feeds, multiple Ebrun category XML feeds, and the five Google News regional RSS feeds can be enabled at the same time, and each feed can have its own default category and enabled state. HTML subscription-center pages belong in `feed_directory_url`, not in feed URLs. Every default enabled fetchable feed is tested item by item: RSS/Atom/Aggregated feeds must parse at least one item, while Reference sources only retain a reference URL and future collector lead.
|
||||
|
||||
Items that still remain Reference are not treated as broken feeds; no stable directly consumable RSS/Atom feed was verified:
|
||||
|
||||
- BusinessWire documents customizable RSS/Atom support, but the public pages do not expose a stable industry feed URL; the e-commerce industry page is kept as an announcement lead.
|
||||
- McKinsey and Deloitte retail insight pages are report/article collections, not public RSS feeds.
|
||||
- The US Census press-release RSS is reachable, but its items currently have empty links; the Quarterly E-Commerce page remains an official data reference.
|
||||
- MOFCOM data and China e-commerce logistics index pages do not expose stable RSS feeds yet; they should become dedicated collectors or be replaced with administrator-provided fetchable feeds.
|
||||
|
||||
## Source Property Tags and News Categories
|
||||
|
||||
News sources have `source_tags`, shown in Admin as source property tags. They describe the source, not the media name and not the content category of an individual story:
|
||||
|
||||
- `official_data`
|
||||
- `business_news`
|
||||
- `ecommerce`
|
||||
- `finance`
|
||||
- `retail`
|
||||
- `logistics`
|
||||
- `industry_insight`
|
||||
- `press_release`
|
||||
- `china`, `global`, `us`
|
||||
- `aggregated`, `low_stability`
|
||||
|
||||
Each news item has exactly one primary `category`. Defaults are politics, business, e-commerce, finance, sports, technology, military, disaster, energy, society, culture, and other. `item_tags` are item-level secondary tags, such as cross-border e-commerce, live commerce, retail data, logistics fulfillment, platform governance, AI, semiconductor, election, oil price, football, and supply chain.
|
||||
|
||||
The primary category is generated by a rule-based scorer over title, summary, and source text. If the rules do not match, the feed child default category is used first, then the source default category. AI enrichment does not block news display.
|
||||
|
||||
## Importance
|
||||
|
||||
Each item includes:
|
||||
|
||||
- `importance_score`
|
||||
- `importance_level`
|
||||
- `importance_reasons`
|
||||
- `market_impact`
|
||||
|
||||
Official data, e-commerce metrics, major platforms, and numeric business signals increase importance. Press releases start with a lower baseline and rise only when they match stronger platform, amount, M&A, or regulatory signals.
|
||||
|
||||
## Configuration and Cache
|
||||
|
||||
`GET /api/v1/earth/news-sources` returns the default or saved configuration. `PUT /api/v1/earth/news-sources` saves it, increments `cache_version`, and clears the process region cache. `POST /api/v1/earth/news-sources/reset` restores defaults. `POST /api/v1/earth/news-sources/test` tests one RSS/Atom/Aggregated source without writing news items.
|
||||
|
||||
## Feed Query and Category Filtering
|
||||
|
||||
The Web Earth client and UE client both consume `GET /api/v1/news/earth-feed`. The endpoint supports server-side filtering, so clients do not need to fetch the full list and apply the primary category filter locally.
|
||||
|
||||
- `lat` / `lon`: infer the active region from the current view, used by the Web Earth client.
|
||||
- `region`: explicitly select a region for UE or service integrations. Supported values include `global`, `americas`, `europe`, `asia-pacific`, and `middle-east-africa`. `global` is an aggregate view and can include every region; non-global regions include only their own region plus `global` sources.
|
||||
- `categories`: comma-separated news category keys, for example `business,ecommerce`. Omit it when all categories are selected.
|
||||
- `locale`: display locale, currently `zh-CN` or `en-US`, defaulting to `zh-CN`. Chinese RSS items are stored as Chinese source content and enriched with `en-US`; English RSS items are enriched with `zh-CN`.
|
||||
|
||||
Examples:
|
||||
|
||||
```http
|
||||
GET /api/v1/news/earth-feed?region=europe&categories=business,ecommerce
|
||||
GET /api/v1/news/earth-feed?lat=48&lon=10&categories=technology
|
||||
GET /api/v1/news/earth-feed?region=global&categories=business,ecommerce&locale=zh-CN
|
||||
```
|
||||
|
||||
Unknown category or locale values return `422` with the allowed values. The response includes `filters`, which confirms the region, category, and locale filters applied by the backend. `items` and `cruise_items` use the same category filter set.
|
||||
|
||||
The Web Earth category chips only store the current browser preference; changing them triggers a new API request. UE should pass its selected categories through the `categories` query parameter and does not need to perform the primary filtering itself.
|
||||
|
||||
Source testing only proves that a specific RSS/Atom/XML feed can be parsed. It does not mean those items have already been written to the news table or are visible in the current region/category view. Saving or resetting news sources increments the configuration version and clears cache; if an enabled feed has no recent stored items, the next `earth-feed` request supplements from RSS so newly enabled sources such as 36Kr and Ebrun are not masked by fresh Google News rows.
|
||||
|
||||
## Connectivity Monitoring
|
||||
|
||||
`POST /api/v1/earth/news-sources/test` tests one source and writes the result to `earth_news_sources.health[source_id]`. Normal RSS/Atom fetches update the same health map.
|
||||
|
||||
Health results include:
|
||||
|
||||
- `status`: `ok`, `empty`, `format_error`, `http_error`, `timeout`, `network_error`, or `reference`.
|
||||
- `status_code`, `content_type`, `item_count`, `latency_ms`, `error`, and `fetched_at`.
|
||||
- `feed_results`: per-feed diagnostics for multi-feed sources, including `feed_id`, `feed_name`, `feed_type`, `feed_url`, status, item count, and error.
|
||||
|
||||
Common diagnostics:
|
||||
|
||||
- HTML response: the configured URL is not an RSS/Atom feed, for example a web page listing RSS options.
|
||||
- HTTP 403: the source or CDN rejected the crawler request.
|
||||
- Reference: the source is a reference link only and must be converted to RSS, Atom, or Aggregated before fetch testing.
|
||||
|
||||
The Admin entry is `Earth Content -> News Sources`. It is not a raw whole-payload JSON editor. The UI has two layers:
|
||||
|
||||
- **News sources**: a left-side source list with filters for enabled, disabled, reference links, RSS/Atom/Aggregated, region, and source property tags; the right side edits one selected source and its feed child list.
|
||||
- **Policy rules**: global source property tags, news categories, item tag rules, and default health policy. Advanced JSON is reserved for diagnostics, not the default edit path.
|
||||
|
||||
The single-source form is split into source information and feed children:
|
||||
|
||||
- Source information covers name, ID, region, homepage URL, Feed Directory URL, source type, enabled state, source property tags, importance weight, fetch interval, timeout, failure threshold, and circuit breaker.
|
||||
- Feed children cover feed ID, name, real feed URL, type, enabled switch, default news category, priority, and feed tags. The `+` button under the feed child list creates a frontend-only draft; saving the source persists it, while canceling destroys the draft.
|
||||
|
||||
The per-source “test source” action tests all enabled feed children under the current source. The feed-row test action tests only that feed child. Both send to `/api/v1/earth/news-sources/test`, but the feed-row action submits the current source with only the selected feed child.
|
||||
|
||||
Reference links show that they only record a homepage, report page, or future collector lead and do not participate in RSS/Atom fetching. They can remain as commercial or official-data leads, but must be converted to RSS, Atom, or Aggregated with fetchable feed URLs before they can be enabled for fetching.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Admin["Admin: Earth Content / News Sources"] --> Source["Source config"]
|
||||
Source --> Feed["Feed children"]
|
||||
Feed --> ConfigAPI["/api/v1/earth/news-sources"]
|
||||
ConfigAPI --> Config["SystemSetting: earth_news_sources"]
|
||||
|
||||
Earth["Earth News Panel"] --> NewsAPI["/api/v1/news/earth-feed"]
|
||||
NewsAPI --> Resolver["Source Resolver"]
|
||||
Resolver --> Config
|
||||
Resolver --> Cache["Region Feed Cache"]
|
||||
Resolver --> Fetcher["RSS / Atom Fetcher"]
|
||||
Fetcher --> Parser["Feed Parser"]
|
||||
Parser --> Classifier["Classifier: category + item_tags + importance"]
|
||||
Classifier --> Store["earth_news_items"]
|
||||
Fetcher --> Health["source health"]
|
||||
Health --> Config
|
||||
Store --> EnrichQueue["Location / Localization Queue"]
|
||||
EnrichQueue --> AI["AI Provider"]
|
||||
Store --> NewsAPI
|
||||
UE["UE Client"] --> NewsAPI
|
||||
```
|
||||
@@ -117,6 +117,8 @@ Admin runtime errors are reported through [runtimeLogs.ts](/home/ray/dev/linkong
|
||||
|
||||
The Logs page follows log increments through the `/ws` `logs_tail` channel. File logs and database logs are both normalized into line events by the backend. When adding a new log source, wire it through the backend source registry and tail manager instead of adding a page-local poller.
|
||||
|
||||
The Logs page now opens in the grouped view by default. It reads `/api/v1/system/logs/observability/groups`, groups Earth, Admin, and service runtime reports by `fingerprint`, and then reads `/api/v1/system/logs/observability/groups/{fingerprint}/events` when an operator opens one group. Raw logs and audit logs remain separate views; only the raw-log view can follow WebSocket updates. Frontend reporters coalesce repeated errors in a short window and submit `occurrence_count`, while the backend writes both `system_logs` and `observability_events` / `observability_event_groups`, so the page should not add another browser-side aggregation pass over identical messages.
|
||||
|
||||
The datasource task queue `View Logs` action opens `/logs?source=system-db&search=task_id=<id>`. Backend database-log search indexes must expand simple JSON context fields into `key=value` aliases such as `task_id=26906` and `datasource_id=20`, so historical task logs remain discoverable without rerunning the task.
|
||||
|
||||
## Current Shared Components
|
||||
|
||||
@@ -300,6 +300,8 @@ Adopt All is for batch processing the compute-center unresolved queue. It starts
|
||||
|
||||
The settings panel is grouped into Runtime, Display, Panels, Motion, Shortcuts, and System. It covers rotate / cruise / motion mode, cruise modules (BGP/news/compute centers/vessels/cables/satellites), view (satellite display style, hover tooltip, satellite idle breathing, real satellite altitude, track display, compact dots, day-night mode, panel toggles), motion debug mode / input source / skeleton-only, shortcut enablement and remapping, default globe size, terrain opacity, reset.
|
||||
|
||||
News categories use the same chip selector as Cruise Modules. They only filter the news panel and news cruise items in the current browser; they do not affect layers, TV, data points, basemap, boundaries, collector jobs, or admin news-source configuration.
|
||||
|
||||
"Real Satellite Altitude" is enabled by default: satellite positions use a compressed display height based on TLE/SGP4 orbital altitude. LEO satellites remain close to the globe, while high-orbit satellites render farther out without leaving the normal view. The high-orbit display height is capped at about one quarter of the globe radius, so GEO / MEO objects remain visually separated from LEO without spreading trails and selection targets too far apart. Turning it off restores the legacy same-sphere satellite display. "Track Display" controls satellite trail visibility; trails are unavailable while the satellite layer is hidden.
|
||||
|
||||
"Hover Tooltip" controls the tooltip shown when the pointer hovers over the globe surface: `Country` shows country details only when land matches a country, and stays silent over oceans such as the Pacific; `Position` shows latitude, longitude, and elevation over land and ocean; `Full` is the default and shows country + position over land and position over ocean.
|
||||
|
||||
@@ -23,6 +23,7 @@
|
||||
- [智能星球可交互图标接入](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-usage.md)：`Interactable` 的接口、生命周期和接入示例
|
||||
- [智能星球可交互图标聚类策略](/home/ray/dev/linkong/planet/docs/technical/zh/earth-interactable-clustering.md)：可插拔 cluster strategy、稳定球面聚类和动态屏幕聚类的适用边界
|
||||
- [智能星球工具栏与浮层协同](/home/ray/dev/linkong/planet/docs/technical/zh/earth-toolbar-overlay-coordination.md)：工具栏按钮与搜索、设置、新闻、图层浮层的关闭矩阵
|
||||
- [智能星球新闻源配置](/home/ray/dev/linkong/planet/docs/technical/zh/earth-news-sources.md)：默认新闻源、Feed 子项、源属性标签、内容类型、重要度规则和配置接口
|
||||
|
||||
## 前端技术实现
|
||||
|
||||
|
||||
@@ -75,6 +75,8 @@ Earth 收到 `/ws` 的 `earth_updates` 时只把它当作刷新提示，真实
|
||||
|
||||
Earth 设置面板现在按 `data-settings-tab` 和 `data-settings-tab-panel` 分类组织。桌面端和移动端使用同一组分类语义：运行、显示、面板、动捕、快捷键、系统。新增设置项时应先判断它属于哪个分类，再补 DOM、持久化字段和恢复逻辑；不要把所有控件继续堆到一个长面板里。
|
||||
|
||||
`显示` 分类里的新闻类型选择复用巡航模块的 chip 选择器形态，只控制星球端当前浏览器的新闻分类显示。它不会打开或关闭图层、底图、边界、TV、数据点、BGP、船舶、卫星或算力中心；这些仍由图层面板、媒体面板和控制台配置各自负责。`controls.js` 只持久化 `shared.newsCategoryFilters` 并广播 `earth:news-category-filters-change`，`news.js` 会把选中的类型拼到 `/api/v1/news/earth-feed?categories=...&locale=zh-CN`，让 Web 和 UE 走同一套后端类型过滤。
|
||||
|
||||
快捷键配置属于设备本地偏好，由 `controls.js` 负责读取、捕获、启用/禁用和重置。它不应写入后端用户设置，也不应影响其它浏览器。后续新增快捷键时，必须同时提供默认键、显示标签、可禁用状态和重置路径，避免只在 keydown handler 中硬编码。
|
||||
|
||||
### 4. UI 与状态消息
|
||||
|
||||
161
docs/technical/zh/earth-news-sources.md
Normal file
161
docs/technical/zh/earth-news-sources.md
Normal file
@@ -0,0 +1,161 @@
|
||||
# Earth 新闻源配置
|
||||
|
||||
Earth 态势新闻使用 `/api/v1/news/earth-feed` 输出给前端。新闻源配置存放在 `SystemSetting.category = "earth_news_sources"`；没有数据库配置时，后端使用内置默认源作为 fallback seed。
|
||||
|
||||
## 默认源
|
||||
|
||||
默认源包含四类：
|
||||
|
||||
- **新闻源**：BBC World、DW Top Stories、CNBC Business、BBC Business、Guardian Business、NPR Business、MarketWatch、TechCrunch、Retail Dive、PR Newswire Retail、36氪、亿邦动力、国家统计局数据发布。
|
||||
- **行业洞察源**：McKinsey Retail、Deloitte Retail。
|
||||
- **官方数据源**：国家统计局数据发布、US Census Retail / E-Commerce、商务数据中心、商务部电商动态、电商物流指数。
|
||||
- **线索源**：BusinessWire Electronic Commerce；Google News 作为一个聚合 source，下面挂 global / americas / europe / middle-east-africa / asia-pacific 五个区域 Feed 子项。
|
||||
|
||||
配置型数据源默认保留在 Admin 配置中，但若不是稳定 RSS/Atom，则默认不参与自动抓取。管理员可以在 `Earth 内容 -> 新闻源` 中改成可抓取 RSS、启用或禁用。
|
||||
|
||||
| 来源 | 类型 | 默认状态 | 默认主类型 | 主要标签 | 用途 |
|
||||
| --- | --- | --- | --- | --- | --- |
|
||||
| BBC World | RSS | 启用 | 政治 | `official_media`, `global` | 全球公共新闻基线 |
|
||||
| DW Top Stories | RSS | 启用 | 政治 | `official_media`, `europe` | 欧洲与国际新闻基线 |
|
||||
| CNBC Business | RSS | 启用 | 商业 | `business_news`, `us`, `global` | 国际商业新闻 |
|
||||
| BBC Business / Guardian Business / NPR Business / MarketWatch | RSS | 启用 | 商业 / 金融 | `business_news`, `finance` | 英美商业与金融基线 |
|
||||
| TechCrunch / Retail Dive / PR Newswire Retail | RSS | 启用 | 科技 / 商业 | `business_news`, `ecommerce`, `retail`, `press_release` | 科技、电商、零售和企业公告 |
|
||||
| 36氪 | RSS | 启用 | 商业 | `business_news`, `ecommerce`, `china` | 国内商业、创投和快讯；主页是 `https://www.36kr.com/`，Feed 信息页是 `https://www.36kr.com/rss-center`，Feed 子项是综合资讯、文章资讯、最新快讯、动态内容 |
|
||||
| 亿邦动力 | RSS | 启用 | 电商 | `ecommerce`, `business_news`, `china`, `retail` | 国内电商行业新闻；主页是 `https://www.ebrun.com/`，Feed 信息页是 `https://www.ebrun.com/rss/`，Feed 子项是 B2C、B2B、零售、O2O、服务、数据、政策 XML |
|
||||
| 国家统计局数据发布 | RSS | 启用 | 电商 | `official_data`, `ecommerce`, `retail`, `china` | 官方数据发布 RSS；社零和网上零售条目由分类/重要度规则识别 |
|
||||
| Google News | Aggregated | 启用 | 政治 | `aggregated`, `low_stability` | 一个聚合 source，Feed 子项为全球、美洲、欧洲、中东与非洲、亚太区域兜底 RSS；优先级低于真实 RSS |
|
||||
| BusinessWire Electronic Commerce | Reference | 禁用 | 电商 | `press_release`, `ecommerce`, `low_stability` | 企业公告线索 |
|
||||
| McKinsey Retail Insights | Reference | 禁用 | 商业 | `industry_insight`, `retail` | 零售行业洞察 |
|
||||
| Deloitte Retail | Reference | 禁用 | 商业 | `industry_insight`, `retail` | 零售行业洞察 |
|
||||
| US Census Retail / E-Commerce | Reference | 禁用 | 电商 | `official_data`, `ecommerce`, `retail`, `us` | 美国零售和电商官方数据 |
|
||||
| 商务数据中心 | Reference | 禁用 | 商业 | `official_data`, `china` | 国内商务数据 |
|
||||
| 商务部电商动态 | Reference | 禁用 | 电商 | `official_data`, `ecommerce`, `china` | 国内电商政策与动态 |
|
||||
| 电商物流指数 | Reference | 禁用 | 电商 | `official_data`, `ecommerce`, `logistics`, `china` | 物流履约与电商景气度 |
|
||||
|
||||
`Reference` 源表示参考链接/未来采集器线索，只记录官网、报告页或数据页，不参与 RSS/Atom 抓取。这样可以把商业与官方数据源先纳入后台治理，同时避免不可抓取页面拖垮新闻 feed。
|
||||
|
||||
新闻源模型是两层结构：
|
||||
|
||||
- `source` 表示来源品牌或聚合器，例如 36氪、亿邦动力、Google News、BBC。
|
||||
- `homepage_url` 表示来源官网、栏目页或报告页。
|
||||
- `feed_directory_url` 表示 Feed 信息页，也就是 RSS 订阅中心或 Feed 聚合页，只用于人工查看，不参与抓取。
|
||||
- `feeds` 表示该来源下真正抓取的 RSS、Atom 或 Aggregated 子项。每个 Feed 子项都有 `id / name / url / type / enabled / default_category / tags / priority`。
|
||||
|
||||
后端会遍历同一 source 下所有启用的 Feed 子项，逐个抓取、合并去重，并把单个子项的检测结果写入 `health.feed_results`。这不是“备用地址”逻辑；36氪的四个订阅地址、亿邦的多个分类 XML、Google News 的五个区域 RSS 都可以同时启用，并且每个 Feed 可以单独配置默认新闻类型和启用状态。HTML 订阅中心或聚合页只能放在 `feed_directory_url`，不能放进 Feed 地址。默认启用的可抓 Feed 已逐项连通性检测：RSS/Atom/Aggregated Feed 必须解析到条目，Reference 源只保留参考地址和后续采集器线索。
|
||||
|
||||
当前仍保留为 Reference 的项不是“坏源”，而是没有找到稳定、可直接消费的 RSS/Atom：
|
||||
|
||||
- BusinessWire 官方说明支持可定制 RSS/Atom，但公开页面未暴露稳定行业 feed URL；当前保留电子商务行业页作为公告线索。
|
||||
- McKinsey / Deloitte 的零售洞察页是报告和文章集合，不是公开 RSS。
|
||||
- US Census 的 press release RSS 可访问，但条目链接为空；Quarterly E-Commerce 页面保留为官方数据参考链接。
|
||||
- 商务部数据、电商物流指数目前未找到稳定 RSS，后续应做专用 collector 或人工配置可抓 feed。
|
||||
|
||||
## 源属性标签与新闻类型
|
||||
|
||||
新闻源有 `source_tags`，在 Admin 中显示为“源属性标签”。它用于描述 source 的属性，不是媒体来源名，也不是新闻条目的内容类型。例如：
|
||||
|
||||
- `official_data`：官方数据
|
||||
- `business_news`：商业新闻
|
||||
- `ecommerce`：电商
|
||||
- `finance`：金融
|
||||
- `retail`：零售
|
||||
- `logistics`：物流
|
||||
- `industry_insight`：行业洞察
|
||||
- `press_release`：企业公告
|
||||
- `china`、`global`、`us`
|
||||
- `aggregated`、`low_stability`
|
||||
|
||||
单条新闻有一个主类型 `category`，默认类型包括：政治、商业、电商、金融、体育、科技、军事、灾害、能源、社会、文化、其他。`item_tags` 是条目级补充标签，例如跨境电商、直播电商、零售数据、物流履约、平台治理、AI、半导体、选举、油价、足球、供应链。
|
||||
|
||||
主类型优先由规则引擎根据标题、摘要、来源名打分生成；规则未命中时优先使用 Feed 子项的默认类型，再回退 source 默认类型。AI enrichment 不阻塞新闻展示。
|
||||
|
||||
## 重要度
|
||||
|
||||
每条新闻输出：
|
||||
|
||||
- `importance_score`
|
||||
- `importance_level`
|
||||
- `importance_reasons`
|
||||
- `market_impact`
|
||||
|
||||
官方数据源、电商指标、平台型公司、量化指标会提高重要度；企业公告基础权重较低，只有命中大平台、金额、并购、监管等信号时提升。
|
||||
|
||||
## 配置与缓存
|
||||
|
||||
`GET /api/v1/earth/news-sources` 返回默认或已保存配置。`PUT /api/v1/earth/news-sources` 保存配置并递增 `cache_version`，同时清理进程内 region cache。`POST /api/v1/earth/news-sources/reset` 恢复默认源。`POST /api/v1/earth/news-sources/test` 只测试单个 RSS/Atom/Aggregated 源，不写入新闻表。
|
||||
|
||||
## Feed 查询与类型过滤
|
||||
|
||||
星球端和 UE 端统一使用 `GET /api/v1/news/earth-feed` 获取新闻。接口支持服务端过滤，不要求客户端拿全量列表后自行筛选。
|
||||
|
||||
- `lat` / `lon`：按当前视角推断区域，适合 Web 星球端。
|
||||
- `region`：显式指定区域，适合 UE 端或服务端集成；可选值包括 `global`、`americas`、`europe`、`asia-pacific`、`middle-east-africa`。`global` 是全局聚合视图，会展示所有区域来源；其它区域只展示该区域和 `global` 来源。
|
||||
- `categories`：逗号分隔的新闻类型 key，例如 `business,ecommerce`。全选时可以不传。
|
||||
- `locale`：展示语言，支持 `zh-CN` 和 `en-US`，默认 `zh-CN`。中文 RSS 会以中文原文入库，并由后台补 `en-US`；英文 RSS 则由后台补 `zh-CN`。
|
||||
|
||||
示例：
|
||||
|
||||
```http
|
||||
GET /api/v1/news/earth-feed?region=europe&categories=business,ecommerce
|
||||
GET /api/v1/news/earth-feed?lat=48&lon=10&categories=technology
|
||||
GET /api/v1/news/earth-feed?region=global&categories=business,ecommerce&locale=zh-CN
|
||||
```
|
||||
|
||||
非法新闻类型或语言会返回 `422`，响应中包含允许值。响应体会带 `filters`，用于确认后端实际应用的区域、类型和语言过滤。`items` 和 `cruise_items` 使用同一套类型过滤规则。
|
||||
|
||||
Web 星球端的新闻类型按钮只保存当前浏览器的显示偏好；偏好变化后会重新请求接口。UE 端应直接把类型选择拼到 `categories` 参数里，不需要再做主过滤。
|
||||
|
||||
源测试只证明当前 RSS/Atom/XML 能解析到条目，不等于这些条目已经入库展示。展示链路还会检查区域、类型过滤和数据库新鲜度。保存或重置新闻源会递增配置版本并清理缓存；如果当前启用的 Feed 子项在库里没有近期条目，下一次 `earth-feed` 请求会补抓，避免新启用的 36氪、亿邦被旧 Google News 缓存挡住。
|
||||
|
||||
## 连通性监测
|
||||
|
||||
`POST /api/v1/earth/news-sources/test` 会测试单个源并把结果写入 `earth_news_sources.health[source_id]`。实际 RSS/Atom 抓取也会更新同一份健康状态。
|
||||
|
||||
健康结果包含：
|
||||
|
||||
- `status`：`ok`、`empty`、`format_error`、`http_error`、`timeout`、`network_error`、`reference`。
|
||||
- `status_code`、`content_type`、`item_count`、`latency_ms`、`error`、`fetched_at`。
|
||||
- `feed_results`：多 Feed source 的逐 Feed 子项检测结果，包含 `feed_id`、`feed_name`、`feed_type`、`feed_url`、状态、条数和错误。
|
||||
|
||||
常见诊断：
|
||||
|
||||
- 返回 HTML 页面：说明配置 URL 不是 RSS/Atom feed，例如把网页中心页当成 feed。
|
||||
- HTTP 403：通常是 CDN、反爬或源站拒绝抓取。
|
||||
- Reference：参考链接，不参与抓取；需要改为 RSS、Atom 或 Aggregated 后才可测试抓取。
|
||||
|
||||
Admin 入口是 `Earth 内容 -> 新闻源`。界面不是整包 JSON 编辑，而是两层：
|
||||
|
||||
- **新闻源**：左侧逐个 source 列表，支持按启用、停用、参考链接、RSS/Atom/Aggregated、区域和源属性标签筛选；右侧编辑当前 source 字段和 Feed 子项列表。
|
||||
- **策略规则**：保留源属性标签、新闻类型、条目标签规则、默认健康策略等全局规则。高级 JSON 只用于排障，不作为默认编辑路径。
|
||||
|
||||
单源表单分为“来源信息”和“Feed 子项”：
|
||||
|
||||
- 来源信息包括名称、ID、区域、主页 URL、Feed 信息页、源类型、启用开关、源属性标签、重要度权重、抓取间隔、超时、失败阈值和熔断开关。
|
||||
- Feed 子项包括 Feed ID、名称、真实 Feed URL、类型、启用开关、默认新闻类型、优先级和 Feed 标签。Feed 子项底部的 `+` 只新增一个前端草稿；保存 source 后才写入配置，取消会销毁草稿。
|
||||
|
||||
单源“测试源”会测试当前 source 下全部启用 Feed；Feed 子项上的测试按钮只测试当前 Feed。测试请求仍发送到 `/api/v1/earth/news-sources/test`，但 payload 里只带当前 source 和选中的 Feed 子项。
|
||||
|
||||
参考链接会显示“只记录官网、报告页或未来采集器线索，不参与 RSS/Atom 抓取”。它可作为商业或官方数据线索保留在配置中，但启用抓取前必须改成 RSS、Atom 或 Aggregated，并提供可抓取的 Feed 地址。
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
Admin["Admin: Earth 内容 / 新闻源"] --> Source["Source 配置"]
|
||||
Source --> Feed["Feed 子项"]
|
||||
Feed --> ConfigAPI["/api/v1/earth/news-sources"]
|
||||
ConfigAPI --> Config["SystemSetting: earth_news_sources"]
|
||||
|
||||
Earth["Earth 新闻面板"] --> NewsAPI["/api/v1/news/earth-feed"]
|
||||
NewsAPI --> Resolver["Source Resolver"]
|
||||
Resolver --> Config
|
||||
Resolver --> Cache["Region Feed Cache"]
|
||||
Resolver --> Fetcher["RSS / Atom Fetcher"]
|
||||
Fetcher --> Parser["Feed Parser"]
|
||||
Parser --> Classifier["Classifier: category + item_tags + importance"]
|
||||
Classifier --> Store["earth_news_items"]
|
||||
Fetcher --> Health["source health"]
|
||||
Health --> Config
|
||||
Store --> EnrichQueue["Location / Localization Queue"]
|
||||
EnrichQueue --> AI["AI Provider"]
|
||||
Store --> NewsAPI
|
||||
UE["UE Client"] --> NewsAPI
|
||||
```
|
||||
@@ -117,6 +117,8 @@ Admin 的状态标签统一走 [StatusText](/home/ray/dev/linkong/planet/fronten
|
||||
|
||||
日志页通过 `/ws` 的 `logs_tail` channel 跟随日志增量；文件日志和数据库日志都由后端统一转换成行事件。新增日志源时优先接入后端 source registry 和 tail manager，不要在日志页写独立轮询器。
|
||||
|
||||
日志页默认进入“重复统计”视图，读取 `/api/v1/system/logs/observability/groups`，按 `fingerprint` 聚合 Earth、Admin 和服务端运行时上报；点击聚合项再读取 `/api/v1/system/logs/observability/groups/{fingerprint}/events` 展示发生明细。原始日志和审计日志仍保留为独立视图；只有原始日志视图允许通过 WebSocket 跟随。前端上报器会在短时间窗口内合并同一错误并提交 `occurrence_count`，后端同时写 `system_logs` 和 `observability_events` / `observability_event_groups`，所以日志页不要再按相同消息在浏览器端二次聚合。
|
||||
|
||||
数据源任务队列的“查看日志”入口跳转到 `/logs?source=system-db&search=task_id=<id>`。后端数据库日志搜索索引必须把 JSON context 中的简单字段同时展开为 `key=value` 别名，例如 `task_id=26906`、`datasource_id=20`，这样历史任务日志不依赖重新执行任务也能被精确查到。
|
||||
|
||||
## 当前共享组件
|
||||
|
||||
@@ -299,6 +299,8 @@ AIS 船只图例按船型显示颜色：货轮、油轮、客船、渔船、军
|
||||
|
||||
设置面板按分类组织：运行、显示、面板、动捕、快捷键、系统。里面包含旋转模式 / 巡航模式 / 动捕模式、巡航模块（BGP/新闻/算力中心/船只/海缆/卫星）、视图设置（卫星显示风格、悬停提示、卫星呼吸闪烁、真实卫星高度、轨迹显示、低缩放圆点、日夜模式、面板显示开关）、动捕调试模式 / 输入源 / 只显示骨骼、快捷键启用与改键、地球默认大小、地形透明度、重置设置。
|
||||
|
||||
新闻类型使用与巡航模块一致的标签选择器，只筛选当前浏览器里的新闻面板和新闻巡航条目，不影响图层、TV、数据点、底图、边界、采集任务或后台新闻源配置。
|
||||
|
||||
“真实卫星高度”默认开启：卫星会按 TLE/SGP4 算出的真实轨道高度做压缩分层显示，低轨仍靠近地球，高轨会更远但不会脱离当前视图。高轨显示高度会被压到地球半径外约四分之一以内，这样 GEO / MEO 仍能和 LEO 分层，但不会把视线、轨迹和选择操作拉得过散；关闭后恢复旧版所有卫星位于同一显示球面的效果。“轨迹显示”控制卫星轨迹线显隐，卫星图层关闭时轨迹也不可见。
|
||||
|
||||
“悬停提示”控制鼠标悬停地表时的 tooltip 内容：`国家` 只在陆地命中国家时显示国家信息，太平洋等海洋区域不弹出地表提示；`位置` 在陆地和海洋都显示纬度、经度和海拔；`完整` 是默认模式，陆地显示国家 + 位置，海洋显示位置。
|
||||
|
||||
Reference in New Issue
Block a user