Skip to content

Commit 9b19dcf

Browse files
Sam-BollingSam-Bolling
committed
Document Global Spot forecast publisher status
1 parent 3aaa637 commit 9b19dcf

1 file changed

Lines changed: 118 additions & 0 deletions

File tree

‎docs/research/new-publisher-source-planning/Met_Office_Global_Spot_Forecast_Publisher_Status_2026-05-26.md‎

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
# Met Office Global Spot Forecast Publisher Status
2+
3+
Date: 2026-05-26
4+
5+
## Purpose
6+
7+
Record the first implementation slice for the Met Office Weather DataHub Global Spot / Site-Specific Forecast publisher and the remaining live-validation work.
8+
9+
## Implementation Status
10+
11+
Implemented and pushed in commit `3aaa637` (`Add Met Office Global Spot forecast publisher`).
12+
13+
New package:
14+
15+
- `publishers/met_office_global_spot/`
16+
- `publishers/met_office_global_spot/bootstrap_met_office_global_spot.py`
17+
- `publishers/met_office_global_spot/met_office_global_spot_publisher.py`
18+
- `publishers/met_office_global_spot/forecast_points.json`
19+
- `publishers/met_office_global_spot/README.md`
20+
- `publishers/met_office_global_spot/Dockerfile`
21+
- `tests/test_met_office_global_spot_parser.py`
22+
23+
Fleet wiring:
24+
25+
- `publishers/.env.example` documents Global Spot key and endpoint overrides.
26+
- `publishers/docker-compose.yml` includes access-gated `met-office-global-spot` service.
27+
- `publishers/README.md` lists the Global Spot publisher and bootstrap/run commands.
28+
29+
## Source Model
30+
31+
Global Spot is treated as true forecast data, not observed telemetry.
32+
33+
The CSAPI model uses:
34+
35+
- one forecasting procedure for Met Office Global Spot hourly retrieval,
36+
- one virtual forecast-point system per curated location,
37+
- one datastream per selected forecast parameter,
38+
- one deployment group for curated Global Spot forecast points,
39+
- one observation per selected forecast parameter and valid time.
40+
41+
Published result payloads preserve:
42+
43+
- forecast type,
44+
- issued time,
45+
- valid time,
46+
- lead time in hours,
47+
- curated point ID,
48+
- parameter name,
49+
- value and unit,
50+
- source URL.
51+
52+
This is intentionally separate from `publishers/met_office_datahub`, which publishes Land Observations from physical observation locations.
53+
54+
## Curated Initial Slice
55+
56+
Forecast points:
57+
58+
- London Heathrow Area
59+
- Stornoway Coastal Area
60+
- Cairngorm Upland Area
61+
62+
Forecast parameters:
63+
64+
- Forecast Air Temperature
65+
- Forecast Relative Humidity
66+
- Forecast Wind Speed
67+
- Forecast Wind Gust
68+
- Forecast Precipitation Probability
69+
- Forecast Weather Code
70+
71+
With three locations at one request per location per hour, the default operational cadence is about 72 requests/day, comfortably below the documented 360 calls/day free-plan allowance.
72+
73+
## Validation Completed
74+
75+
Local validation completed without printing or changing credentials:
76+
77+
- Python compile passed for the new bootstrap, publisher, and parser test files.
78+
- Bootstrap dry run successfully constructed the procedure, 3 virtual systems, 18 datastreams, and deployment hierarchy.
79+
- Focused parser tests passed: `2 passed`.
80+
- Repository was pushed cleanly to `origin/main` at `3aaa637`.
81+
82+
## Live Validation Boundary
83+
84+
The local `publishers/.env` in this workspace has OSH credentials but does not currently include a Global Spot / Site-Specific Forecast key. The publisher therefore has not yet been live-probed against `/sitespecific/v0` from this machine.
85+
86+
The runtime is deliberately configurable for the final subscribed endpoint check:
87+
88+
```text
89+
MET_OFFICE_GLOBAL_SPOT_API_KEY=...
90+
MET_OFFICE_GLOBAL_SPOT_API_KEY_FILE=/etc/os4csapi/secrets/met-office-global-spot.key
91+
MET_OFFICE_GLOBAL_SPOT_BASE_URL=https://data.hub.api.metoffice.gov.uk/sitespecific/v0
92+
MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH=/point/hourly
93+
MET_OFFICE_GLOBAL_SPOT_FORECAST_HOURS=24
94+
MET_OFFICE_DATAHUB_API_KEY_HEADER=apikey
95+
```
96+
97+
`MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH` can be an absolute URL, a path under the base URL, or a format string containing `{lat}`, `{lon}`, `{latitude}`, and `{longitude}`.
98+
99+
## Next Steps
100+
101+
1. Place the existing Site-Specific Forecast / Global Spot key in a host-local secret file or environment variable without rotating or printing it.
102+
2. Run `python -m publishers.met_office_global_spot.met_office_global_spot_publisher --probe --locations london-heathrow-area` to confirm the exact endpoint path and response shape.
103+
3. If the endpoint path differs from the conservative `/point/hourly` default, set `MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH` rather than changing credentials.
104+
4. Run `python -m publishers.met_office_global_spot.bootstrap_met_office_global_spot --force-sml` on the target OSH server.
105+
5. Run `python -m publishers.met_office_global_spot.met_office_global_spot_publisher --dry-run --once` and then one live `--once` cycle.
106+
6. Verify Explorer behavior: forecast points must be labeled as forecast, and existing rich source cards such as BuoyCAM and water monitoring media must remain unchanged.
107+
7. Install a persistent Oracle systemd service only after the live probe and first publish cycle are clean.
108+
109+
## Explorer Follow-Up
110+
111+
Explorer should add a forecast-specific card section only for true forecast datastreams. This must be additive:
112+
113+
- preserve thumbnails,
114+
- preserve BuoyCAM and NIMS camera imagery,
115+
- preserve water-monitoring media/source links,
116+
- preserve latest readings and recent trends for observed telemetry,
117+
- avoid labeling observed telemetry as forecast,
118+
- avoid labeling Global Spot forecast points as physical deployed sensors.

0 commit comments

Comments
 (0)