Add Met Office Global Spot forecast publisher

Author: Sam-Bolling

publishers/.env.example

@@ -30,6 +30,19 @@ OSH_ROOT=sensorhub
 # Optional override if Met Office changes/publishes a different gateway URL.
 # MET_OFFICE_LAND_OBSERVATIONS_BASE_URL=https://data.hub.api.metoffice.gov.uk/observation-land/1
 
+# Met Office Weather DataHub Global Spot / Site-Specific Forecast API key.
+# Required for publishers.met_office_global_spot.
+# MET_OFFICE_GLOBAL_SPOT_API_KEY=
+
+# Optional alternative for systemd/Oracle deployments: point to a root-owned
+# file containing either the raw key or MET_OFFICE_GLOBAL_SPOT_API_KEY=...
+# MET_OFFICE_GLOBAL_SPOT_API_KEY_FILE=/etc/os4csapi/secrets/met-office-global-spot.key
+
+# Optional overrides for the access-gated Site-Specific Forecast endpoint.
+# MET_OFFICE_GLOBAL_SPOT_BASE_URL=https://data.hub.api.metoffice.gov.uk/sitespecific/v0
+# MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH=/point/hourly
+# MET_OFFICE_GLOBAL_SPOT_FORECAST_HOURS=24
+
 # Optional API-key header name. Default is apikey.
 # MET_OFFICE_DATAHUB_API_KEY_HEADER=apikey
 

publishers/README.md

@@ -21,7 +21,8 @@ server (e.g. [OpenSensorHub](https://opensensorhub.org/)).
 | **Environment Agency Hydrology** | EA river level, flow, rainfall, groundwater | 15 min |
 | **UK-AIR** | Defra UK-AIR NO2, O3, PM10, PM2.5 | 1 h |
 | **BGS SensorThings** | BGS UKGEOS Glasgow groundwater/geothermal telemetry | 15 min |
-| **Met Office DataHub** | Land Observations hourly weather data | Access-gated / planned |
+| **Met Office DataHub** | Land Observations hourly weather data | Access-gated / 1 h |
+| **Met Office Global Spot** | Site-specific deterministic hourly forecasts | Access-gated / 1 h |
 
 ## Quick Start
 
@@ -62,6 +63,8 @@ python -m publishers.uk_air.bootstrap_uk_air
 python -m publishers.bgs_sensorthings.bootstrap_bgs_sensorthings
 # Met Office DataHub is access-gated; see publishers/met_office_datahub/README.md
 python -m publishers.met_office_datahub.bootstrap_met_office_datahub
+# Met Office Global Spot is access-gated; see publishers/met_office_global_spot/README.md
+python -m publishers.met_office_global_spot.bootstrap_met_office_global_spot
 python -m publishers.iss.bootstrap_iss
 ```
 
@@ -72,6 +75,7 @@ cd publishers
 docker compose up -d          # start all
 docker compose up -d nws      # start one
 docker compose --profile access-gated up -d met-office-datahub
+docker compose --profile access-gated up -d met-office-global-spot
 docker compose logs -f nws    # follow logs
 docker compose ps             # status
 docker compose down            # stop all
@@ -121,6 +125,8 @@ python -m publishers.nws.nws_publisher --interval 3600
 | `USGS_API_KEY` | No | USGS API key for higher rate limits |
 | `MET_OFFICE_LAND_OBSERVATIONS_API_KEY` | For Met Office | Met Office Weather DataHub Land Observations subscription key |
 | `MET_OFFICE_LAND_OBSERVATIONS_API_KEY_FILE` | For Met Office | Host-local file containing the Land Observations key; useful for Oracle/systemd secrets |
+| `MET_OFFICE_GLOBAL_SPOT_API_KEY` | For Met Office Global Spot | Met Office Weather DataHub Site-Specific Forecast / Global Spot subscription key |
+| `MET_OFFICE_GLOBAL_SPOT_API_KEY_FILE` | For Met Office Global Spot | Host-local file containing the Global Spot key; useful for Oracle/systemd secrets |
 | `BUOYCAM_CACHE_ROOT` | No | Local directory for BuoyCAM image cache |
 | `BUOYCAM_CACHE_BASE_URL` | No | Public URL serving the cached images |
 
@@ -143,4 +149,10 @@ python -m publishers.nws.nws_publisher --interval 3600
   caches nearest geohash lookups in ignored runtime state to stay within the
   free-plan call budget. In Docker Compose it is behind the `access-gated`
   profile, so public publishers can still start without a Met Office key.
+- **Met Office Global Spot** uses the access-gated Site-Specific Forecast API
+  and reads `MET_OFFICE_GLOBAL_SPOT_API_KEY` from the environment, or
+  `MET_OFFICE_GLOBAL_SPOT_API_KEY_FILE` from a host-local secret file. It
+  publishes deterministic hourly forecast values for virtual forecast points,
+  not physical sensor observations, and is also behind the `access-gated`
+  Docker Compose profile.
 - All publishers use `--interval <seconds>` and `--dry-run` CLI flags.

publishers/docker-compose.yml

@@ -131,6 +131,24 @@ services:
       MET_OFFICE_DATAHUB_429_BACKOFF: ${MET_OFFICE_DATAHUB_429_BACKOFF:-3600}
     command: ["--interval", "3600"]
 
+  # ── Met Office Weather DataHub Global Spot Forecast (1h cadence) ──
+  met-office-global-spot:
+    profiles: ["access-gated"]
+    build:
+      context: ..
+      dockerfile: publishers/met_office_global_spot/Dockerfile
+    restart: always
+    environment:
+      <<: *osh-env
+      MET_OFFICE_GLOBAL_SPOT_API_KEY: ${MET_OFFICE_GLOBAL_SPOT_API_KEY:-}
+      MET_OFFICE_GLOBAL_SPOT_BASE_URL: ${MET_OFFICE_GLOBAL_SPOT_BASE_URL:-https://data.hub.api.metoffice.gov.uk/sitespecific/v0}
+      MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH: ${MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH:-/point/hourly}
+      MET_OFFICE_GLOBAL_SPOT_FORECAST_HOURS: ${MET_OFFICE_GLOBAL_SPOT_FORECAST_HOURS:-24}
+      MET_OFFICE_DATAHUB_API_KEY_HEADER: ${MET_OFFICE_DATAHUB_API_KEY_HEADER:-apikey}
+      MET_OFFICE_GLOBAL_SPOT_REQUEST_DELAY: ${MET_OFFICE_GLOBAL_SPOT_REQUEST_DELAY:-1.0}
+      MET_OFFICE_GLOBAL_SPOT_429_BACKOFF: ${MET_OFFICE_GLOBAL_SPOT_429_BACKOFF:-3600}
+    command: ["--interval", "3600"]
+
   # ── USGS Earthquake Feed (60s cadence) ──
   usgs-eq:
     build:

publishers/met_office_global_spot/Dockerfile

@@ -0,0 +1,15 @@
+FROM python:3.12-slim
+
+WORKDIR /app
+
+COPY publishers /app/publishers
+
+ENV PYTHONUNBUFFERED=1
+ENV MET_OFFICE_GLOBAL_SPOT_API_KEY=
+ENV MET_OFFICE_GLOBAL_SPOT_API_KEY_FILE=
+ENV MET_OFFICE_GLOBAL_SPOT_BASE_URL=https://data.hub.api.metoffice.gov.uk/sitespecific/v0
+ENV MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH=/point/hourly
+ENV MET_OFFICE_DATAHUB_API_KEY_HEADER=apikey
+
+ENTRYPOINT ["python", "-m", "publishers.met_office_global_spot.met_office_global_spot_publisher"]
+CMD ["--interval", "3600"]

publishers/met_office_global_spot/README.md

@@ -0,0 +1,91 @@
+# Met Office Global Spot Forecast Publisher
+
+Status: initial implementation for the access-gated Met Office Weather DataHub Site-Specific Forecast / Global Spot product. Live validation requires a `MET_OFFICE_GLOBAL_SPOT_API_KEY` subscription key.
+
+This publisher is separate from `publishers/met_office_datahub`, which publishes observed Land Observations station telemetry. Global Spot values are deterministic forecasts for configured points, not readings from physical sensors.
+
+## Source
+
+- Weather DataHub landing page: https://datahub.metoffice.gov.uk/
+- Site-specific forecast overview: https://datahub.metoffice.gov.uk/docs/g/category/site-specific/overview
+- Site-specific pricing: https://datahub.metoffice.gov.uk/pricing/site-specific
+- Weather DataHub docs: https://datahub.metoffice.gov.uk/docs
+
+The subscribed product context from planning notes is `/sitespecific/v0`, API name `SiteSpecificForecast`, with a free-plan allowance of 360 calls/day.
+
+## CSAPI Model
+
+- One procedure for Met Office Global Spot hourly forecast retrieval.
+- One virtual forecast-point system per curated location.
+- One datastream per selected forecast parameter.
+- One deployment group for curated Global Spot forecast points.
+- Each published observation uses forecast valid time as `phenomenonTime` and preserves issued time, valid time, lead time, parameter, unit, source URL, and forecast type in the result payload.
+
+These resources intentionally avoid claiming the forecast points are physical stations.
+
+## Configuration
+
+Store the API key outside git. In this workspace the preferred local file is `publishers/.env`, which is ignored by git.
+
+```text
+MET_OFFICE_GLOBAL_SPOT_API_KEY=...
+```
+
+On live hosts, prefer a service environment variable or a root-owned secret file:
+
+```text
+MET_OFFICE_GLOBAL_SPOT_API_KEY_FILE=/etc/os4csapi/secrets/met-office-global-spot.key
+```
+
+The key file may contain the raw key on the first non-comment line, or one of these assignments:
+
+```text
+MET_OFFICE_GLOBAL_SPOT_API_KEY=...
+MET_OFFICE_SITE_SPECIFIC_FORECAST_API_KEY=...
+MET_OFFICE_DATAHUB_API_KEY=...
+```
+
+Optional overrides:
+
+```text
+MET_OFFICE_GLOBAL_SPOT_BASE_URL=https://data.hub.api.metoffice.gov.uk/sitespecific/v0
+MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH=/point/hourly
+MET_OFFICE_GLOBAL_SPOT_FORECAST_HOURS=24
+MET_OFFICE_DATAHUB_API_KEY_HEADER=apikey
+MET_OFFICE_GLOBAL_SPOT_REQUEST_DELAY=1.0
+MET_OFFICE_GLOBAL_SPOT_429_BACKOFF=3600
+```
+
+`MET_OFFICE_GLOBAL_SPOT_HOURLY_PATH` may be an absolute URL, a path under the base URL, or a format string containing `{lat}`, `{lon}`, `{latitude}`, and `{longitude}`.
+
+## Commands
+
+Probe the authenticated API and inspect response shape:
+
+```bash
+python -m publishers.met_office_global_spot.met_office_global_spot_publisher --probe --locations london-heathrow-area
+```
+
+Run a dry publisher cycle without POSTing to OSH:
+
+```bash
+python -m publishers.met_office_global_spot.met_office_global_spot_publisher --dry-run --once
+```
+
+Bootstrap CSAPI resources:
+
+```bash
+python -m publishers.met_office_global_spot.bootstrap_met_office_global_spot
+```
+
+Run one live publish cycle:
+
+```bash
+python -m publishers.met_office_global_spot.met_office_global_spot_publisher --once
+```
+
+## Implementation Notes
+
+The initial curated points mirror the Met Office Land Observations demo geography: London Heathrow, Stornoway, and Cairngorm. At one hourly API request per point per cycle, a one-hour cadence uses 72 calls/day, comfortably below the 360 calls/day free-plan allowance.
+
+The runtime client has a configurable endpoint path because public DataHub pages confirm the Site-Specific Forecast / Global Spot product and `/sitespecific/v0` context, but the exact hourly endpoint path should be validated with the live subscribed API before production service installation.

publishers/met_office_global_spot/__init__.py

@@ -0,0 +1 @@
+"""Met Office Weather DataHub Global Spot forecast publisher package."""