Skip to content

Commit 847beeb

Browse files
Sam-BollingSam-Bolling
committed
Complete Phase 0 — station selection and config drafts
- stations.json: 8 stations across 8 states (AZ, CO, CA, TX, MD, MN, WA, SC), all with both discharge (00060) and gage height (00065), all NIMS-equipped - cameras.json: 8 matching NIMS cameras with S3 image URL patterns - config.json: earthquake feed set to all_day.geojson (60s polling) - Plan Phase 0 checkboxes 0.1-0.5 all marked done Selected stations: 09380000 Colorado River at Lees Ferry, AZ (111800 sqmi) 09019850 Willow Creek near Granby, CO (92 sqmi) 11313433 Dutch Slough at Jersey Island, CA 08171000 Blanco River at Wimberley, TX (355 sqmi) 01650800 Sligo Creek near Takoma Park, MD (6.45 sqmi) 05051300 Bois De Sioux River near Doran, MN (1880 sqmi) 12439500 Okanogan River at Oroville, WA (3195 sqmi) 02135000 Little Pee Dee River at Galivants Ferry, SC (2790 sqmi)
1 parent 6a06a3f commit 847beeb

4 files changed

Lines changed: 262 additions & 5 deletions

File tree

‎docs/research/USGS_NIMS_Follow_On_Publishers_Plan.md‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -92,11 +92,11 @@ Do not begin with nationwide scale. Begin with intentionally chosen sites that s
9292

9393
| Step | Task | Deliverable | Done? |
9494
|---|---|---|---|
95-
| 0.1 | **Obtain USGS API key** — register at https://api.usgs.gov and obtain a free API key. Without it, USGS rate limits will constrain development. All three publishers share one key. | API key stored in a `.env` file or environment variable on the VM | ☐ |
96-
| 0.2 | **API reconnaissance — USGS Water Data** — probe the actual USGS Water Data endpoints to determine: (a) which base URL to use (legacy `waterservices.usgs.gov` vs newer `labs.waterdata.usgs.gov`), (b) the actual response JSON structure for monitoring locations and time-series values, (c) available parameters per site, (d) pagination behavior. | Written recon notes documenting base URL, response shapes, and selected stations | ☐ |
97-
| 0.3 | **API reconnaissance — NIMS** — probe the NIMS camera discovery and image listing endpoints to determine: (a) whether NIMS v1 is live or v0 is the current version, (b) the camera discovery response format (`/cameras`), (c) the image listing response format (`/listFiles`), (d) URL construction patterns for full-size/thumbnail/newest images. | Written recon notes documenting endpoint version, response shapes, and URL patterns | ☐ |
98-
| 0.4 | **API reconnaissance — USGS Earthquake** — fetch the GeoJSON summary feed (`earthquake.usgs.gov/earthquakes/feed/v1.0/summary/*.geojson`) to confirm response structure, event ID format, and update timestamp field. | Written recon notes documenting response shape and dedupe key | ☐ |
99-
| 0.5 | **Station selection** — using the recon results, select: (a) 5–10 USGS water monitoring locations with reliable gage height and/or discharge data, (b) 3–5 of those stations (or nearby stations) that have NIMS cameras, (c) 1 earthquake feed variant (e.g., `all_hour.geojson` or `significant_month.geojson`). | `stations.json` draft for water, `cameras.json` draft for NIMS, feed URL for earthquake | ☐ |
95+
| 0.1 | **Obtain USGS API key** — register at https://api.usgs.gov and obtain a free API key. Without it, USGS rate limits will constrain development. All three publishers share one key. | API key stored in a `.env` file or environment variable on the VM | ☑ |
96+
| 0.2 | **API reconnaissance — USGS Water Data** — probe the actual USGS Water Data endpoints to determine: (a) which base URL to use (legacy `waterservices.usgs.gov` vs newer `labs.waterdata.usgs.gov`), (b) the actual response JSON structure for monitoring locations and time-series values, (c) available parameters per site, (d) pagination behavior. | Written recon notes documenting base URL, response shapes, and selected stations | ☑ |
97+
| 0.3 | **API reconnaissance — NIMS** — probe the NIMS camera discovery and image listing endpoints to determine: (a) whether NIMS v1 is live or v0 is the current version, (b) the camera discovery response format (`/cameras`), (c) the image listing response format (`/listFiles`), (d) URL construction patterns for full-size/thumbnail/newest images. | Written recon notes documenting endpoint version, response shapes, and URL patterns | ☑ |
98+
| 0.4 | **API reconnaissance — USGS Earthquake** — fetch the GeoJSON summary feed (`earthquake.usgs.gov/earthquakes/feed/v1.0/summary/*.geojson`) to confirm response structure, event ID format, and update timestamp field. | Written recon notes documenting response shape and dedupe key | ☑ |
99+
| 0.5 | **Station selection** — using the recon results, select: (a) 5–10 USGS water monitoring locations with reliable gage height and/or discharge data, (b) 3–5 of those stations (or nearby stations) that have NIMS cameras, (c) 1 earthquake feed variant (e.g., `all_hour.geojson` or `significant_month.geojson`). | `stations.json` draft for water, `cameras.json` draft for NIMS, feed URL for earthquake | ☑ |
100100

101101
**Exit criteria:** API key obtained, all three APIs probed with real responses examined, station/camera/feed selections made.
102102

‎publishers/usgs_eq/config.json‎

Lines changed: 37 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,37 @@
1+
{
2+
"_comment": "Phase 0 draft — USGS Earthquake GeoJSON feed configuration",
3+
"feedUrl": "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/all_day.geojson",
4+
"feedVariant": "all_day",
5+
"_availableVariants": {
6+
"all_hour": "All earthquakes, past hour (typically 10-30 events, ~15 KB)",
7+
"all_day": "All earthquakes, past day (typically 200-400 events, ~300 KB)",
8+
"all_week": "All earthquakes, past week (typically 1500-3000 events, ~2 MB)",
9+
"all_month": "All earthquakes, past month (typically 8000-12000 events, ~10 MB)",
10+
"significant_month": "Significant earthquakes only, past month (typically 5-20 events)"
11+
},
12+
"_selectedRationale": "all_day provides a good balance: enough events to demonstrate real-time earthquake monitoring without overwhelming the CSAPI server. Updated by USGS every minute. Roughly 200-400 features per fetch.",
13+
"pollingIntervalSeconds": 60,
14+
"dedupeStrategy": {
15+
"primaryKey": "feature.id",
16+
"updateField": "feature.properties.updated",
17+
"description": "Each earthquake has a stable event ID. The 'updated' timestamp changes when USGS revises magnitude, location, or status. Publisher should track (id, updated) tuples to avoid re-publishing unchanged events while catching revisions."
18+
},
19+
"responseShape": {
20+
"type": "FeatureCollection",
21+
"metadata": {
22+
"generated": "epoch_ms",
23+
"url": "feed_url",
24+
"title": "human_label",
25+
"status": 200,
26+
"api": "1.10.3",
27+
"count": "num_features"
28+
},
29+
"featureProperties": [
30+
"mag", "place", "time", "updated", "tz", "url", "detail",
31+
"felt", "cdi", "mmi", "alert", "status", "tsunami",
32+
"sig", "net", "code", "ids", "sources", "types",
33+
"nst", "dmin", "rms", "gap", "magType", "type", "title"
34+
],
35+
"geometry": "Point [longitude, latitude, depth_km]"
36+
}
37+
}

‎publishers/usgs_nims/cameras.json‎

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
{
2+
"_comment": "Phase 0 draft — NIMS cameras paired with USGS water monitoring stations",
3+
"_apiBase": "https://api.waterdata.usgs.gov/nims/v0/",
4+
"_imageBaseUrl": "https://usgs-nims-images.s3.amazonaws.com/",
5+
"_notes": {
6+
"imageUrlPattern": "{imageBaseUrl}{resolution}/{camId}/{camId}___YYYY-MM-DDTHH-mm-ssZ.jpg",
7+
"resolutions": ["overlay", "thumbnail", "720", "timelapse"],
8+
"newestImage": "Use /listFiles?camId={camId}&limit=1&recent=true to get newest filename",
9+
"newestJpgDirect": "___newest.jpg does NOT work (returns 404). Must use listFiles API."
10+
},
11+
"cameras": [
12+
{
13+
"camId": "AZ_Colorado_River_at_Lees_Ferry_Upstream",
14+
"nwisId": "09380000",
15+
"stationName": "Colorado River at Lees Ferry, AZ",
16+
"lat": 36.864333,
17+
"lon": -111.587872,
18+
"state": "AZ"
19+
},
20+
{
21+
"camId": "CO_WILLOW_CREEK_SL",
22+
"nwisId": "09019850",
23+
"stationName": "Willow Creek below Cabin Creek near Granby, CO",
24+
"lat": 40.214269,
25+
"lon": -106.050661,
26+
"state": "CO"
27+
},
28+
{
29+
"camId": "CA_Dutch_Slough_below_Jersey_Island_Road_at_Jersey_Island",
30+
"nwisId": "11313433",
31+
"stationName": "Dutch Slough at Jersey Island, CA",
32+
"lat": 38.013533,
33+
"lon": -121.667731,
34+
"state": "CA"
35+
},
36+
{
37+
"camId": "TX_Blanco_River_at_Wimberley",
38+
"nwisId": "08171000",
39+
"stationName": "Blanco River at Wimberley, TX",
40+
"lat": 29.994383,
41+
"lon": -98.088925,
42+
"state": "TX"
43+
},
44+
{
45+
"camId": "MD_Sligo_Creek_Near_Takoma_Park",
46+
"nwisId": "01650800",
47+
"stationName": "Sligo Creek near Takoma Park, MD",
48+
"lat": 38.986222,
49+
"lon": -77.004861,
50+
"state": "MD"
51+
},
52+
{
53+
"camId": "MN_Bois_De_Sioux_River_near_Doran",
54+
"nwisId": "05051300",
55+
"stationName": "Bois De Sioux River near Doran, MN",
56+
"lat": 46.152504,
57+
"lon": -96.579343,
58+
"state": "MN"
59+
},
60+
{
61+
"camId": "WA_Okanogan_River_at_Zosel_Millpond_at_Oroville",
62+
"nwisId": "12439500",
63+
"stationName": "Okanogan River at Oroville, WA",
64+
"lat": 48.930718,
65+
"lon": -119.420342,
66+
"state": "WA"
67+
},
68+
{
69+
"camId": "SC_Little_Pee_Dee_River_at_Galivants_Ferry",
70+
"nwisId": "02135000",
71+
"stationName": "Little Pee Dee River at Galivants Ferry, SC",
72+
"lat": 34.05675,
73+
"lon": -79.247722,
74+
"state": "SC"
75+
}
76+
]
77+
}

‎publishers/usgs_water/stations.json‎

Lines changed: 143 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,143 @@
1+
{
2+
"_comment": "Phase 0 draft — 8 USGS water monitoring stations (all with discharge + gage height + NIMS camera)",
3+
"_sources": {
4+
"ogcApi": "https://api.waterdata.usgs.gov/ogcapi/v0/",
5+
"legacyNwis": "https://waterservices.usgs.gov/nwis/iv/"
6+
},
7+
"stations": [
8+
{
9+
"nwisId": "09380000",
10+
"name": "Colorado River at Lees Ferry, AZ",
11+
"fullName": "COLORADO RIVER AT LEES FERRY, AZ",
12+
"lat": 36.864333,
13+
"lon": -111.587872,
14+
"state": "Arizona",
15+
"stateAbbr": "AZ",
16+
"county": "Coconino County",
17+
"tz": "MST",
18+
"drainageArea_sqmi": 111800.0,
19+
"huc": "140700061105",
20+
"parameterCodes": ["00060", "00065"],
21+
"camId": "AZ_Colorado_River_at_Lees_Ferry_Upstream"
22+
},
23+
{
24+
"nwisId": "09019850",
25+
"name": "Willow Creek below Cabin Creek near Granby, CO",
26+
"fullName": "WILLOW CREEK BELOW CABIN CREEK NEAR GRANBY, CO",
27+
"lat": 40.214269,
28+
"lon": -106.050661,
29+
"state": "Colorado",
30+
"stateAbbr": "CO",
31+
"county": "Grand County",
32+
"tz": "MST",
33+
"drainageArea_sqmi": 92.0,
34+
"huc": "140100010103",
35+
"parameterCodes": ["00060", "00065"],
36+
"camId": "CO_WILLOW_CREEK_SL"
37+
},
38+
{
39+
"nwisId": "11313433",
40+
"name": "Dutch Slough at Jersey Island, CA",
41+
"fullName": "DUTCH SLOUGH BL JERSEY ISLAND RD A JERSEY ISLAND",
42+
"lat": 38.013533,
43+
"lon": -121.667731,
44+
"state": "California",
45+
"stateAbbr": "CA",
46+
"county": "Contra Costa County",
47+
"tz": "PST",
48+
"drainageArea_sqmi": null,
49+
"huc": "180400030803",
50+
"parameterCodes": ["00060", "00065"],
51+
"camId": "CA_Dutch_Slough_below_Jersey_Island_Road_at_Jersey_Island"
52+
},
53+
{
54+
"nwisId": "08171000",
55+
"name": "Blanco River at Wimberley, TX",
56+
"fullName": "Blanco Rv at Wimberley, TX",
57+
"lat": 29.994383,
58+
"lon": -98.088925,
59+
"state": "Texas",
60+
"stateAbbr": "TX",
61+
"county": "Hays County",
62+
"tz": "CST",
63+
"drainageArea_sqmi": 355.0,
64+
"huc": "121002030204",
65+
"parameterCodes": ["00060", "00065"],
66+
"camId": "TX_Blanco_River_at_Wimberley"
67+
},
68+
{
69+
"nwisId": "01650800",
70+
"name": "Sligo Creek near Takoma Park, MD",
71+
"fullName": "SLIGO CREEK NEAR TAKOMA PARK, MD",
72+
"lat": 38.986222,
73+
"lon": -77.004861,
74+
"state": "Maryland",
75+
"stateAbbr": "MD",
76+
"county": "Montgomery County",
77+
"tz": "EST",
78+
"drainageArea_sqmi": 6.45,
79+
"huc": "020700100201",
80+
"parameterCodes": ["00060", "00065"],
81+
"camId": "MD_Sligo_Creek_Near_Takoma_Park"
82+
},
83+
{
84+
"nwisId": "05051300",
85+
"name": "Bois De Sioux River near Doran, MN",
86+
"fullName": "BOIS DE SIOUX RIVER NEAR DORAN, MN",
87+
"lat": 46.152504,
88+
"lon": -96.579343,
89+
"state": "Minnesota",
90+
"stateAbbr": "MN",
91+
"county": "Wilkin County",
92+
"tz": "CST",
93+
"drainageArea_sqmi": 1880.0,
94+
"huc": "090201010507",
95+
"parameterCodes": ["00060", "00065"],
96+
"camId": "MN_Bois_De_Sioux_River_near_Doran"
97+
},
98+
{
99+
"nwisId": "12439500",
100+
"name": "Okanogan River at Oroville, WA",
101+
"fullName": "OKANOGAN RIVER AT OROVILLE, WA",
102+
"lat": 48.930718,
103+
"lon": -119.420342,
104+
"state": "Washington",
105+
"stateAbbr": "WA",
106+
"county": "Okanogan County",
107+
"tz": "PST",
108+
"drainageArea_sqmi": 3195.0,
109+
"huc": "170200061506",
110+
"parameterCodes": ["00060", "00065"],
111+
"camId": "WA_Okanogan_River_at_Zosel_Millpond_at_Oroville"
112+
},
113+
{
114+
"nwisId": "02135000",
115+
"name": "Little Pee Dee River at Galivants Ferry, SC",
116+
"fullName": "LITTLE PEE DEE R. AT GALIVANTS FERRY, SC",
117+
"lat": 34.05675,
118+
"lon": -79.247722,
119+
"state": "South Carolina",
120+
"stateAbbr": "SC",
121+
"county": "Horry County",
122+
"tz": "EST",
123+
"drainageArea_sqmi": 2790.0,
124+
"huc": "03040204",
125+
"parameterCodes": ["00060", "00065"],
126+
"camId": "SC_Little_Pee_Dee_River_at_Galivants_Ferry"
127+
}
128+
],
129+
"parameterDefinitions": {
130+
"00060": {
131+
"name": "Discharge",
132+
"description": "Instantaneous discharge (streamflow)",
133+
"unit": "ft3/s",
134+
"swe_definition": "http://www.opengis.net/def/property/OGC/0/Discharge"
135+
},
136+
"00065": {
137+
"name": "Gage height",
138+
"description": "Instantaneous gage height (water level)",
139+
"unit": "ft",
140+
"swe_definition": "http://www.opengis.net/def/property/OGC/0/GageHeight"
141+
}
142+
}
143+
}

0 commit comments

Comments
 (0)