#12 Added an EnvVar to specify the function for updating the observedArea of datastreams

[jph6366]

#12

Summary

Adding environment variables for specifiying what kind of aggregate function to use in update_data_observedArea function

• This affects how PostGIS will read the spatial bounds of the set of geometries
  • istSOS4 uses [[http:postgis.net/docs/ST_ConvexHull.html][ST_ConvexHull]] to compute the smallest /convex/ geometry that encloses all geometries in the input.
  • Specifically the FastAPI endpoints to insert, update_patch, and delete when we update_datastream_observedArea, we aggegrate features that are distinctly selected from the SensorThings /Observations/ and optionally /FeaturesOfInterest/ given datastream id, Optionally feature id.

Implementation

• DONE
  SCHEDULED: <2025-05-15 Thu 08:00>
  i 2025/05/15 08:14:46 istSOS4
  o 2025/05/15 11:06:14

query context method DELETE /query params/ (conn, $1 datastream_id, $2 feature_id )

• SELECT DISTINCT state is used to return distinct values (on sensorthings.FeaturesOfInterest.id) WHERE sensorthings.Observation.featuresofinterest_id = on distinct foi.id
  AND sensorthings.Observation.datastream_id = $1
• SELECT ST_ConvexHull uses ST_Collect which accepts a minimum of 2 geometries, an array of geometries, or an aggregate function accepting a rowset of geometries.
• UPDATE sensorthings.Datastream.observedArea WHERE id = $1 on-the-fly with SET to change the runtime configuration to the aggregate geometry from the convex hull

query context method POST /query params/ (conn, $1 datastream_id, $2 foi_id )

• UPDATE sensorthings.Datastream.observedArea SET to ST_ConvexHull to collect observedAreas FROM sensorthings.FeaturesOfInterest WHERE id = $1 = $2

query context method PATCH /query params/ (conn, $1 datastream_id)

• SELECT DISTINCT on sensorthings.FeaturesOfInterest.id AND sensorthings.FeaturesOfInteres.feature WHERE sensorthings.Observation.featureofinterest_id = foi.id AND o.datastream_id = $1
  Aggregate geometry with ST_ConvexHull from distinct features
• UPDATE sensorthings.Datastream SET observedArea = aggregate geometry WHERE id = $1

We also use it in our dummy data generator.py in the update_datastream_observed_area

• Here we use it to generate the observedArea by iterating through the distinct ids in sensorthings.Datastream then querying the distinct featuresOfInterest ids associated with the datastream.
• Initialize geometries pointer to [].
  • For the foi in distinct featuresofinterest_ids, SELECT feature FROM sensorthings.FeaturesOfInterest WHERE id = $1
  • if feature is not null/undefined then append to geometries
  • if geometries is not empty, then UPDATE sensorthings.Datastream SET observedArea to __ WHERE id = $1

ST_ConvexHull(
   ST_Collect(
      ARRAY[{', '.join(f"'{g}'::geometry" for g in geometries)}]
   )
)

TODO [#A] <2025-06-18 Tue 08:00>

DEADLINE: <2025-06-18 Wed>
Rewrites from @ClaudioPrimerano committed, time to refactor and add tests to isolate the new update_data_observedArea functions.py

• The rewrite ports overs from using monolith python files for CRUD to instead using feature folders for the CRUD actions.
  • we now only use ST_ConvexHull, to compute the smalllest convex geometry, inside the base CRUD functions.py and inside the create directory functions.py

updating the code is much easier, now lets try testing

[#B] Happy Testing

• create async update_datastream_observedArea
  • create/functions.py
• update/delete async update_datastream_observedArea is called in the query contexts [see above] but isolated in their own sensorthings.* entity file.
  • delete/observations.py
  • delete/feature_of_interest.py
  • update/observation.py
  • update/feature_of_interest.py

POSTMAN REQUESTS

1. Create DB (deep insert)
2. GET /Datastreams(id) - verify deep insertt
3. PATCH /Observations(id) - update
4. GET /Datastreams(id) - verify patch
5. DELETE /Observations(id) - delete
6. GET /Datastreams(id) - verify delete

[#C] UnHappy Testing

1. 404 not found
2. missing required fields
3. invalid geometry
4. invalid query params
5. race condition
6. transaction isolation

[danistrigaro]

Hi @jph6366 , can you adjust your modification based on the new version that we have merged into the main branch? I saw that you worked on the main where there was old code of the software. Now we updated it.
Let me now!
Thanks
Daniele

[jph6366]

Hi @jph6366 , can you adjust your modification based on the new version that we have merged into the main branch? I saw that you worked on the main where there was old code of the software. Now we updated it. Let me now! Thanks Daniele

Great the changes are a nice improvement to developer experience and the postman_collection also very helpful.

Should I be writing a test using FastAPI's Starlette or Add to the postman_collection.json with a new item for testing ST_Aggregate envVar