Features DB Read Replica for External Tools

[anayeaye]

What

This ticket is to provide a persistent document of a RDS read replica maintained by the VEDA Data Services Team and depended upon by users of external systems built on direct database connections. This will hopefully be a way to coordinate user and maintainer needs as well as prevent unexpected outages while dependencies exist.

Contacts

Team	Contact
Fire Tracking Lab	@mccabete, @eorland, @zebbecker
VEDA	@brianna-corremonte, @smohiudd, @anayeaye

Summary

This database is a read replica of the RDS instance maintained for the VEDA Features API

• TiPg: https://developmentseed.org/tipg/
• Deployment Code: https://github.com/NASA-IMPACT/veda-features-api-cdk
• TiPg OGC API stack owning the parent database: staging.openveda.cloud/api/features/docs

The database is updated by an automated airflow DAG when new files are uploaded to staging bucket using (this step of the vector ingest DAG for details). The source files are generated in the MAAP DPS.

Some possible long term options

The data services team will maintain the RDS and loading infrastructure for the short term while longer term decisions are made. There are a couple of longer term options that should be considered (and other ideas should be raised).

1. The services built on direct database access transition to using the OGC Features API/Tiles API (this lets the owning team continue standard operations including redirecting urls to new backends when legacy systems are replaced).
2. The Wildfire Tracking Lab team or ESRI team deploys a copy of the TiPg backend and Airflow system to persist an in-account database for the ESRI tooling. Both projects have open source code that includes IAC for standing up the database and services in an AWS account.

Actions

• Stakeholders and developer sync on migration to OGC APIs (are there TiPg changes that would support this??) and yearly fires record reset in Nov/Dec 2025
• Stakeholders and developer sync to plan sunsetting this temporary solution Jan/Feb 2026

Internal Notes:

• This read replica is hosted in smce-veda us-west-2
• The Manually creating a read replica database section of the operations wiki describes how the replica was created and configured
• The connection parameters are stored in the AWS Secrets Manager veda-featuresdb-read-replica-1adbtW

[mccabete]

Again, I don't want to agree to any timelines until Brian Freitag gets a chance to weigh in. This seems like a bigger issue with ESRI standards vs OGC standards being misaligned on some things but not others, and what do we do on those cases.

I've been talking to other folks, and even with some improvements to the OGC API, we still won't have some features NASA Disasters needs.

[mccabete]

For Reference: Esri's stated OGC support levels

The folks on the Esri side of this are Cole Walts (who has been trying to help us with this from the beginning, but who doesn't do direct OGC development for Esri, is more of a mediator) and Satish Sankaran, who does do development for OGC but who wasn't able to support us three years ago.

[mccabete]

On the EGIS (NASA's internal Esri/ GIS experts and coordinators) side, we have been working with Brian Tisdale, Leah Schwizer, and David Babson. On the NASA Disasters side, we have been working with Patrick Rea and Garrett Layne.