From 432607cdba11552bc5157f396e9703d039dfee67 Mon Sep 17 00:00:00 2001 From: regularfry Date: Fri, 15 Dec 2023 18:52:29 +0000 Subject: [PATCH 1/2] Draft cloud database doc --- practices/cloud-databases.md | 29 +++++++++++++++++++++++++++++ 1 file changed, 29 insertions(+) create mode 100644 practices/cloud-databases.md diff --git a/practices/cloud-databases.md b/practices/cloud-databases.md new file mode 100644 index 00000000..05c56b59 --- /dev/null +++ b/practices/cloud-databases.md @@ -0,0 +1,29 @@ +# Cloud databases + +- [Cloud databases](#cloud-databases) + - [Context](#context) + - [Details](#details) + - [Choosing the type of database you need](#choosing-the-type-of-database-you-need) + - [Infrastructure](#infrastructure) + +## Context + +- These notes are part of a broader set of [principles](../principles.md) +- Practices in this section contribute to [service reliability](service-reliability.md) +- See also [observability](observability.md) + +## Details + +### Choosing the type of database you need + +Do not reflexively reach for a specific database product without considering whether it matches your use case. In general you should prefer relational stores because of their feature set and design choices which make them broadly applicable. Choose a non-relational store only if the trade-off makes sense for your application. An example would be needing extremely high write rates (in the range of millions of operations per minute) where eventual consistency is tolerable: in that case DynamoDB or equivalent may be appropriate. + +Prefer to learn how to use the right tool for the job over picking the tool a team happens to be familiar with. + +### Infrastructure + +Successfully operating relational databases in a cloud environment, especially a serverless one, requires attention to the specific qualities of the database products as implemented by the cloud platforms. + +- Do use a small number of long-lived server instances. Do not create new servers per application environment. Database servers can be very slow to instantiate, which means that tools like `terraform` can time out waiting for them to start. Instead separate your infrastructure code so that your servers are configured once per AWS account. The `terraform` code that configures an application instance should provision its resources by calling into the already-provisioned server in the account to create environment-specific databases. +- Do manage your database migrations. Use a tool like `alembic`, or an alternative. If building a serverless application, deploy your migrations into a cloud function that will be called by your `terraform` deployment. +- Learn how to refactor database schemas safely. Several coupled migrations and code changes may be needed to successfully change the schema with no downtime. From b6e57b62864903632466dee8833c7931658ec81d Mon Sep 17 00:00:00 2001 From: regularfry Date: Tue, 19 Dec 2023 13:01:53 +0000 Subject: [PATCH 2/2] Slightly less punchy, for less controversy --- practices/cloud-databases.md | 11 ++--------- 1 file changed, 2 insertions(+), 9 deletions(-) diff --git a/practices/cloud-databases.md b/practices/cloud-databases.md index 05c56b59..76b87cb5 100644 --- a/practices/cloud-databases.md +++ b/practices/cloud-databases.md @@ -3,7 +3,6 @@ - [Cloud databases](#cloud-databases) - [Context](#context) - [Details](#details) - - [Choosing the type of database you need](#choosing-the-type-of-database-you-need) - [Infrastructure](#infrastructure) ## Context @@ -14,16 +13,10 @@ ## Details -### Choosing the type of database you need - -Do not reflexively reach for a specific database product without considering whether it matches your use case. In general you should prefer relational stores because of their feature set and design choices which make them broadly applicable. Choose a non-relational store only if the trade-off makes sense for your application. An example would be needing extremely high write rates (in the range of millions of operations per minute) where eventual consistency is tolerable: in that case DynamoDB or equivalent may be appropriate. - -Prefer to learn how to use the right tool for the job over picking the tool a team happens to be familiar with. - ### Infrastructure Successfully operating relational databases in a cloud environment, especially a serverless one, requires attention to the specific qualities of the database products as implemented by the cloud platforms. -- Do use a small number of long-lived server instances. Do not create new servers per application environment. Database servers can be very slow to instantiate, which means that tools like `terraform` can time out waiting for them to start. Instead separate your infrastructure code so that your servers are configured once per AWS account. The `terraform` code that configures an application instance should provision its resources by calling into the already-provisioned server in the account to create environment-specific databases. -- Do manage your database migrations. Use a tool like `alembic`, or an alternative. If building a serverless application, deploy your migrations into a cloud function that will be called by your `terraform` deployment. +- Do use a small number of long-lived server instances. Do not create new servers per application environment. Database servers can be very slow to instantiate, which means that tools like `terraform` can time out waiting for them to start. Instead separate your infrastructure code between configuration for the application and configuration for the account, so that your servers are configured once per AWS account. The `terraform` code that configures an application instance should provision its resources by calling into the already-provisioned server in the account to create environment-specific databases. +- Do manage your database migrations. If you are using a web framework like Django, or Rails, you will have tooling built in to do this. Otherwise, use a tool like `alembic` if several people are likely to be working on features which change the database structure at the same time. If your needs are simpler, you may find a more straightforward approach - storing SQL scripts and sequentially running them, for instance - works just as well with fewer dependencies. If building a serverless application, deploy your migrations into a cloud function to be called by your `terraform` deployment. - Learn how to refactor database schemas safely. Several coupled migrations and code changes may be needed to successfully change the schema with no downtime.