diff --git a/docs/docs/adding-field-with-default.md b/docs/docs/adding-field-with-default.md index 59e0c26c..da81d3e9 100644 --- a/docs/docs/adding-field-with-default.md +++ b/docs/docs/adding-field-with-default.md @@ -18,28 +18,30 @@ This statement is only safe when your default is non-volatile. ::: ```sql -ALTER TABLE "core_recipe" ADD COLUMN "foo" integer DEFAULT 10 NOT NULL; +-- blocks reads and writes while schema is changed (fast) +ALTER TABLE "account" ADD COLUMN "foo" integer DEFAULT 10; ``` ### adding a volatile default -Add the field as nullable, then set a default, backfill, and remove nullabilty. +Add the field without a default, set the default, and then backfill existing rows in batches. Instead of: ```sql -ALTER TABLE "core_recipe" ADD COLUMN "foo" integer DEFAULT 10 NOT NULL; +-- blocks reads and writes while table is rewritten (slow) +ALTER TABLE "account" ADD COLUMN "ab_group" integer DEFAULT random(); ``` Use: ```sql -ALTER TABLE "core_recipe" ADD COLUMN "foo" integer; -ALTER TABLE "core_recipe" ALTER COLUMN "foo" SET DEFAULT 10; --- backfill column in batches -ALTER TABLE "core_recipe" ALTER COLUMN "foo" SET NOT NULL; -``` +-- blocks reads and writes while schema is changed (fast) +ALTER TABLE "account" ADD COLUMN "ab_group" integer; +-- blocks reads and writes while schema is changed (fast) +ALTER TABLE "account" ALTER COLUMN "ab_group" SET DEFAULT random(); -We add our column as nullable, set a default for new rows, backfill our column (ideally done in batches to limit locking), and finally remove nullability. +-- backfill existing rows in batches to set the "ab_group" column +``` See ["How not valid constraints work"](constraint-missing-not-valid.md#how-not-valid-validate-works) for more information on adding constraints as `NOT VALID`. diff --git a/docs/docs/adding-not-nullable-field.md b/docs/docs/adding-not-nullable-field.md index ea5eba0f..666de82b 100644 --- a/docs/docs/adding-not-nullable-field.md +++ b/docs/docs/adding-not-nullable-field.md @@ -17,24 +17,41 @@ an `ACCESS EXCLUSIVE` lock. Reads and writes will be disabled while this stateme ## solutions -### adding a non-nullable column +### adding a non-nullable column with a non-volatile default in Postgres 11+ + +:::note +This statement is only safe when your default is non-volatile. +::: + +```sql +-- blocks reads and writes while schema is changed (fast) +ALTER TABLE "account" ADD COLUMN "foo" integer DEFAULT 10 NOT NULL; +``` + +### adding a non-nullable column with a volatile default Add a column as nullable and use a check constraint to verify integrity. The check constraint should be added as `NOT NULL` and then validated. Instead of: ```sql -ALTER TABLE "core_recipe" ADD COLUMN "foo" integer DEFAULT 10 NOT NULL; +-- blocks reads and writes while table is rewritten (slow) +ALTER TABLE "account" ADD COLUMN "ab_group" integer DEFAULT random() NOT NULL; ``` Use: ```sql -ALTER TABLE "core_recipe" ADD COLUMN "foo" integer DEFAULT 10; -ALTER TABLE "core_recipe" ADD CONSTRAINT foo_not_null - CHECK ("foo" IS NOT NULL) NOT VALID; --- backfill column so it's not null -ALTER TABLE "core_recipe" VALIDATE CONSTRAINT foo_not_null; +-- blocks reads and writes while schema is changed (fast) +ALTER TABLE "account" ADD COLUMN "ab_group" integer; +-- blocks reads and writes while schema is changed (fast) +ALTER TABLE "account" ALTER COLUMN "ab_group" SET DEFAULT random(); +ALTER TABLE "account" ADD CONSTRAINT ab_group_not_null + CHECK ("ab_group" IS NOT NULL) NOT VALID; + +-- backfill column in batches so it's not null + +ALTER TABLE "account" VALIDATE CONSTRAINT ab_group_not_null; ``` Add the column as nullable, add a check constraint as `NOT VALID` to verify new rows and updates are, backfill the column so it no longer contains null values, validate the constraint to verify existing rows are valid. diff --git a/docs/docs/safe_migrations.md b/docs/docs/safe_migrations.md index 0e29f0bd..0def55c4 100644 --- a/docs/docs/safe_migrations.md +++ b/docs/docs/safe_migrations.md @@ -49,4 +49,41 @@ With a short `lock_timeout` of 1 second, queries will be blocked for up to 1 sec ## further reading -Benchling's ["Move fast and migrate things: how we automated migrations in Postgres"](https://benchling.engineering/move-fast-and-migrate-things-how-we-automated-migrations-in-postgres-d60aba0fc3d4) and GoCardless's ["Zero-downtime Postgres migrations - the hard parts"](https://gocardless.com/blog/zero-downtime-postgres-migrations-the-hard-parts/) provide more background on `lock_timeout` and `statement_timeout` in a production environment. \ No newline at end of file +Benchling's ["Move fast and migrate things: how we automated migrations in Postgres"](https://benchling.engineering/move-fast-and-migrate-things-how-we-automated-migrations-in-postgres-d60aba0fc3d4) and GoCardless's ["Zero-downtime Postgres migrations - the hard parts"](https://gocardless.com/blog/zero-downtime-postgres-migrations-the-hard-parts/) provide more background on `lock_timeout` and `statement_timeout` in a production environment. + +## experiementing with locks + +Create some example + +```sql +-- create table +create table "account" ( + id bigint generated always as identity primary key, + created_at timestamptz not null default now() +); +create table "account_email" ( + id bigint generated always as identity primary key, + account_id bigint not null, + email text not null +); + +-- open a transaction +begin; + +-- run your migration +alter table account_email + add constraint fk_account + foreign key ("account_id") references "account" ("id") not valid; + +-- check locks +select + locktype, + relation::regclass, + mode, + transactionid as tid, + virtualtransaction as vtid, + pid, + granted +from pg_locks; + +``` diff --git a/linter/src/rules/adding_field_with_default.rs b/linter/src/rules/adding_field_with_default.rs index 97763df6..c76ac51e 100644 --- a/linter/src/rules/adding_field_with_default.rs +++ b/linter/src/rules/adding_field_with_default.rs @@ -106,6 +106,7 @@ ALTER TABLE "core_recipe" ADD COLUMN "foo" integer DEFAULT uuid(); let ok_sql = r#" -- NON-VOLATILE ALTER TABLE "core_recipe" ADD COLUMN "foo" integer DEFAULT 10; +ALTER TABLE "account" ADD COLUMN "last_modified" timestamptz DEFAULT now(); "#; let pg_version_11 = Some(Version::from_str("11.0.0").unwrap());