Fix test/install, add test-build, verify-results, docs

HISTORY.asc

@@ -1,5 +1,14 @@
 1.0.0
 -----
+== Add test-build for pre-test validation
+When CREATE EXTENSION fails due to a SQL syntax error, PostgreSQL reports only a cryptic error with limited context. test-build runs your extension SQL directly through pg_regress first, so syntax errors show the exact file, line, and position — cutting debugging time significantly. Place SQL files in `test/build/` to enable; auto-detects based on file presence.
+
+== Add test/install for one-time test setup
+Extensions that install dependencies or run expensive setup in every test file pay that cost once per test. test/install runs setup SQL once before the entire test suite, and all regular tests share the resulting database state. This can dramatically speed up test suites that install extensions or load fixtures. Place SQL files in `test/install/` to enable; auto-detects based on file presence.
+
+== Add verify-results safeguard for make results
+`make results` now refuses to run when tests are failing (detected via `regression.diffs`). Prevents accidentally blessing incorrect output as the new expected results. Enabled by default; disable with `PGXNTOOL_ENABLE_VERIFY_RESULTS=no`.
+
 == Fix broken multi-extension support
 Prior to this fix, distributions with multiple extensions or extensions with versions different from the PGXN distribution version were completely broken. Extension versions are now correctly read from each `.control` file's `default_version` instead of using META.json's distribution version.
 

README.asc

@@ -47,6 +47,121 @@ Runs unit tests via the PGXS `installcheck` target. Unlike a simple `make instal
 
 NOTE: While you can still run `make installcheck` or any other valid PGXS make target directly, it's recommended to use `make test` when using pgxntool. The `test` target ensures clean builds, proper test isolation, and correct dependency installation.
 
+=== test-build
+Validates that extension SQL files are syntactically correct before running the full test suite. This feature runs SQL files from `test/build/` through `pg_regress`, providing better error messages than `CREATE EXTENSION` failures when there are syntax errors in your extension code.
+
+**How it works:**
+
+1. Place SQL files in `test/build/*.sql` (or `test/build/input/*.source` for source-processed files)
+2. Place expected output in `test/build/expected/*.out`
+3. These files run through `pg_regress` before `make test` runs the main test suite
+4. If any build test fails, the test run stops immediately with clear error messages
+
+**Directory structure:**
+
+----
+test/build/
+├── *.sql              # SQL test files (checked in)
+├── input/             # Optional: .source files for pg_regress processing (checked in)
+│   └── *.source       # Checked in; pg_regress substitutes tokens into sql/ copies
+├── expected/          # Expected output files (checked in)
+│   └── *.out
+└── sql/               # GENERATED - do not edit or check in
+    └── *.sql          # Copied from *.sql above and generated from input/*.source
+----
+
+The `sql/` subdirectory is generated automatically by `make test-build`. It is listed in `.gitignore` and removed by `make clean`. Do not place files directly in `test/build/sql/`.
+
+**Configuration:**
+
+The feature auto-detects based on whether `test/build/*.sql` or `test/build/input/*.source` files exist:
+
+- Files present → feature enabled automatically
+- No files → feature disabled (no impact on existing projects)
+
+You can override auto-detection by setting `PGXNTOOL_ENABLE_TEST_BUILD`:
+----
+# In your Makefile
+PGXNTOOL_ENABLE_TEST_BUILD = yes  # or no
+----
+
+**Example: Validate extension SQL compiles**
+
+Create `test/build/build.sql` to run your extension's SQL directly:
+
+----
+\set ECHO none
+-- Sets ON_ERROR_STOP, VERBOSITY verbose, and ON_ERROR_ROLLBACK
+\i test/pgxntool/psql.sql
+-- Suppress column headers and row counts for cleaner expected output
+\t
+
+BEGIN;
+SET client_min_messages = WARNING;
+
+-- Install dependencies your extension requires
+CREATE EXTENSION IF NOT EXISTS pgtap CASCADE;
+
+-- Clean slate
+DROP EXTENSION IF EXISTS myext;
+DROP SCHEMA IF EXISTS myext;
+CREATE SCHEMA myext;
+
+-- Run the actual extension SQL (not CREATE EXTENSION)
+\i sql/myext.sql
+
+-- If we get here, the build succeeded; ON_ERROR_STOP would have aborted on any error above
+\echo # BUILD TEST SUCCEEDED
+ROLLBACK;
+----
+
+This approach catches SQL syntax errors *before* running `CREATE EXTENSION`, giving clearer error messages with line numbers. The `ROLLBACK` ensures nothing persists—this is purely validation.
+
+**Why use `\i` instead of `CREATE EXTENSION`?**
+
+When `CREATE EXTENSION` fails, PostgreSQL shows only "syntax error" with limited context. Running the SQL directly via `\i` shows the exact line and position of errors, making debugging much faster.
+
+=== test/install
+Runs setup files before the main test suite within the same `pg_regress` invocation. This allows expensive one-time operations (like extension installation) to set up state that persists into the regular test files.
+
+**How it works:**
+
+1. Place SQL files in `test/install/*.sql`
+2. Place expected output alongside as `test/install/*.out`
+3. A schedule file is auto-generated that lists install files with `../install/` relative paths
+4. `pg_regress` processes the install schedule first, then runs regular test files — all in one invocation, so database state persists
+
+**Directory structure:**
+
+----
+test/install/
+├── *.sql              # SQL setup files (checked in)
+├── *.out              # Expected output (checked in, alongside .sql)
+├── .gitignore         # Ignores pg_regress artifacts (*.out.diff)
+└── schedule           # GENERATED - auto-created by make
+----
+
+The `schedule` file is generated automatically and listed in `.gitignore`. Do not edit it.
+
+**Configuration:**
+
+The feature auto-detects based on whether `test/install/*.sql` files exist:
+
+- Files present → feature enabled automatically
+- No files → feature disabled (no impact on existing projects)
+
+You can override auto-detection by setting `PGXNTOOL_ENABLE_TEST_INSTALL`:
+----
+# In your Makefile
+PGXNTOOL_ENABLE_TEST_INSTALL = yes  # or no
+----
+
+**Why this is useful:**
+
+Without `test/install`, each test file typically needs to run `CREATE EXTENSION` in its setup, which adds overhead and doesn't allow validating the installation step separately. With `test/install`, setup runs once before all tests, and any state it creates (tables, extensions, etc.) is available to every subsequent test file.
+
+**Key detail:** Install files and regular tests run in a single `pg_regress` invocation. This means the database is NOT dropped between install and test phases — state created by install files persists into the main test suite.
+
 === testdeps
 This rule allows you to ensure certain actions have taken place before running tests. By default it has a single prerequisite, `pgtap`, which will attempt to install http://pgtap.org[pgtap] from PGXN. This depneds on having the pgxn client installed.
 
@@ -78,6 +193,24 @@ IMPORTANT: *`make results` requires manual verification first*. The correct work
 
 Never run `make results` without first verifying the test changes are correct. The `results` target copies files from `test/results/` to `test/expected/`, so running it blindly will make incorrect output become the new expected behavior.
 
+==== verify-results safeguard
+By default, `make results` will refuse to run if `test/results/regression.diffs` exists (indicating failing tests). This prevents accidentally updating expected files with incorrect output.
+
+If tests are failing, you'll see:
+----
+ERROR: Tests are failing. Cannot run 'make results'.
+Fix test failures first, then run 'make results'.
+----
+
+To disable this safeguard (not recommended):
+----
+# In your Makefile
+PGXNTOOL_ENABLE_VERIFY_RESULTS = no
+
+# Or on the command line
+make PGXNTOOL_ENABLE_VERIFY_RESULTS=no results
+----
+
 === tag
 `make tag` will create a git branch for the current version of your extension, as determined by the META.json file. The reason to do this is so you can always refer to the exact code that went into a released version.
 
@@ -223,7 +356,7 @@ Location of `asciidoc` or equivalent executable.
 If not set PGXNtool will search for first `asciidoctor`, then `asciidoc`.
 ASCIIDOC_EXTS::
 File extensions to consider as Asciidoc.
-Defined as `+= adoc asciidoc`.
+Defined as `+= adoc asciidoc asc`.
 ASCIIDOC_FILES::
 Asciidoc input files.
 PGXNtool searches each `$(DOC_DIRS)` directory, looking for files with any `$(ASCIIDOC_EXTS)` extension.