Fix test/install, add test-build, verify-results, docs#18
Fix test/install, add test-build, verify-results, docs#18jnasbyupgrade wants to merge 18 commits intoPostgres-Extensions:masterfrom
Conversation
- Add .claude/settings.json with references to test repos - Add CLAUDE.md documenting the meta-framework architecture - Include git commit guidelines Co-Authored-By: Claude <noreply@anthropic.com>
This ensures all projects using pgxntool will ignore Claude Code local configuration files Co-Authored-By: Claude <noreply@anthropic.com>
Add *.md to export-ignore to prevent markdown files (including CLAUDE.md) from being included in extension distributions Co-Authored-By: Claude <noreply@anthropic.com>
Add `.claude/commands/commit.md` with comprehensive commit workflow that will be shared with pgxntool-test via symlink. This ensures consistent commit standards across both repos. Document META.json generation process in `build_meta.sh` to explain why we generate from template (PGXN.org doesn't like empty optional fields) and future possibilities (could generate control files from template). Co-Authored-By: Claude <noreply@anthropic.com>
Add testing section to `CLAUDE.md` with critical rules: never use `make installcheck` directly, never run `make results` without verification, database connection requirements. Enhance `README.asc` to recommend `make test`, document `make results` verification workflow, and emphasize pgTap benefits. Co-Authored-By: Claude <noreply@anthropic.com>
… validation - Add `make_results.sh` script to handle copying results while respecting `output/*.source` files as source of truth - Update `base.mk` to properly handle ephemeral files from `.source` files: - Track `TEST_OUT_SOURCE_FILES` and `TEST_EXPECTED_FROM_SOURCE` - Add ephemeral files to `EXTRA_CLEAN` so `make clean` removes them - Create `test/results/` directory automatically - Remove rule that created `test/output/` directory (it's an optional input) - Add validation in `dist-only` target to ensure `.gitattributes` is committed before creating distribution (git archive only respects export-ignore for committed files) - Update `README.asc` and `CLAUDE.md` to document that development should be done from pgxntool-test repository Co-Authored-By: Claude <noreply@anthropic.com>
- Add `pgtle` make target to generate pg_tle registration SQL for extensions - Support pg_tle version ranges (1.0.0-1.5.0 and 1.5.0+) with appropriate API usage - Add `pgtle.sh` script to generate pg_tle SQL from extension control files and versioned SQL - Update `base.mk` to include pg_tle generation with proper dependencies on SQL files - Add pg_tle/ directory to EXTRA_CLEAN for `make clean` - Enhance `.claude/commands/commit.md` to require checking all 3 repos before committing - Add explicit guidance to commit all repos with changes together (no empty commits) - Document multi-repo commit workflow in steps - Update `CLAUDE.md` with pg_tle development context and documentation - Update `README.asc` and `README.html` with pg_tle usage documentation Co-Authored-By: Claude <noreply@anthropic.com>
Add "OPTIONAL TEST FEATURES" section documenting `test-build`, `test/install`, and `verify-results` configuration. Also clarify that `verify-results` works with all `pg_regress`-based tests, not just pgTap, and fix assignment to use `:=` to avoid recursion. Co-Authored-By: Claude <noreply@anthropic.com>
Replace local commit.md with reference to canonical version in pgxntool-test. All three repos (pgxntool, pgxntool-test, pgxntool-test-template) now share the same commit workflow. Co-Authored-By: Claude <noreply@anthropic.com>
Add new pg_tle version range to handle API changes in 1.4.0: - Split 1.0.0-1.5.0 range into 1.0.0-1.4.0 and 1.4.0-1.5.0 - Version 1.4.0 added uninstall function (backward-incompatible) - Version 1.5.0 added schema parameter (another boundary) - Update `base.mk` pattern rules for three version ranges - Update `pgtle.sh` documentation and logic for new range Extract shared utility functions into `lib.sh`: - Move `error()`, `die()`, and `debug()` functions from `pgtle.sh` - Update `pgtle.sh` and `setup.sh` to source `lib.sh` - Reduces code duplication and improves maintainability Refine `.gitattributes` export rules: - Add specific excludes: `CLAUDE.md`, `PLAN-*.md`, `.DS_Store` - Remove blanket `*.md` exclude (too broad) - Allows README.md and other docs to be included in distributions Add documentation about directory purity in `CLAUDE.md`: - Emphasize that pgxntool directory contains ONLY embedded files - Warn against adding temporary files or planning documents - Clarify that such files belong in pgxntool-test instead Co-Authored-By: Claude <noreply@anthropic.com>
Remove references to pgxntool-test-template (now consolidated): - Template files moved to pgxntool-test/template/ - Remove template directory from `.claude/settings.json` - Update `CLAUDE.md` for two-repository pattern Convert `.claude/commands/commit.md` to symlink: - Development happens in pgxntool-test, avoid duplication - Symlink: `commit.md -> ../../../pgxntool-test/.claude/commands/commit.md` Related changes in pgxntool-test: - Consolidate template files into pgxntool-test/template/ - Simplify commit workflow to two-phase (remove amend step) Co-Authored-By: Claude <noreply@anthropic.com>
Merges master's control.mk system and update-setup-files.sh while preserving test-build, test-install, and verify-results features from the build-test branch. Key changes: - Added control.mk.sh for parsing .control files - Added update-setup-files.sh for 3-way merging after pgxntool-sync - Added SETUP_FILES/SETUP_SYMLINKS config to lib.sh - Simplified meta.mk.sh (defers extension handling to control.mk.sh) - Used master's variable naming (TEST__SOURCE__*) - Preserved PGXNTOOL_ENABLE_TEST_BUILD feature - Preserved PGXNTOOL_ENABLE_TEST_INSTALL feature - Preserved PGXNTOOL_ENABLE_VERIFY_RESULTS feature Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Enhancement request: Override mechanism for install/build file listsI'm planning to use the test/install support for pg_tle integration in my extensions monorepo. The auto-detection is great for the common case, but I need to control which files run and in what order. Two requests: 1. Documented override for test/install and test-build file lists
Suggested approach: Expose an intermediate variable that the schedule generation uses, documented as the public API for overriding: # Default: auto-detected from test/install/*.sql (unordered)
# Projects can override for ordering or conditional inclusion.
PGXNTOOL_INSTALL_TESTS ?= $(notdir $(basename $(TEST_INSTALL_SQL_FILES)))
$(PGXNTOOL_INSTALL_SCHEDULE):
@echo "# Auto-generated - DO NOT EDIT" > $@
@for f in $(PGXNTOOL_INSTALL_TESTS); do \
echo "test: ../install/$$f" >> $@; \
doneThen projects can do: include pgxntool/base.mk
# Explicit ordering + conditional pg_tle setup
PGXNTOOL_INSTALL_TESTS = extensions
ifdef PGTLE_MODE
PGXNTOOL_INSTALL_TESTS := pgtle-setup $(PGXNTOOL_INSTALL_TESTS)
endifSame pattern would be useful for test-build ( 2. Document new make targets to help with conflictsProjects that already hand-coded Suggestions:
|
…s, docs Replace broken separate-invocation test-install with schedule-based approach: install files now run in the same pg_regress invocation as regular tests via an auto-generated schedule file with ../install/ relative paths. State created by install files persists into the main test suite. - Add `test-build` feature for pre-test SQL validation via `test/build/` - Add `verify-results` safeguard preventing `make results` when tests fail - Add `.asc` to recognized asciidoc extensions - Document test-build, test/install, and verify-results in README - Update `_.gitignore` for auto-generated `test/install/schedule` - Remove `.claude/` directory (moved to pgxntool-test) Related changes in pgxntool-test: - Add install persistence test validating state survives across test phases - Rework `test-test-install.bats` for schedule-based approach - Add template marker files for install persistence validation Co-Authored-By: Claude <noreply@anthropic.com>
91da35b to
f36629c
Compare
README.asc
Outdated
| ---- | ||
| test/build/ | ||
| ├── *.sql # SQL test files (checked in) | ||
| ├── input/ # Optional: .source files for pg_regress processing |
There was a problem hiding this comment.
.source files would also be checked in
| # resolves install files from their original location without copying. | ||
| # | ||
| TEST_INSTALL_SQL_FILES = $(wildcard $(TESTDIR)/install/*.sql) | ||
| ifdef PGXNTOOL_ENABLE_TEST_INSTALL |
There was a problem hiding this comment.
This is almost the same code as for test/build; propose ways (in a comment) it could be refactored. We certainly want the code to remain clear, but I'm hoping we can remove some of the duplication.
See verify-results as well; it's pretty similar as well.
base.mk
Outdated
| ifeq ($(PGXNTOOL_ENABLE_TEST_BUILD),yes) | ||
| TEST_DEPS += test-build | ||
| endif | ||
| TEST_DEPS += install |
There was a problem hiding this comment.
can't these two lines be combined with 267?
| # | ||
| # Variable: PGXNTOOL_ENABLE_VERIFY_RESULTS | ||
| # - Can be set manually in Makefile or command line | ||
| # - Allowed values: "yes" or "no" (case-insensitive) |
There was a problem hiding this comment.
Hmm... lets also add pgtap as an option (and make it the default). This mode would assume that all test output is from pgTap, and look for anything that indicates a test failure. This should be in addition the checks we already have around regression.diff.
Here's the way I'm handling this in another project:
verify-results:
@failed=0; \
if grep -Hn '^not ok' test/results/*.out 2>/dev/null | grep -v '# TODO'; then \
failed=1; \
fi; \
if grep -Hn 'Looks like you planned' test/results/*.out 2>/dev/null; then \
failed=1; \
fi; \
if [ $$failed -eq 1 ]; then \
echo "ERROR: Failed tests found in results files (see above)"; \
exit 1; \
fi
@echo "test/results/ files are clean (no 'not ok' lines or plan count mismatches found, TODO tests ignored)"
base.mk
Outdated
| @for file in $(TEST_BUILD_SQL_FILES); do \ | ||
| cp "$$file" "$(TEST_BUILD_SQL_DIR)/$$(basename $$file)"; \ | ||
| done | ||
| @# Create empty expected files if needed |
There was a problem hiding this comment.
Need a comment on why we even need to do this
README.asc
Outdated
| ├── expected/ # Expected output files (checked in) | ||
| │ └── *.out | ||
| └── sql/ # GENERATED - do not edit or check in | ||
| └── *.sql # Copied from *.sql above; generated from input/*.source |
| SET client_min_messages = WARNING; | ||
|
|
||
| -- Install dependencies your extension requires | ||
| CREATE EXTENSION IF NOT EXISTS pgtap CASCADE; |
There was a problem hiding this comment.
drop this line
|
|
||
| ---- | ||
| \set ECHO none | ||
| \i test/pgxntool/psql.sql |
There was a problem hiding this comment.
Add a comment that this sets onerrorfail (or whatever the variable is called)
| ---- | ||
| \set ECHO none | ||
| \i test/pgxntool/psql.sql | ||
| \t |
There was a problem hiding this comment.
add comment on why this is desirable
| -- Run the actual extension SQL (not CREATE EXTENSION) | ||
| \i sql/myext.sql | ||
|
|
||
| \echo # BUILD TEST SUCCEEDED |
There was a problem hiding this comment.
Add comment above this line that if the build actually failed psql would stop immediately because of error handling set by psql.sql.
- Combine TEST_DEPS lines: merge separate install/installcheck additions into a single line alongside testdeps in the initial assignment - Add pgtap mode to verify-results: new PGXNTOOL_VERIFY_RESULTS_MODE variable (default: pgtap) scans test/results/*.out for "not ok" and plan mismatch lines; also retains regression.diffs check as fallback. "diffs" mode preserves legacy behavior for non-pgtap projects. - Add comment explaining why empty expected files are created: pg_regress aborts immediately if expected/*.out is missing rather than running the test and showing a diff. - Extract test-build shell setup into pgxntool/run-test-build.sh: directory creation, SQL file copying, and empty expected file seeding are now in a dedicated script; base.mk only contains the make installcheck invocation and diffs check. - Replace PGXNTOOL_distclean with EXTRA_CLEAN: META.json, meta.mk, and control.mk are now listed in EXTRA_CLEAN (cleaned by PGXS make clean) instead of a custom distclean target. Removes the distclean: target and the clean: distclean prerequisite. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
For all three feature-enable variables (PGXNTOOL_ENABLE_TEST_BUILD, PGXNTOOL_ENABLE_TEST_INSTALL, PGXNTOOL_ENABLE_VERIFY_RESULTS), add a note in the doc comment that setting the variable to empty on the command line (e.g. PGXNTOOL_ENABLE_TEST_BUILD=) also disables the feature, the same as =no. This works because Make's ifdef sees an empty value as "not defined", so the validation block is skipped. But the Makefile's default assignment cannot override a command-line variable (even an empty one), so the feature-enable ifeq check sees "" and stays disabled. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
test-build: Useful error messages for SQL syntax errors
When
CREATE EXTENSIONfails due to a syntax error, PostgreSQL reports acryptic error with limited context. test-build runs your extension SQL directly
through pg_regress first, so errors show the exact file, line, and position —
cutting debugging time significantly.
Place SQL files in
test/build/to enable. Auto-detects based on file presence.test/install: Run expensive setup once instead of per-test
Extensions that install dependencies or load fixtures in every test file pay
that cost once per test. test/install runs setup SQL once before the entire
suite, and all regular tests share the resulting database state. This can
dramatically speed up test suites.
Place SQL files in
test/install/to enable. Auto-detects based on file presence.invocation (database state persists across phases)
.sqlfiles astest/install/*.outverify-results: Prevent blessing incorrect test output
make resultsnow refuses to run when tests are failing. Prevents accidentallyupdating expected files with wrong output. Disable with
PGXNTOOL_ENABLE_VERIFY_RESULTS=no.Other changes
.ascas a recognized asciidoc extensionCompanion PR: Postgres-Extensions/pgxntool-test#7