Skip to content

Commit 0a14d2a

Browse files
committed
Pull pgxntool from git@github.com:decibel/pgxntool.git release
2 parents 40625c2 + 81f3ddd commit 0a14d2a

File tree

19 files changed

+2519
-320
lines changed

19 files changed

+2519
-320
lines changed
Lines changed: 78 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,78 @@
1+
---
2+
description: Create a git commit following project standards and safety protocols
3+
allowed-tools: Bash(git status:*), Bash(git log:*), Bash(git add:*), Bash(git diff:*), Bash(git commit:*), Bash(make test:*)
4+
---
5+
6+
# commit
7+
8+
Create a git commit following all project standards and safety protocols for pgxntool-test.
9+
10+
**CRITICAL REQUIREMENTS:**
11+
12+
1. **Git Safety**: Never update `git config`, never force push to `main`/`master`, never skip hooks unless explicitly requested
13+
14+
2. **Commit Attribution**: Do NOT add "Generated with Claude Code" to commit message body. The standard Co-Authored-By trailer is acceptable per project CLAUDE.md.
15+
16+
3. **Testing**: ALL tests must pass before committing:
17+
- Run `make test`
18+
- Check the output carefully for any "not ok" lines
19+
- Count passing vs total tests
20+
- **If ANY tests fail: STOP. Do NOT commit. Ask the user what to do.**
21+
- There is NO such thing as an "acceptable" failing test
22+
- Do NOT rationalize failures as "pre-existing" or "unrelated"
23+
24+
**WORKFLOW:**
25+
26+
1. Run in parallel: `git status`, `git diff --stat`, `git log -10 --oneline`
27+
28+
2. Check test status - THIS IS MANDATORY:
29+
- Run `make test 2>&1 | tee /tmp/test-output.txt`
30+
- Check for failing tests: `grep "^not ok" /tmp/test-output.txt`
31+
- If ANY tests fail: STOP immediately and inform the user
32+
- Only proceed if ALL tests pass
33+
34+
3. Analyze changes and draft concise commit message following this repo's style:
35+
- Look at `git log -10 --oneline` to match existing style
36+
- Be factual and direct (e.g., "Fix BATS dist test to create its own distribution")
37+
- Focus on "why" when it adds value, otherwise just describe "what"
38+
- List items in roughly decreasing order of impact
39+
- Keep related items grouped together
40+
- **In commit messages**: Wrap all code references in backticks - filenames, paths, commands, function names, variables, make targets, etc.
41+
- Examples: `helpers.bash`, `make test-recursion`, `setup_sequential_test()`, `TEST_REPO`, `.envs/`, `01-meta.bats`
42+
- Prevents markdown parsing issues and improves clarity
43+
44+
4. **PRESENT the proposed commit message to the user and WAIT for approval before proceeding**
45+
46+
5. After receiving approval, stage changes appropriately using `git add`
47+
48+
6. **VERIFY staged files with `git status`**:
49+
- If user did NOT specify a subset: Confirm ALL modified/untracked files are staged
50+
- If user specified only certain files: Confirm ONLY those files are staged
51+
- STOP and ask user if staging doesn't match intent
52+
53+
7. After verification, commit using `HEREDOC` format:
54+
```bash
55+
git commit -m "$(cat <<'EOF'
56+
Subject line (imperative mood, < 72 chars)
57+
58+
Additional context if needed, wrapped at 72 characters.
59+
60+
Co-Authored-By: Claude <noreply@anthropic.com>
61+
EOF
62+
)"
63+
```
64+
65+
8. Run `git status` after commit to verify success
66+
67+
9. If pre-commit hook modifies files: Check authorship (`git log -1 --format='%an %ae'`) and branch status, then amend if safe or create new commit
68+
69+
**REPOSITORY CONTEXT:**
70+
71+
This is pgxntool-test, a test harness for the pgxntool framework. Key facts:
72+
- Tests live in `tests/` directory
73+
- `.envs/` contains test environments (gitignored)
74+
75+
**RESTRICTIONS:**
76+
- DO NOT push unless explicitly asked
77+
- DO NOT commit files with actual secrets (`.env`, `credentials.json`, etc.)
78+
- Never use `-i` flags (`git commit -i`, `git rebase -i`, etc.)

pgxntool/.claude/settings.json

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
{
2+
"permissions": {
3+
"allow": [
4+
"Bash(cat:*)",
5+
"Bash(make test:*)",
6+
"Bash(tee:*)",
7+
"Bash(echo:*)",
8+
"Bash(git show:*)",
9+
"Bash(git log:*)",
10+
"Bash(ls:*)",
11+
"Bash(find:*)",
12+
"Bash(git checkout:*)",
13+
"Bash(head:*)"
14+
],
15+
"additionalDirectories": [
16+
"../pgxntool-test/"
17+
]
18+
}
19+
}

pgxntool/.gitattributes

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
11
.gitattributes export-ignore
2+
.claude/ export-ignore
3+
*.md export-ignore
4+
.DS_Store export-ignore
25
*.asc export-ignore
36
*.adoc export-ignore
47
*.html export-ignore

pgxntool/.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1 +1,2 @@
11
.*.swp
2+
.claude/*.local.json

pgxntool/CLAUDE.md

Lines changed: 255 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,255 @@
1+
# CLAUDE.md
2+
3+
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
4+
5+
## Git Commit Guidelines
6+
7+
**IMPORTANT**: When creating commit messages, do not attribute commits to yourself (Claude). Commit messages should reflect the work being done without AI attribution in the message body. The standard Co-Authored-By trailer is acceptable.
8+
9+
## Critical: What This Repo Actually Is
10+
11+
**pgxntool is NOT a standalone project.** It is a meta-framework that exists ONLY to be embedded into PostgreSQL extension projects via `git subtree`. This repo cannot be built, tested, or run directly.
12+
13+
**Think of it like this**: pgxntool is to PostgreSQL extensions what a Makefile template library is to C projects - it's infrastructure code that gets copied into other projects, not a project itself.
14+
15+
## Critical: Directory Purity - NO Temporary Files
16+
17+
**This directory contains ONLY files that get embedded into extension projects.** When extension developers run `git subtree add`, they pull the entire pgxntool directory into their project.
18+
19+
**ABSOLUTE RULE**: NO temporary files, scratch work, or development tools may be added to this directory.
20+
21+
**Examples of what NEVER belongs here:**
22+
- Temporary files (scratch notes, test output, debugging artifacts)
23+
- Development scripts or tools (these go in pgxntool-test/)
24+
- Planning documents (PLAN-*.md files go in pgxntool-test/)
25+
- Any file you wouldn't want in every extension project that uses pgxntool
26+
27+
**CLAUDE.md exception**: CLAUDE.md exists here for AI assistant guidance, but is excluded from distributions via `.gitattributes export-ignore`. Same with `.claude/` directory.
28+
29+
**Why this matters**: Any file you add here will be pulled into hundreds of extension projects via git subtree. Keep this directory lean and clean.
30+
31+
## Development Workflow: Work from pgxntool-test
32+
33+
**CRITICAL**: All development work on pgxntool should be done from the pgxntool-test repository, NOT from this repository.
34+
35+
**For complete development workflow documentation, see:**
36+
https://github.com/Postgres-Extensions/pgxntool-test
37+
38+
## Two-Repository Development Pattern
39+
40+
This codebase uses a two-repository pattern:
41+
42+
1. **pgxntool/** (this repo) - The framework code that gets embedded into extension projects
43+
2. **pgxntool-test** - The test harness that validates pgxntool functionality
44+
45+
**For development and testing workflow, see:**
46+
https://github.com/Postgres-Extensions/pgxntool-test
47+
48+
## How Extension Developers Use pgxntool
49+
50+
Extension projects include pgxntool via git subtree:
51+
52+
```bash
53+
git subtree add -P pgxntool --squash git@github.com:decibel/pgxntool.git release
54+
pgxntool/setup.sh
55+
```
56+
57+
After setup, their Makefile typically contains just:
58+
```makefile
59+
include pgxntool/base.mk
60+
```
61+
62+
## Architecture: Two-Phase Build System
63+
64+
### Phase 1: Meta Generation (`build_meta.sh`)
65+
- Processes `META.in.json` (template with placeholders/empty values)
66+
- Strips out X_comment fields and empty values
67+
- Produces clean `META.json`
68+
69+
### Phase 2: Variable Extraction (`meta.mk.sh`)
70+
- Parses `META.json` using `JSON.sh` (a bash-based JSON parser)
71+
- Generates `meta.mk` with Make variables:
72+
- `PGXN` - distribution name
73+
- `PGXNVERSION` - version number
74+
- `EXTENSIONS` - list of extensions provided
75+
- `EXTENSION_*_VERSION` - per-extension versions
76+
- `EXTENSION_VERSION_FILES` - auto-generated versioned SQL files
77+
- `base.mk` includes `meta.mk` via `-include`
78+
79+
### The Magic of base.mk
80+
81+
`base.mk` provides a complete PGXS-based build system:
82+
- Auto-detects extension SQL files in `sql/`
83+
- Auto-detects C modules in `src/*.c`
84+
- Auto-detects tests in `test/sql/*.sql`
85+
- Auto-generates versioned extension files (`extension--version.sql`)
86+
- Handles Asciidoc → HTML conversion
87+
- Integrates with PGXN distribution format
88+
- Manages git tagging and release packaging
89+
90+
## File Structure for Consumer Projects
91+
92+
Projects using pgxntool follow this layout:
93+
```
94+
project/
95+
├── Makefile # include pgxntool/base.mk
96+
├── META.in.json # Template metadata (customize for your extension)
97+
├── META.json # Auto-generated from META.in.json
98+
├── extension.control # Standard PostgreSQL control file
99+
├── pgxntool/ # This repo, embedded via git subtree
100+
├── sql/
101+
│ └── extension.sql # Base extension SQL
102+
├── src/ # Optional C code (*.c files)
103+
├── test/
104+
│ ├── deps.sql # Load extension and test dependencies
105+
│ ├── sql/*.sql # Test SQL files
106+
│ └── expected/*.out # Expected test outputs
107+
└── doc/ # Optional docs (*.adoc, *.asciidoc)
108+
```
109+
110+
## Commands for Extension Developers (End Users)
111+
112+
These are the commands extension developers use (documented for context):
113+
114+
```bash
115+
make # Build extension (generates versioned SQL, docs)
116+
make test # Full test: testdeps → install → installcheck → show diffs
117+
make results # Run tests and update expected output files
118+
make html # Generate HTML from Asciidoc sources
119+
make tag # Create git branch for current META.json version
120+
make dist # Create PGXN .zip (auto-tags, places in ../)
121+
make pgtle # Generate pg_tle registration SQL (see pg_tle Support below)
122+
make check-pgtle # Check pg_tle installation and report version
123+
make install-pgtle # Install pg_tle registration SQL files into database
124+
make pgxntool-sync # Update to latest pgxntool via git subtree pull
125+
```
126+
127+
## Testing with pgxntool
128+
129+
### Critical Testing Rules
130+
131+
**NEVER use `make installcheck` directly**. Always use `make test` instead. The `make test` target ensures:
132+
- Clean builds before testing
133+
- Proper test isolation
134+
- Correct test dependency installation
135+
- Proper cleanup and result comparison
136+
137+
**Database Connection Requirement**: PostgreSQL must be running before executing `make test`. If you get connection errors (e.g., "could not connect to server"), stop and ask the user to start PostgreSQL.
138+
139+
**Claude Code MUST NEVER run `make results`**. This target updates test expected output files and requires manual human verification of test changes before execution.
140+
141+
**Claude Code MUST NEVER modify files in `test/expected/`**. These are expected test outputs that define correct behavior and must only be updated through the `make results` workflow.
142+
143+
The workflow is:
144+
1. Human runs `make test` and examines diffs
145+
2. Human manually verifies changes are correct
146+
3. Human manually runs `make results` to update expected files
147+
148+
### Test Output Mechanics
149+
150+
pgxntool uses PostgreSQL's pg_regress test framework:
151+
- **Actual test output**: Written to `test/results/` directory
152+
- **Expected output**: Stored in `test/expected/` directory
153+
- **Test comparison**: pg_regress compares actual vs expected and generates diffs; `make test` displays them
154+
- **Updating expectations**: `make results` copies `test/results/``test/expected/`
155+
156+
When tests fail, examine the diff output carefully. The actual test output in `test/results/` shows what your code produced, while `test/expected/` shows what was expected.
157+
158+
## Key Implementation Details
159+
160+
### PostgreSQL Version Handling
161+
- `MAJORVER` = version × 10 (e.g., 9.6 → 96, 13 → 130)
162+
- Tests use `--load-language=plpgsql` for versions < 13
163+
- Version detection via `pg_config --version`
164+
165+
### Test System (pg_regress based)
166+
- Tests in `test/sql/*.sql`, outputs compared to `test/expected/*.out`
167+
- Setup via `test/pgxntool/setup.sql` (loads pgTap and deps.sql)
168+
- `.IGNORE: installcheck` allows `make test` to handle errors (show diffs, then exit with error status)
169+
- `make results` updates expected outputs after test runs
170+
171+
### Document Generation
172+
- Auto-detects `asciidoctor` or `asciidoc`
173+
- Generates HTML from `*.adoc` and `*.asciidoc` in `$(DOC_DIRS)`
174+
- HTML required for `make dist`, optional for `make install`
175+
- Template-based rules via `ASCIIDOC_template`
176+
177+
### Distribution Packaging
178+
- `make dist` creates `../PGXN-VERSION.zip`
179+
- Always creates git branch tag matching version
180+
- Uses `git archive` to package
181+
- Validates repo is clean before tagging
182+
183+
### Subtree Sync Support
184+
- `make pgxntool-sync` pulls latest release
185+
- Multiple sync targets: release, stable, local variants
186+
- Uses `git subtree pull --squash`
187+
- Requires clean repo (no uncommitted changes)
188+
189+
### pg_tle Support
190+
191+
pgxntool can generate pg_tle (Trusted Language Extensions) registration SQL for deploying extensions in AWS RDS/Aurora without filesystem access.
192+
193+
**Usage:** `make pgtle` or `make pgtle PGTLE_VERSION=1.5.0+`
194+
195+
**Output:** `pg_tle/{version_range}/{extension}.sql`
196+
197+
**For version range details and API compatibility boundaries, see:** `pgtle_versions.md`
198+
199+
**Installation targets:**
200+
201+
- `make check-pgtle` - Checks if pg_tle is installed and reports the version. Reports version from `pg_extension` if extension has been created, or newest available version from `pg_available_extension_versions` if available but not created. Errors if pg_tle not available in cluster. Assumes `PG*` environment variables are configured.
202+
203+
- `make install-pgtle` - Auto-detects pg_tle version and installs appropriate registration SQL files. Updates or creates pg_tle extension as needed. Determines which version range files to install based on detected version. Runs all generated SQL files via `psql` to register extensions with pg_tle. Assumes `PG*` environment variables are configured.
204+
205+
**Version notation:**
206+
- `X.Y.Z+` means >= X.Y.Z
207+
- `X.Y.Z-A.B.C` means >= X.Y.Z and < A.B.C (note boundary)
208+
209+
**Key implementation details:**
210+
- Script: `pgxntool/pgtle-wrap.sh` (bash)
211+
- Parses `.control` files for metadata (NOT META.json)
212+
- Fixed delimiter: `$_pgtle_wrap_delimiter_$` (validated not in source)
213+
- Each output file contains ALL versions and ALL upgrade paths
214+
- Multi-extension support (multiple .control files)
215+
- Output directory `pg_tle/` excluded from git
216+
- Depends on `make all` to ensure versioned SQL files exist first
217+
- Only processes versioned files (`sql/{ext}--{version}.sql`), not base files
218+
219+
**SQL file handling:**
220+
- **Version files** (`sql/{ext}--{version}.sql`): Generated automatically by `make all` from base `sql/{ext}.sql` file
221+
- **Upgrade scripts** (`sql/{ext}--{v1}--{v2}.sql`): Created manually by users when adding new extension versions
222+
- The script ensures the default_version file exists if the base file exists (creates it from base file if missing)
223+
- All version files and upgrade scripts are discovered and included in the generated pg_tle registration SQL
224+
225+
**Dependencies:**
226+
Generated files depend on:
227+
- Control file (metadata source)
228+
- All SQL files (sql/{ext}--*.sql) - must run `make all` first
229+
- Generator script itself
230+
231+
**Limitations:**
232+
- No C code support (pg_tle requires trusted languages only)
233+
- PostgreSQL 14.5+ required (pg_tle not available on earlier versions)
234+
235+
## Critical Gotchas
236+
237+
1. **Empty Variables**: If `DOCS` or `MODULES` is empty, base.mk sets to empty to prevent PGXS errors
238+
2. **testdeps Pattern**: Never add recipes to `testdeps` - create separate target and make it a prerequisite
239+
3. **META.json is Generated**: Always edit `META.in.json`, never `META.json` directly
240+
4. **Control File Versions**: No automatic validation that `.control` matches `META.json` version
241+
5. **PGXNTOOL_NO_PGXS_INCLUDE**: Setting this skips PGXS inclusion (for special scenarios)
242+
6. **Distribution Placement**: `.zip` files go in parent directory (`../`) to avoid repo clutter
243+
244+
## Scripts
245+
246+
- **setup.sh** - Initializes pgxntool in a new extension project (copies templates, creates directories)
247+
- **build_meta.sh** - Strips empty fields from META.in.json to create META.json
248+
- **meta.mk.sh** - Parses META.json via JSON.sh and generates meta.mk with Make variables
249+
- **JSON.sh** - Third-party bash JSON parser (MIT licensed)
250+
- **safesed** - Utility for safe sed operations
251+
252+
## Related Repositories
253+
254+
- **pgxntool-test** - Test harness for validating pgxntool functionality: https://github.com/Postgres-Extensions/pgxntool-test
255+
- Never produce any kind of metrics or estimates unless you have data to back them up. If you do have data you MUST reference it.

pgxntool/HISTORY.asc

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,14 @@
1-
STABLE
2-
------
1+
1.0.0
2+
-----
3+
== Fix broken multi-extension support
4+
Prior to this fix, distributions with multiple extensions or extensions with versions different from the PGXN distribution version were completely broken. Extension versions are now correctly read from each `.control` file's `default_version` instead of using META.json's distribution version.
5+
6+
== Add pg_tle support
7+
New `make pgtle` target generates pg_tle registration SQL for extensions. Supports pg_tle version ranges (1.0.0-1.4.0, 1.4.0-1.5.0, 1.5.0+) with appropriate API calls for each range. See README for usage.
8+
9+
== Use git tags for distribution versioning
10+
The `tag` and `rmtag` targets now create/delete git tags instead of branches.
11+
312
== Support 13+
413
The `--load-language` option was removed from `pg_regress` in 13.
514

pgxntool/LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
Copyright (c) 2015, Jim Nasby, Blue Treble Solutions
1+
Copyright (c) 2015-2026, Jim Nasby, Blue Treble Solutions
22
All rights reserved.
33

44
Redistribution and use in source and binary forms, with or without

0 commit comments

Comments
 (0)