|
| 1 | +# CLAUDE.md |
| 2 | + |
| 3 | +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. |
| 4 | + |
| 5 | +## Git Commit Guidelines |
| 6 | + |
| 7 | +**IMPORTANT**: When creating commit messages, do not attribute commits to yourself (Claude). Commit messages should reflect the work being done without AI attribution in the message body. The standard Co-Authored-By trailer is acceptable. |
| 8 | + |
| 9 | +## Critical: What This Repo Actually Is |
| 10 | + |
| 11 | +**pgxntool is NOT a standalone project.** It is a meta-framework that exists ONLY to be embedded into PostgreSQL extension projects via `git subtree`. This repo cannot be built, tested, or run directly. |
| 12 | + |
| 13 | +**Think of it like this**: pgxntool is to PostgreSQL extensions what a Makefile template library is to C projects - it's infrastructure code that gets copied into other projects, not a project itself. |
| 14 | + |
| 15 | +## Critical: Directory Purity - NO Temporary Files |
| 16 | + |
| 17 | +**This directory contains ONLY files that get embedded into extension projects.** When extension developers run `git subtree add`, they pull the entire pgxntool directory into their project. |
| 18 | + |
| 19 | +**ABSOLUTE RULE**: NO temporary files, scratch work, or development tools may be added to this directory. |
| 20 | + |
| 21 | +**Examples of what NEVER belongs here:** |
| 22 | +- Temporary files (scratch notes, test output, debugging artifacts) |
| 23 | +- Development scripts or tools (these go in pgxntool-test/) |
| 24 | +- Planning documents (PLAN-*.md files go in pgxntool-test/) |
| 25 | +- Any file you wouldn't want in every extension project that uses pgxntool |
| 26 | + |
| 27 | +**CLAUDE.md exception**: CLAUDE.md exists here for AI assistant guidance, but is excluded from distributions via `.gitattributes export-ignore`. Same with `.claude/` directory. |
| 28 | + |
| 29 | +**Why this matters**: Any file you add here will be pulled into hundreds of extension projects via git subtree. Keep this directory lean and clean. |
| 30 | + |
| 31 | +## Development Workflow: Work from pgxntool-test |
| 32 | + |
| 33 | +**CRITICAL**: All development work on pgxntool should be done from the pgxntool-test repository, NOT from this repository. |
| 34 | + |
| 35 | +**For complete development workflow documentation, see:** |
| 36 | +https://github.com/Postgres-Extensions/pgxntool-test |
| 37 | + |
| 38 | +## Two-Repository Development Pattern |
| 39 | + |
| 40 | +This codebase uses a two-repository pattern: |
| 41 | + |
| 42 | +1. **pgxntool/** (this repo) - The framework code that gets embedded into extension projects |
| 43 | +2. **pgxntool-test** - The test harness that validates pgxntool functionality |
| 44 | + |
| 45 | +**For development and testing workflow, see:** |
| 46 | +https://github.com/Postgres-Extensions/pgxntool-test |
| 47 | + |
| 48 | +## How Extension Developers Use pgxntool |
| 49 | + |
| 50 | +Extension projects include pgxntool via git subtree: |
| 51 | + |
| 52 | +```bash |
| 53 | +git subtree add -P pgxntool --squash git@github.com:decibel/pgxntool.git release |
| 54 | +pgxntool/setup.sh |
| 55 | +``` |
| 56 | + |
| 57 | +After setup, their Makefile typically contains just: |
| 58 | +```makefile |
| 59 | +include pgxntool/base.mk |
| 60 | +``` |
| 61 | + |
| 62 | +## Architecture: Two-Phase Build System |
| 63 | + |
| 64 | +### Phase 1: Meta Generation (`build_meta.sh`) |
| 65 | +- Processes `META.in.json` (template with placeholders/empty values) |
| 66 | +- Strips out X_comment fields and empty values |
| 67 | +- Produces clean `META.json` |
| 68 | + |
| 69 | +### Phase 2: Variable Extraction (`meta.mk.sh`) |
| 70 | +- Parses `META.json` using `JSON.sh` (a bash-based JSON parser) |
| 71 | +- Generates `meta.mk` with Make variables: |
| 72 | + - `PGXN` - distribution name |
| 73 | + - `PGXNVERSION` - version number |
| 74 | + - `EXTENSIONS` - list of extensions provided |
| 75 | + - `EXTENSION_*_VERSION` - per-extension versions |
| 76 | + - `EXTENSION_VERSION_FILES` - auto-generated versioned SQL files |
| 77 | +- `base.mk` includes `meta.mk` via `-include` |
| 78 | + |
| 79 | +### The Magic of base.mk |
| 80 | + |
| 81 | +`base.mk` provides a complete PGXS-based build system: |
| 82 | +- Auto-detects extension SQL files in `sql/` |
| 83 | +- Auto-detects C modules in `src/*.c` |
| 84 | +- Auto-detects tests in `test/sql/*.sql` |
| 85 | +- Auto-generates versioned extension files (`extension--version.sql`) |
| 86 | +- Handles Asciidoc → HTML conversion |
| 87 | +- Integrates with PGXN distribution format |
| 88 | +- Manages git tagging and release packaging |
| 89 | + |
| 90 | +## File Structure for Consumer Projects |
| 91 | + |
| 92 | +Projects using pgxntool follow this layout: |
| 93 | +``` |
| 94 | +project/ |
| 95 | +├── Makefile # include pgxntool/base.mk |
| 96 | +├── META.in.json # Template metadata (customize for your extension) |
| 97 | +├── META.json # Auto-generated from META.in.json |
| 98 | +├── extension.control # Standard PostgreSQL control file |
| 99 | +├── pgxntool/ # This repo, embedded via git subtree |
| 100 | +├── sql/ |
| 101 | +│ └── extension.sql # Base extension SQL |
| 102 | +├── src/ # Optional C code (*.c files) |
| 103 | +├── test/ |
| 104 | +│ ├── deps.sql # Load extension and test dependencies |
| 105 | +│ ├── sql/*.sql # Test SQL files |
| 106 | +│ └── expected/*.out # Expected test outputs |
| 107 | +└── doc/ # Optional docs (*.adoc, *.asciidoc) |
| 108 | +``` |
| 109 | + |
| 110 | +## Commands for Extension Developers (End Users) |
| 111 | + |
| 112 | +These are the commands extension developers use (documented for context): |
| 113 | + |
| 114 | +```bash |
| 115 | +make # Build extension (generates versioned SQL, docs) |
| 116 | +make test # Full test: testdeps → install → installcheck → show diffs |
| 117 | +make results # Run tests and update expected output files |
| 118 | +make html # Generate HTML from Asciidoc sources |
| 119 | +make tag # Create git branch for current META.json version |
| 120 | +make dist # Create PGXN .zip (auto-tags, places in ../) |
| 121 | +make pgtle # Generate pg_tle registration SQL (see pg_tle Support below) |
| 122 | +make check-pgtle # Check pg_tle installation and report version |
| 123 | +make install-pgtle # Install pg_tle registration SQL files into database |
| 124 | +make pgxntool-sync # Update to latest pgxntool via git subtree pull |
| 125 | +``` |
| 126 | + |
| 127 | +## Testing with pgxntool |
| 128 | + |
| 129 | +### Critical Testing Rules |
| 130 | + |
| 131 | +**NEVER use `make installcheck` directly**. Always use `make test` instead. The `make test` target ensures: |
| 132 | +- Clean builds before testing |
| 133 | +- Proper test isolation |
| 134 | +- Correct test dependency installation |
| 135 | +- Proper cleanup and result comparison |
| 136 | + |
| 137 | +**Database Connection Requirement**: PostgreSQL must be running before executing `make test`. If you get connection errors (e.g., "could not connect to server"), stop and ask the user to start PostgreSQL. |
| 138 | + |
| 139 | +**Claude Code MUST NEVER run `make results`**. This target updates test expected output files and requires manual human verification of test changes before execution. |
| 140 | + |
| 141 | +**Claude Code MUST NEVER modify files in `test/expected/`**. These are expected test outputs that define correct behavior and must only be updated through the `make results` workflow. |
| 142 | + |
| 143 | +The workflow is: |
| 144 | +1. Human runs `make test` and examines diffs |
| 145 | +2. Human manually verifies changes are correct |
| 146 | +3. Human manually runs `make results` to update expected files |
| 147 | + |
| 148 | +### Test Output Mechanics |
| 149 | + |
| 150 | +pgxntool uses PostgreSQL's pg_regress test framework: |
| 151 | +- **Actual test output**: Written to `test/results/` directory |
| 152 | +- **Expected output**: Stored in `test/expected/` directory |
| 153 | +- **Test comparison**: pg_regress compares actual vs expected and generates diffs; `make test` displays them |
| 154 | +- **Updating expectations**: `make results` copies `test/results/` → `test/expected/` |
| 155 | + |
| 156 | +When tests fail, examine the diff output carefully. The actual test output in `test/results/` shows what your code produced, while `test/expected/` shows what was expected. |
| 157 | + |
| 158 | +## Key Implementation Details |
| 159 | + |
| 160 | +### PostgreSQL Version Handling |
| 161 | +- `MAJORVER` = version × 10 (e.g., 9.6 → 96, 13 → 130) |
| 162 | +- Tests use `--load-language=plpgsql` for versions < 13 |
| 163 | +- Version detection via `pg_config --version` |
| 164 | + |
| 165 | +### Test System (pg_regress based) |
| 166 | +- Tests in `test/sql/*.sql`, outputs compared to `test/expected/*.out` |
| 167 | +- Setup via `test/pgxntool/setup.sql` (loads pgTap and deps.sql) |
| 168 | +- `.IGNORE: installcheck` allows `make test` to handle errors (show diffs, then exit with error status) |
| 169 | +- `make results` updates expected outputs after test runs |
| 170 | + |
| 171 | +### Document Generation |
| 172 | +- Auto-detects `asciidoctor` or `asciidoc` |
| 173 | +- Generates HTML from `*.adoc` and `*.asciidoc` in `$(DOC_DIRS)` |
| 174 | +- HTML required for `make dist`, optional for `make install` |
| 175 | +- Template-based rules via `ASCIIDOC_template` |
| 176 | + |
| 177 | +### Distribution Packaging |
| 178 | +- `make dist` creates `../PGXN-VERSION.zip` |
| 179 | +- Always creates git branch tag matching version |
| 180 | +- Uses `git archive` to package |
| 181 | +- Validates repo is clean before tagging |
| 182 | + |
| 183 | +### Subtree Sync Support |
| 184 | +- `make pgxntool-sync` pulls latest release |
| 185 | +- Multiple sync targets: release, stable, local variants |
| 186 | +- Uses `git subtree pull --squash` |
| 187 | +- Requires clean repo (no uncommitted changes) |
| 188 | + |
| 189 | +### pg_tle Support |
| 190 | + |
| 191 | +pgxntool can generate pg_tle (Trusted Language Extensions) registration SQL for deploying extensions in AWS RDS/Aurora without filesystem access. |
| 192 | + |
| 193 | +**Usage:** `make pgtle` or `make pgtle PGTLE_VERSION=1.5.0+` |
| 194 | + |
| 195 | +**Output:** `pg_tle/{version_range}/{extension}.sql` |
| 196 | + |
| 197 | +**For version range details and API compatibility boundaries, see:** `pgtle_versions.md` |
| 198 | + |
| 199 | +**Installation targets:** |
| 200 | + |
| 201 | +- `make check-pgtle` - Checks if pg_tle is installed and reports the version. Reports version from `pg_extension` if extension has been created, or newest available version from `pg_available_extension_versions` if available but not created. Errors if pg_tle not available in cluster. Assumes `PG*` environment variables are configured. |
| 202 | + |
| 203 | +- `make install-pgtle` - Auto-detects pg_tle version and installs appropriate registration SQL files. Updates or creates pg_tle extension as needed. Determines which version range files to install based on detected version. Runs all generated SQL files via `psql` to register extensions with pg_tle. Assumes `PG*` environment variables are configured. |
| 204 | + |
| 205 | +**Version notation:** |
| 206 | +- `X.Y.Z+` means >= X.Y.Z |
| 207 | +- `X.Y.Z-A.B.C` means >= X.Y.Z and < A.B.C (note boundary) |
| 208 | + |
| 209 | +**Key implementation details:** |
| 210 | +- Script: `pgxntool/pgtle-wrap.sh` (bash) |
| 211 | +- Parses `.control` files for metadata (NOT META.json) |
| 212 | +- Fixed delimiter: `$_pgtle_wrap_delimiter_$` (validated not in source) |
| 213 | +- Each output file contains ALL versions and ALL upgrade paths |
| 214 | +- Multi-extension support (multiple .control files) |
| 215 | +- Output directory `pg_tle/` excluded from git |
| 216 | +- Depends on `make all` to ensure versioned SQL files exist first |
| 217 | +- Only processes versioned files (`sql/{ext}--{version}.sql`), not base files |
| 218 | + |
| 219 | +**SQL file handling:** |
| 220 | +- **Version files** (`sql/{ext}--{version}.sql`): Generated automatically by `make all` from base `sql/{ext}.sql` file |
| 221 | +- **Upgrade scripts** (`sql/{ext}--{v1}--{v2}.sql`): Created manually by users when adding new extension versions |
| 222 | +- The script ensures the default_version file exists if the base file exists (creates it from base file if missing) |
| 223 | +- All version files and upgrade scripts are discovered and included in the generated pg_tle registration SQL |
| 224 | + |
| 225 | +**Dependencies:** |
| 226 | +Generated files depend on: |
| 227 | +- Control file (metadata source) |
| 228 | +- All SQL files (sql/{ext}--*.sql) - must run `make all` first |
| 229 | +- Generator script itself |
| 230 | + |
| 231 | +**Limitations:** |
| 232 | +- No C code support (pg_tle requires trusted languages only) |
| 233 | +- PostgreSQL 14.5+ required (pg_tle not available on earlier versions) |
| 234 | + |
| 235 | +## Critical Gotchas |
| 236 | + |
| 237 | +1. **Empty Variables**: If `DOCS` or `MODULES` is empty, base.mk sets to empty to prevent PGXS errors |
| 238 | +2. **testdeps Pattern**: Never add recipes to `testdeps` - create separate target and make it a prerequisite |
| 239 | +3. **META.json is Generated**: Always edit `META.in.json`, never `META.json` directly |
| 240 | +4. **Control File Versions**: No automatic validation that `.control` matches `META.json` version |
| 241 | +5. **PGXNTOOL_NO_PGXS_INCLUDE**: Setting this skips PGXS inclusion (for special scenarios) |
| 242 | +6. **Distribution Placement**: `.zip` files go in parent directory (`../`) to avoid repo clutter |
| 243 | + |
| 244 | +## Scripts |
| 245 | + |
| 246 | +- **setup.sh** - Initializes pgxntool in a new extension project (copies templates, creates directories) |
| 247 | +- **build_meta.sh** - Strips empty fields from META.in.json to create META.json |
| 248 | +- **meta.mk.sh** - Parses META.json via JSON.sh and generates meta.mk with Make variables |
| 249 | +- **JSON.sh** - Third-party bash JSON parser (MIT licensed) |
| 250 | +- **safesed** - Utility for safe sed operations |
| 251 | + |
| 252 | +## Related Repositories |
| 253 | + |
| 254 | +- **pgxntool-test** - Test harness for validating pgxntool functionality: https://github.com/Postgres-Extensions/pgxntool-test |
| 255 | +- Never produce any kind of metrics or estimates unless you have data to back them up. If you do have data you MUST reference it. |
0 commit comments