Refine XDMF compat groups: field projection, standalone Vec I/O, tensor repacking

CLAUDE.md

@@ -220,11 +220,20 @@ Underworld development team with AI support from Claude Code
 - Requires complete rebuild (~1 hour) if relocated
 
 ### Rebuild After Source Changes
-**After modifying source files, always run `pixi run underworld-build`!**
+**After modifying source files, always run `./uw build`!**
 - Underworld3 is installed as a package in the pixi environment
 - Changes go to `.pixi/envs/default/lib/python3.12/site-packages/underworld3/`
 - Verify with `uw.model.__file__`
 
+**⚠️ STALE BUILD CACHE**: If `./uw build` succeeds but Python still uses old code
+(e.g. a new parameter is "unknown"), pip's wheel cache is stale. Fix with:
+```bash
+rm -rf build/lib.* build/bdist.*
+pixi run -e default pip install --no-build-isolation --force-reinstall --no-deps .
+```
+This is the most common build issue — `./uw build` reuses cached wheels when the
+version number hasn't changed. Always verify changes are installed before debugging.
+
 ### Test Quality Principles
 **New tests must be validated before making code changes to fix them!**
 - Validate test correctness before changing main code
@@ -534,10 +543,10 @@ When working on specific subsystems, these documents provide detailed guidance.
 
 ## Quick Reference
 
-### Pixi Commands
+### Build & Test Commands
 ```bash
-pixi run underworld-build    # Rebuild after source changes
-pixi run underworld-test     # Run test suite
+./uw build                   # Rebuild after source changes (preferred)
+./uw test                    # Run test suite
 pixi run -e default python   # Run Python in environment
 ```
 

docs/beginner/create_xdmf.md

@@ -4,7 +4,7 @@ title: "Create XDMF from PETSc HDF5 Output"
 
 # Create XDMF from PETSc HDF5 Output
 
-This guide explains how to generate ParaView-ready XDMF/HDF5 outputs with `mesh.write_timestep(create_xdmf=True)`, why this is needed with newer PETSc HDF5 layouts, and how tensors are converted for correct ParaView interpretation.
+This guide explains the ParaView-ready XDMF/HDF5 outputs generated by `mesh.write_timestep()` (enabled by default via `create_xdmf=True`), why this is needed with newer PETSc HDF5 layouts, and how tensors are converted for correct ParaView interpretation.
 
 This note explains the output-format issue seen with newer PETSc versions and the fix now available in `Mesh.write_timestep()`.
 
@@ -79,13 +79,13 @@ So, `/fields` is useful raw data, but it is not always directly visualization-re
 
 ## 3. What changed in `write_timestep()` and why
 
-`Mesh.write_timestep()` now supports:
+`Mesh.write_timestep()` generates XDMF by default (`create_xdmf=True`):
 
 ```python
-mesh.write_timestep(..., create_xdmf=True)
+mesh.write_timestep(...)  # XDMF enabled by default
 ```
 
-When `create_xdmf=True`:
+When `create_xdmf=True` (the default):
 
 1. It writes compatibility datasets per variable:
    - node-like vars -> `/vertex_fields/coordinates` and `/vertex_fields/<name>_<name>`
@@ -120,27 +120,44 @@ Then:
 - `is_cell == True`  -> write `/cell_fields/<name>_<name>` and XDMF `Center="Cell"`
 - `is_cell == False` -> write `/vertex_fields/<name>_<name>` and XDMF `Center="Node"`
 
-### How `/fields` data is remapped (KDTree)
+### How vertex data is obtained
 
-When a variable is vertex-centered but `/fields` does not already match mesh vertex count:
+The compatibility group writer uses PETSc's `createInterpolation` to project
+data to mesh vertices:
 
-1. Source coordinates/values are read from `/fields/coordinates` and `/fields/<name>`.
-2. Packed high-order layouts are unpacked into point-wise coordinate/value rows.
-3. A nearest-neighbor search is done with `uw.kdtree.KDTree` to map source points to mesh vertices.
-4. The mapped values are written to `/vertex_fields/<name>_<name>`.
+- **P1 continuous variables**: DOFs match mesh vertices exactly, so the owned
+  global-vector data is written directly to `/vertex_fields/` — no interpolation needed.
+- **P2+ continuous variables**: a scratch DM at degree 1 is created, PETSc builds
+  the interpolation matrix, and the projected global vector is written. This gives
+  exact polynomial-consistent values at vertex positions.
+- **Cell (discontinuous / degree-0) variables**: owned global-vector data is written
+  directly to `/cell_fields/`.
 
-For cell-centered variables, values are written into `/cell_fields/<name>_<name>` using row slices.
+### Tensor repacking
 
-### Parallel execution details
+Tensor variables are repacked from UW3's internal `_data_layout` storage order to
+ParaView's 9-component row-major 3x3 format before writing to the compatibility
+groups.  The checkpoint data in `/fields/` is always stored in UW3's native ordering.
+
+UW3 `_data_layout` storage (different from ParaView's expected order):
 
-This remapping/writing path is parallel-aware:
+| Type | 2D components | 3D components |
+|------|--------------|--------------|
+| TENSOR | `[xx, xy, yx, yy]` | `[xx, xy, xz, yx, yy, yz, zx, zy, zz]` (same as ParaView) |
+| SYM_TENSOR | `[xx, yy, xy]` | `[xx, yy, zz, xy, xz, yz]` |
 
-- work is partitioned by MPI rank (each rank owns a slice of vertices / rows)
-- each rank computes mapping for its local slice
-- output datasets are written by per-rank slices (no overlapping writes)
-- with MPI-enabled `h5py`, HDF5 writes use parallel I/O; otherwise a serial fallback is used
+```{note}
+The original PR #69 assumed 2D TENSOR stored `[xx, yy, xy, yx]`.
+The corrected implementation uses the actual `_data_layout` row-major
+ordering `[xx, xy, yx, yy]`.
+```
+
+### Parallel execution details
 
-So the compatibility-field generation is computed and written in parallel when MPI-HDF5 is available.
+All I/O uses PETSc's parallel `ViewerHDF5`.  Data is written as standalone Vecs
+(not DM-associated) with `setBlockSize()` to preserve the `(N, ncomp)` dataset
+shape.  This is necessary because DM-associated Vecs ignore `pushGroup()` — the
+DMPlex HDF5 writer internally redirects to `/fields/`.
 
 ## 4. Tensor representation for ParaView (2D and 3D)