NVIDIA-NeMo · johnnygreco · Jan 22, 2026 · Jan 22, 2026 · Jan 22, 2026 · Jan 22, 2026
@@ -0,0 +1,132 @@
+---
+name: create-pr
+description: Create a GitHub PR with a well-formatted description including summary, categorized changes, and attention areas
+argument-hint: [special instructions]
+disable-model-invocation: true
+---
+
+# Create Pull Request
+
+Create a well-formatted GitHub pull request for the current branch.
+
+## Arguments
+
+`$ARGUMENTS` can be used for special instructions, such as:
+- Specifying a base branch: "use base branch: develop"
+- Guiding the summary: "emphasize the performance improvements in the summary"
+- Adding context: "this is part of the auth refactor epic"
+- Any other guidance for PR creation
+
+Default base branch: `main` (unless specified in arguments)
+
+## Step 1: Gather Information
+
+Run these commands in parallel to understand the changes:
+
+1. **Current branch**: `git branch --show-current`
+2. **Uncommitted changes**: `git status --porcelain`
+3. **Commits on branch**: `git log origin/main..HEAD --oneline`
+4. **File changes summary**: `git diff --stat origin/main..HEAD`
+5. **Full diff**: `git diff origin/main..HEAD`
+6. **Recent commit style**: `git log -5 --oneline` (to match PR title convention)
+
+**Important checks:**
+- If uncommitted changes exist, warn the user and ask if they want to commit first
+- If no commits ahead of main, inform the user there's nothing to PR
+- If branch isn't pushed, you'll push it in Step 4
+
+## Step 2: Analyze and Categorize Changes
+
+### By Change Type (from commits and diff)
+- ✨ **Added**: New files, features, capabilities
+- 🔧 **Changed**: Modified existing functionality
+- 🗑️ **Removed**: Deleted files or features
+- 🐛 **Fixed**: Bug fixes
+- 📚 **Docs**: Documentation updates
+- 🧪 **Tests**: Test additions/modifications
+
+### Identify Attention Areas 🔍
+Flag for special reviewer attention:
+- Files with significant changes (>100 lines)
+- Changes to base classes, interfaces, or public API
+- New dependencies (`pyproject.toml`, `requirements.txt`)
+- Configuration schema changes
+- Security-related changes
+
+## Step 3: Generate PR Title
+
+Use conventional commit format matching the repo style:
+- `feat:` for new features
+- `fix:` for bug fixes
+- `docs:` for documentation
+- `refactor:` for refactoring
+- `chore:` for maintenance
+- `test:` for test changes
+
+If commits have mixed types, use the primary/most significant type.
+
+## Step 4: Create the PR
+
+1. **Push branch** (if needed):
+   ```bash
+   git push -u origin <branch-name>
+   ```
+
+2. **Create PR** using this template:
+
+```markdown
+## 📋 Summary
+
+[1-2 sentence overview of what this PR accomplishes]
+
+## 🔄 Changes
+
+### ✨ Added
+- [New features/files - link to key files when helpful]
+
+### 🔧 Changed
+- [Modified functionality - reference commits for specific changes]
+
+### 🗑️ Removed
+- [Deleted items]
+
+### 🐛 Fixed
+- [Bug fixes - if applicable]
+
+## 🔍 Attention Areas
+
+> ⚠️ **Reviewers:** Please pay special attention to the following:
+
+- `path/to/critical/file.py` - [Why this needs attention]
+
+---
+🤖 *Generated with AI*
+```
+
+3. **Execute**:
+   ```bash
+   gh pr create --title "<title>" --body "$(cat <<'EOF'
+   <body>
+   EOF
+   )"
+   ```
+
+4. **Return the PR URL** to the user.
+
+## Section Guidelines
+
+- **Summary**: Always include - be concise and focus on the "why"
+- **Changes**: Group by type, omit empty sections
+- **Attention Areas**: Only include if there are genuinely important items; omit for simple PRs
+- **Links**: Include links to code and commits where helpful for reviewers:
+  - Link to specific files: `[filename](path/to/file.py)` - GitHub auto-links files in the repo
+  - Link to specific lines: `[description](path/to/file.py#L42-L50)` for key code sections
+  - Reference commits: `abc1234` - GitHub auto-links commit SHAs
+  - For multi-commit PRs, reference individual commits when describing specific changes
+
+## Edge Cases
+
+- **No changes**: Inform user there's nothing to create a PR for
+- **Uncommitted work**: Warn and ask before proceeding
+- **Large PRs** (>20 files): Summarize by directory/module
+- **Single commit**: PR title can match commit message
@@ -1,7 +1,8 @@
 ---
-name: new-sdg 
+name: new-sdg
 description: Implement a new synthetic data generator using NeMo Data Designer by defining its configuration and executing a preview job.
 argument-hint: <dataset-description>
+disable-model-invocation: true
 ---
 
 # Your Goal

@@ -24,7 +24,7 @@ jobs:
       - name: Set up Python
         run: uv python install 3.11
       - name: Install dependencies for docs
-        run: uv sync --group docs
+        run: uv sync --all-packages --group docs
       - name: Download artifact from previous step
         uses: actions/download-artifact@v5
         with:

@@ -28,8 +28,7 @@ jobs:
           enable-cache: true
 
       - name: Install dependencies
-        run: |
-          uv sync --group notebooks --group docs
+        run: uv sync --all-packages --group notebooks --group docs
 
       - name: Generate Colab notebooks
         run: |

@@ -8,8 +8,13 @@ on:
   workflow_dispatch:
 
 jobs:
-  test:
-    name: Test (Python ${{ matrix.python-version }} on ${{ matrix.os }})
+  # ===========================================================================
+  # Independent Package Tests
+  # Each package is tested in isolation to ensure proper dependency boundaries
+  # ===========================================================================
+
+  test-config:
+    name: Test Config (Python ${{ matrix.python-version }} on ${{ matrix.os }})
     runs-on: ${{ matrix.os }}
     strategy:
       fail-fast: false
@@ -28,13 +33,120 @@ jobs:
           python-version: ${{ matrix.python-version }}
           enable-cache: true
 
-      - name: Install dependencies
+      - name: Install data-designer-config only
+        run: uv sync --package data-designer-config
+
+      - name: Run config tests
+        run: |
+          uv run --with pytest --with pytest-cov --with pytest-asyncio --with pytest-env \
+            pytest -v \
+            packages/data-designer-config/tests \
+            --cov=data_designer \
+            --cov-report=term-missing
+
+  test-engine:
+    name: Test Engine (Python ${{ matrix.python-version }} on ${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          version: "latest"
+          python-version: ${{ matrix.python-version }}
+          enable-cache: true
+
+      - name: Install data-designer-engine only
+        run: uv sync --package data-designer-engine
+
+      - name: Run engine tests
+        run: |
+          uv run --with pytest --with pytest-cov --with pytest-asyncio --with pytest-httpx --with pytest-env \
+            pytest -v \
+            packages/data-designer-engine/tests \
+            --cov=data_designer \
+            --cov-report=term-missing
+
+  test-interface:
+    name: Test Interface (Python ${{ matrix.python-version }} on ${{ matrix.os }})
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          version: "latest"
+          python-version: ${{ matrix.python-version }}
+          enable-cache: true
+
+      - name: Install data-designer (full package)
+        run: uv sync --package data-designer
+
+      - name: Run interface tests
         run: |
-          uv sync --group dev
+          uv run --with pytest --with pytest-cov --with pytest-asyncio --with pytest-httpx --with pytest-env \
+            pytest -v \
+            packages/data-designer/tests \
+            --cov=data_designer \
+            --cov-report=term-missing
+
+  # ===========================================================================
+  # Combined Coverage Check
+  # Runs all tests together to verify overall coverage threshold
+  # ===========================================================================
+
+  coverage:
+    name: Coverage Check (Python ${{ matrix.python-version }})
+    runs-on: ubuntu-latest
+    strategy:
+      fail-fast: false
+      matrix:
+        python-version: ["3.11"]
+
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Install uv
+        uses: astral-sh/setup-uv@v5
+        with:
+          version: "latest"
+          python-version: ${{ matrix.python-version }}
+          enable-cache: true
+
+      - name: Install all packages
+        run: make install-dev
 
       - name: Run tests with coverage
         run: |
-          uv run pytest -v --cov=data_designer --cov-report=term-missing --cov-report=xml --cov-fail-under=90
+          uv run --group dev pytest -v \
+            packages/data-designer-config/tests \
+            packages/data-designer-engine/tests \
+            packages/data-designer/tests \
+            --cov=data_designer \
+            --cov-report=term-missing \
+            --cov-report=xml \
+            --cov-fail-under=90
+
+  # ===========================================================================
+  # End-to-End Tests
+  # ===========================================================================
 
   test-e2e:
     name: End to end test (Python ${{ matrix.python-version }} on ${{ matrix.os }})
@@ -57,8 +169,11 @@ jobs:
           enable-cache: false
 
       - name: Run e2e tests
-        run: |
-          make test-e2e
+        run: make test-e2e
+
+  # ===========================================================================
+  # Code Quality
+  # ===========================================================================
 
   lint:
     name: Lint and Format Check
@@ -76,16 +191,13 @@ jobs:
           enable-cache: true
 
       - name: Install dependencies
-        run: |
-          uv sync --group dev
+        run: make install-dev
 
       - name: Check formatting
-        run: |
-          uv run ruff format --check
+        run: make format-check
 
       - name: Run linter
-        run: |
-          uv run ruff check
+        run: make lint
 
   license-headers:
     name: Check License Headers
@@ -105,9 +217,37 @@ jobs:
           enable-cache: true
 
       - name: Install dependencies
-        run: |
-          uv sync --group dev
+        run: make install-dev
 
       - name: Check license headers
+        run: make check-license-headers
+
+  # ===========================================================================
+  # Summary Job for Branch Protection
+  # This job creates status checks matching the old job naming convention
+  # so that branch protection rules continue to work.
+  # ===========================================================================
+
+  test-summary:
+    name: Test (Python ${{ matrix.python-version }} on ${{ matrix.os }})
+    runs-on: ubuntu-latest
+    needs: [test-config, test-engine, test-interface]
+    if: always()
+    strategy:
+      matrix:
+        os: [ubuntu-latest, macos-latest]
+        python-version: ["3.10", "3.11", "3.12", "3.13"]
+
+    steps:
+      - name: Check all tests passed
         run: |
-          uv run python scripts/update_license_headers.py --check
+          if [[ "${{ needs.test-config.result }}" != "success" ]] || \
+             [[ "${{ needs.test-engine.result }}" != "success" ]] || \
+             [[ "${{ needs.test-interface.result }}" != "success" ]]; then
+            echo "One or more test jobs failed"
+            echo "test-config: ${{ needs.test-config.result }}"
+            echo "test-engine: ${{ needs.test-engine.result }}"
+            echo "test-interface: ${{ needs.test-interface.result }}"
+            exit 1
+          fi
+          echo "All test jobs passed successfully"
@@ -79,8 +79,13 @@ venv.bak/
 *.tar.gz
 *.zip
 
-# Auto-generated version file
-src/data_designer/_version.py
+# Auto-generated version files
+**/_version.py
+
+# Auto-generated README for data-designer package
+# The top-level README is copied here during build so that PyPI displays the full
+# project documentation when users install the main "data-designer" package.
+packages/data-designer/README.md
 
 # Local scratch space
 .scratch/
@@ -94,3 +99,4 @@ tests_e2e/uv.lock
 
 # Performance profiling
 perf_*.txt
+NOTEPAD.md