[codex] Validate shipped prompt CLI commands

.coderabbit.yaml

@@ -11,12 +11,12 @@
 language: "en-US"
 early_access: false
 tone_instructions: >-
-  Prioritize adapter boundaries between bundled modules and specfact_cli core: registry,
-  module-package.yaml, signing, and docs parity with modules.specfact.io. Flag cross-repo impact when
-  core APIs or contracts change.
+  Focus on bundle/adapter correctness, module signing, docs parity, and specfact_cli impact.
+  Comment only on correctness, security, regressions, data loss, release integrity, or major
+  maintainability risks.
 
 reviews:
-  profile: assertive
+  profile: chill
   request_changes_workflow: false
   high_level_summary: true
   high_level_summary_in_walkthrough: true
@@ -42,8 +42,10 @@ reviews:
     enabled: true
     drafts: false
     auto_incremental_review: true
+    # PRs targeting `dev` (not only the GitHub default branch, e.g. `main`) get automatic reviews.
     base_branches:
       - "^dev$"
+      - "^main$"
   path_instructions:
     - path: "packages/**/src/**/*.py"
       instructions: |

.github/workflows/docs-review.yml

@@ -9,10 +9,13 @@ on:
       - "**/*.md"
       - "**/*.mdc"
       - "docs/**"
+      - "packages/*/resources/prompts/**"
       - "requirements-docs-ci.txt"
       - "scripts/check-docs-commands.py"
+      - "scripts/check-prompt-commands.py"
       - "scripts/docs_site_validation.py"
       - "tests/unit/test_check_docs_commands_script.py"
+      - "tests/unit/test_check_prompt_commands_script.py"
       - "tests/unit/docs/test_docs_review.py"
       - "tests/unit/docs/test_code_review_docs_parity.py"
       - ".github/workflows/docs-review.yml"
@@ -22,10 +25,13 @@ on:
       - "**/*.md"
       - "**/*.mdc"
       - "docs/**"
+      - "packages/*/resources/prompts/**"
       - "requirements-docs-ci.txt"
       - "scripts/check-docs-commands.py"
+      - "scripts/check-prompt-commands.py"
       - "scripts/docs_site_validation.py"
       - "tests/unit/test_check_docs_commands_script.py"
+      - "tests/unit/test_check_prompt_commands_script.py"
       - "tests/unit/docs/test_docs_review.py"
       - "tests/unit/docs/test_code_review_docs_parity.py"
       - ".github/workflows/docs-review.yml"
@@ -70,6 +76,13 @@ jobs:
           python scripts/check-docs-commands.py 2>&1 | tee "$DOCS_COMMAND_LOG"
           exit "${PIPESTATUS[0]:-$?}"
 
+      - name: Validate prompt commands
+        run: |
+          mkdir -p logs/docs-review
+          PROMPT_COMMAND_LOG="logs/docs-review/prompt-command-validation_$(date -u +%Y%m%d_%H%M%S).log"
+          python scripts/check-prompt-commands.py 2>&1 | tee "$PROMPT_COMMAND_LOG"
+          exit "${PIPESTATUS[0]:-$?}"
+
       - name: Upload docs review logs
         if: always()
         uses: actions/upload-artifact@v4

openspec/changes/prompt-command-contract-validation/.openspec.yaml

@@ -0,0 +1,2 @@
+schema: spec-driven
+created: 2026-05-08

openspec/changes/prompt-command-contract-validation/TDD_EVIDENCE.md

@@ -0,0 +1,30 @@
+# TDD Evidence
+
+## Failing Before
+
+- `hatch run pytest tests/unit/test_check_prompt_commands_script.py -q` — failed as expected before implementation.
+  - 7 failures.
+  - Missing `scripts/check-prompt-commands.py`.
+  - Missing docs-review workflow prompt validation trigger/step.
+  - Missing pre-commit prompt validation gate.
+
+## Passing After
+
+- `openspec validate prompt-command-contract-validation --strict` — passed.
+- `hatch run pytest tests/unit/test_check_prompt_commands_script.py -q` — 7 passed.
+- `hatch run validate-prompt-commands` — passed.
+- `hatch run pytest tests/unit/test_check_prompt_commands_script.py tests/unit/test_pre_commit_quality_parity.py tests/unit/test_check_docs_commands_script.py tests/unit/test_validate_repo_manifests_bundle_deps.py tests/unit/test_registry_manifest_bundle_dependencies.py -q` — 30 passed.
+- `hatch run yaml-lint` — passed.
+- `hatch run format` — passed after applying formatter output.
+- `hatch run type-check` — passed.
+- `hatch run lint` — passed.
+- `hatch run verify-modules-signature --payload-from-filesystem --enforce-version-bump --version-check-base HEAD` — passed.
+- `hatch run contract-test` — 663 passed, 2 warnings.
+- `hatch run smart-test` — 663 passed, 2 warnings.
+- `hatch run test` — 663 passed, 2 warnings.
+- `hatch run specfact code review run --bug-hunt --json --out .specfact/code-review.json --scope changed` — passed with zero findings.
+
+## Resource Signing and Registry Evidence
+
+- `hatch run sign-modules --payload-from-filesystem --changed-only --base-ref HEAD --bump-version patch --allow-unsigned` — refreshed changed bundle manifests and checksum integrity.
+- `hatch run sync-registry-from-package --bundle specfact-backlog --bundle specfact-codebase --bundle specfact-govern --bundle specfact-project --bundle specfact-spec` — refreshed registry index and tarball sidecars for changed bundle versions.

openspec/changes/prompt-command-contract-validation/design.md

@@ -0,0 +1,28 @@
+# Overview
+
+The existing docs validator already discovers mounted bundle command paths from Typer apps. This change extends that idea to shipped bundle prompts, because prompt resources are executable guidance consumed by AI IDEs and should fail fast when their command examples no longer match the installed CLI.
+
+## Design
+
+- Add `scripts/check-prompt-commands.py` as the prompt-specific validator.
+- Reuse or mirror the command-discovery shape from `scripts/check-docs-commands.py`: import bundle app modules, convert Typer apps to Click commands, collect command paths, and inspect Click parameters for accepted options.
+- Scan only `packages/*/resources/prompts/**/*.md`; `.github/prompts` is intentionally excluded.
+- Extract command references from fenced shell blocks, inline backticks, slash prompt examples, and parameter/option prose. Resolve placeholders such as `<path>`, `[<bundle-name>]`, `[OPTIONS]`, line continuations, and comments without executing commands.
+- Treat unknown command paths and unknown options as blocking findings with `path:line: [category] message` output.
+- Require each executable prompt file to include a standard CLI reality-check/self-healing instruction. The current `_validate_cli_reality_check_guidance` implementation in `scripts/check-prompt-commands.py` enforces this independently for every prompt file and does not resolve companion prompt includes; companion include resolution is deferred until the validator has explicit include tracking.
+- Add a Hatch script `validate-prompt-commands`.
+- Wire `run_prompt_command_validation_gate` into `scripts/pre-commit-quality-checks.sh` before `check_safe_change` so prompt-only Markdown edits are still validated.
+- Add CI execution to `.github/workflows/docs-review.yml` and include prompt resource paths in workflow triggers.
+- Update `openspec/config.yaml` rules so future prompt-resource changes remember this validation surface.
+
+## Prompt Updates
+
+Prompt text should say:
+
+- Prompt instructions are operating guidance for SpecFact CLI, not the source of truth for the installed CLI.
+- Before running a command, inspect the current command help when unsure.
+- If a command or option from the prompt fails, inspect the nearest valid parent command with `--help`, correct the invocation when the mapping is obvious, and ask the user when no safe correction is clear.
+
+## Compatibility
+
+This change does not add runtime CLI commands for end users. It adds repository validation tooling and prompt resource edits. Because prompt resources are signed bundle payloads, implementation must account for module version/signature enforcement before finalization.

openspec/changes/prompt-command-contract-validation/proposal.md

@@ -0,0 +1,39 @@
+## Why
+
+Primary tester reports show that shipped AI slash prompts can drift from the CLI command surface. Because these prompts are bundle payloads, stale command paths or options can ship to users even when code, docs, and signatures pass.
+
+## What Changes
+
+- Add a prompt command validation gate for bundle-owned prompt resources under `packages/*/resources/prompts`.
+- Validate prompt command examples and option references against the mounted Typer/Click command tree discovered from the current repo.
+- Update shipped prompt guidance so prompts treat their text as operating guidance and verify CLI reality at execution time instead of acting as the source of truth.
+- Wire the validator into Hatch, local pre-commit checks, and Markdown-triggered CI.
+- Exclude `.github/prompts` from this change; OpenSpec helper prompts remain a separate governance surface.
+
+## Capabilities
+
+### New Capabilities
+
+- `prompt-command-validation`: Validation of bundle-owned AI prompt command references against current CLI command contracts.
+
+### Modified Capabilities
+
+- `bundle-packaged-resources`: Bundle-owned prompt resources are additionally required to be command-contract validated before release.
+- `resource-aware-integrity`: Resource prompt edits remain signed payload changes and must continue to trigger version/signature enforcement.
+
+## Impact
+
+- Affected resources: `packages/*/resources/prompts/**/*.md`.
+- Affected validation tooling: `scripts/`, `pyproject.toml`, `scripts/pre-commit-quality-checks.sh`, `.github/workflows/docs-review.yml`, and tests.
+- Bundle prompt edits may require module version/signature updates because prompt resources are signed payloads; this change does not alter `registry/index.json` unless release packaging is performed separately.
+
+## Source Tracking
+
+- GitHub Issue: [#266](https://github.com/nold-ai/specfact-cli-modules/issues/266)
+- Parent Feature: [#163](https://github.com/nold-ai/specfact-cli-modules/issues/163)
+- Parent Epic: [#162](https://github.com/nold-ai/specfact-cli-modules/issues/162)
+- Pull Request: [#265](https://github.com/nold-ai/specfact-cli-modules/pull/265)
+- Project: SpecFact CLI (`Todo`)
+- Labels: `enhancement`, `openspec`, `change-proposal`
+- Blockers: none known
+- Blocked by: none known

openspec/changes/prompt-command-contract-validation/specs/bundle-packaged-resources/spec.md

@@ -0,0 +1,41 @@
+## MODIFIED Requirements
+
+### Requirement: Official bundles SHALL ship module-owned resource payloads
+
+Each official bundle package SHALL include the prompt templates and other non-code resources that are owned by that bundle's workflows or commands. Bundle-owned resources SHALL not depend on fallback storage under the core CLI repository, and shipped prompt resources SHALL pass prompt command validation before release.
+
+#### Scenario: Official bundles ship the audited prompt inventory
+
+- **WHEN** the audited prompt inventory from `RESOURCE_OWNERSHIP_AUDIT.md` is inspected
+- **THEN** each prompt template's canonical packaged source exists under the owning official bundle package
+- **AND** the ownership mapping covers the codebase, project, spec, govern, and backlog bundles for the currently supported prompt set
+
+#### Scenario: Backlog bundle ships the restored slash-prompt inventory
+
+- **WHEN** the backlog bundle package is inspected from source or from an installed artifact
+- **THEN** `resources/prompts/` contains `specfact.backlog-add.md`, `specfact.backlog-daily.md`, `specfact.backlog-refine.md`, and `specfact.sync-backlog.md`
+- **AND** those prompt files are treated as canonical bundle-owned sources rather than historical leftovers
+
+#### Scenario: Prompt companion resources ship with prompt payloads
+
+- **WHEN** an exported prompt template references a companion file by relative path, such as `./shared/cli-enforcement.md`
+- **THEN** the owning bundle package contains that companion resource in a stable relative location
+- **AND** prompt export/copy flows can preserve a resolvable relative layout in the target IDE workspace
+
+#### Scenario: Backlog bundle ships workspace-template seed resources
+
+- **WHEN** the backlog bundle package is inspected from source or from an installed artifact
+- **THEN** the package contains the backlog field mapping templates that `specfact init` or related flows need to copy into workspace state
+- **AND** the packaged set includes both ADO and non-ADO seed templates required by supported backlog flows
+
+#### Scenario: Core no longer remains the source of truth for bundle prompts
+
+- **WHEN** a workflow prompt belongs to an extracted bundle rather than to core lifecycle commands
+- **THEN** that prompt's canonical packaged source exists in the owning bundle package
+- **AND** release packaging does not rely on the core CLI repo as the canonical source for that prompt
+
+#### Scenario: Prompt resources are command-contract validated before release
+
+- **WHEN** a bundle prompt resource is changed
+- **THEN** prompt command validation checks command paths and options against the current mounted CLI command tree
+- **AND** release or PR validation fails until stale prompt command references are corrected

openspec/changes/prompt-command-contract-validation/specs/prompt-command-validation/spec.md

@@ -0,0 +1,67 @@
+## ADDED Requirements
+
+### Requirement: Bundle prompt command references SHALL match mounted CLI contracts
+
+The modules repository SHALL validate command paths and option names embedded in bundle-owned prompt resources against the mounted SpecFact CLI command tree discoverable from the current checkout.
+
+#### Scenario: Valid prompt command example passes
+
+- **GIVEN** a bundle prompt references an implemented command such as `specfact code repro --repo .`
+- **WHEN** prompt command validation runs
+- **THEN** the command path resolves through the mounted CLI command tree
+- **AND** referenced options are accepted by that command or by an ancestor command context
+
+#### Scenario: Invalid prompt command path fails
+
+- **GIVEN** a bundle prompt references a stale command such as `specfact repro --repo .`
+- **WHEN** prompt command validation runs
+- **THEN** the validator reports the prompt path and line number
+- **AND** the validation command exits non-zero
+
+#### Scenario: Invalid prompt option fails
+
+- **GIVEN** a bundle prompt references an option that is not accepted by the resolved command path
+- **WHEN** prompt command validation runs
+- **THEN** the validator reports the stale option, prompt path, and line number
+- **AND** the validation command exits non-zero
+
+### Requirement: Prompt guidance SHALL self-check CLI reality
+
+Bundle-owned prompts SHALL tell AI IDE assistants that prompt text is operational guidance and that current CLI help or validation output is authoritative when prompt instructions disagree with the installed CLI.
+
+#### Scenario: Prompt contains CLI reality check guidance
+
+- **GIVEN** a shipped bundle prompt contains executable SpecFact command guidance
+- **WHEN** prompt command validation inspects that prompt
+- **THEN** the prompt includes a CLI reality-check instruction
+- **AND** the prompt tells the assistant to prefer current CLI help over stale prompt prose
+
+#### Scenario: Broken prompt instruction gives self-healing behavior
+
+- **GIVEN** an assistant using a prompt finds that a referenced command or option is unavailable
+- **WHEN** the prompt includes self-healing guidance
+- **THEN** the assistant is instructed to inspect the nearest valid `--help` output
+- **AND** continue only with a corrected command or ask the user when no safe correction is clear
+
+### Requirement: Prompt validation SHALL run in local and CI gates
+
+Prompt command validation SHALL be available as a Hatch command and SHALL run automatically in local pre-commit and Markdown/resource-triggered CI when bundle prompt resources or validation tooling change.
+
+#### Scenario: Hatch command exposes prompt validation
+
+- **WHEN** a contributor runs `hatch run validate-prompt-commands`
+- **THEN** the bundle prompt validation script runs
+- **AND** exits non-zero on any blocking prompt command finding
+
+#### Scenario: Pre-commit validates staged prompt edits before safe-change skipping
+
+- **GIVEN** a staged edit changes `packages/specfact-project/resources/prompts/specfact.02-plan.md`
+- **WHEN** `scripts/pre-commit-quality-checks.sh block2` runs
+- **THEN** prompt command validation runs before the script decides whether Block 2 can be skipped as a safe change
+
+#### Scenario: CI validates prompt edits
+
+- **GIVEN** a pull request changes a bundle prompt Markdown file
+- **WHEN** the docs review workflow runs
+- **THEN** prompt command validation runs with logs
+- **AND** stale command references fail the workflow

openspec/changes/prompt-command-contract-validation/specs/resource-aware-integrity/spec.md

@@ -0,0 +1,21 @@
+## MODIFIED Requirements
+
+### Requirement: Bundle integrity SHALL cover resource payloads
+
+Bundle signing, verification, and publish validation SHALL treat bundled resource files as part of the signed module payload so that resource-only changes are detected as bundle changes. Prompt resource edits SHALL also be covered by prompt command validation before release.
+
+#### Scenario: Resource edit changes signed payload
+
+- **WHEN** a prompt template or other bundled resource file changes inside a bundle package
+- **THEN** integrity verification detects a payload change until the manifest version and signature are refreshed
+
+#### Scenario: Resource-only change triggers version-bump enforcement
+
+- **WHEN** a bundled resource file changes but the bundle manifest version is not incremented
+- **THEN** the modules-repo version-bump enforcement reports that the bundle payload changed without a version bump
+
+#### Scenario: Prompt resource edit triggers command-contract validation
+
+- **WHEN** a bundled prompt resource changes
+- **THEN** the local and CI validation gates run prompt command validation
+- **AND** stale command paths or options are reported before the changed resource can ship

openspec/changes/prompt-command-contract-validation/tasks.md

@@ -0,0 +1,39 @@
+## 1. OpenSpec and Baseline
+
+- [x] 1.1 Create the `prompt-command-contract-validation` OpenSpec change.
+- [x] 1.2 Add proposal, design, and spec deltas for prompt validation, packaged resources, and resource integrity.
+- [x] 1.3 Validate the change with `openspec validate prompt-command-contract-validation --strict`.
+
+## 2. Tests First
+
+- [x] 2.1 Add unit tests for prompt command extraction from fenced shell blocks, inline backticks, slash examples, comments, placeholders, and line continuations.
+- [x] 2.2 Add unit tests for invalid command path and invalid option findings.
+- [x] 2.3 Add unit tests that prompt files with executable command guidance must contain CLI reality-check/self-healing guidance.
+- [x] 2.4 Run the new tests before implementation and record failing evidence in `TDD_EVIDENCE.md`.
+
+## 3. Validator Implementation
+
+- [x] 3.1 Add `scripts/check-prompt-commands.py` to discover mounted command paths/options and validate `packages/*/resources/prompts/**/*.md`.
+- [x] 3.2 Add `hatch run validate-prompt-commands`.
+- [x] 3.3 Keep validator output deterministic and line-addressed for pre-commit and CI logs.
+
+## 4. Prompt Resource Repairs
+
+- [x] 4.1 Update bundle prompts to use current mounted command paths and option names.
+- [x] 4.2 Add concise CLI reality-check/self-healing guidance to executable bundle prompts and shared companion guidance.
+- [x] 4.3 Preserve bundle-owned resource layout and avoid `.github/prompts` changes in this scope.
+
+## 5. Local and CI Gates
+
+- [x] 5.1 Run prompt command validation from `scripts/pre-commit-quality-checks.sh` before safe-change skipping.
+- [x] 5.2 Add prompt validation triggers and a logged validation step to `.github/workflows/docs-review.yml`.
+- [x] 5.3 Update `openspec/config.yaml` rules to require prompt validation for prompt resource changes.
+
+## 6. Evidence and Finalization
+
+- [x] 6.1 Run `openspec validate prompt-command-contract-validation --strict`.
+- [x] 6.2 Run focused pytest coverage for the prompt validator.
+- [x] 6.3 Run `hatch run validate-prompt-commands`.
+- [x] 6.4 Run relevant quality gates for touched scripts, YAML, prompts, and signed resources.
+- [x] 6.5 Record failing-before and passing-after evidence in `TDD_EVIDENCE.md`.
+- [x] 6.6 Resolve SpecFact code review findings or document any explicit exception.