Skip to content

Freeze reviewer artifact contract #47

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
stacknil merged 1 commit into from
May 26, 2026
Merged

Freeze reviewer artifact contract #47

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -142,7 +142,7 @@ Cooldown behavior:

- [`demos/rule-evaluation-and-dedup-demo/README.md`](demos/rule-evaluation-and-dedup-demo/README.md) explains the third demo and links its committed before/after dedup artifacts
- [`demos/config-change-investigation-demo/README.md`](demos/config-change-investigation-demo/README.md) explains the config-change investigation demo and its committed artifacts
- [`docs/reviewer-pack.md`](docs/reviewer-pack.md) is the top-level no-guessing reviewer pack
- [`docs/reviewer-pack.md`](docs/reviewer-pack.md) is the top-level no-guessing reviewer pack and artifact naming contract
- [`docs/reviewer-path.md`](docs/reviewer-path.md) maps common review questions to the right demo and artifacts
- [`docs/architecture.md`](docs/architecture.md) diagrams the local file-based detection workflow
- [`docs/sample-output.md`](docs/sample-output.md) summarizes the committed sample artifacts
Expand Down
10 changes: 10 additions & 0 deletions docs/reviewer-pack.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,16 @@ The current artifact names are reviewer-facing contracts for the v0.7 / v1.0 con
- If an artifact must be renamed, update the README, reviewer path, this reviewer pack, demo README, tests, and committed sample outputs in the same change.
- Do not introduce platform-style names that imply alert routing, case management, real-time ingestion, or autonomous response.

### Stable Reviewer-Visible Artifacts

| Area | Stable artifact paths |
| --- | --- |
| Default telemetry sample | `data/processed/features.csv`, `data/processed/alerts.csv`, `data/processed/summary.json`, `data/processed/event_count_timeline.png`, `data/processed/error_rate_timeline.png`, `data/processed/alerts_timeline.png` |
| Richer telemetry sample | `data/processed/richer_sample/features.csv`, `data/processed/richer_sample/alerts.csv`, `data/processed/richer_sample/summary.json`, `data/processed/richer_sample/event_count_timeline.png`, `data/processed/richer_sample/error_rate_timeline.png`, `data/processed/richer_sample/alerts_timeline.png` |
| AI-assisted detection demo | `demos/ai-assisted-detection-demo/artifacts/rule_hits.json`, `demos/ai-assisted-detection-demo/artifacts/case_bundles.json`, `demos/ai-assisted-detection-demo/artifacts/case_summaries.json`, `demos/ai-assisted-detection-demo/artifacts/case_report.md`, `demos/ai-assisted-detection-demo/artifacts/audit_traces.jsonl` |
| Rule dedup demo | `demos/rule-evaluation-and-dedup-demo/artifacts/rule_hits_before_dedup.json`, `demos/rule-evaluation-and-dedup-demo/artifacts/rule_hits_after_dedup.json`, `demos/rule-evaluation-and-dedup-demo/artifacts/dedup_explanations.json`, `demos/rule-evaluation-and-dedup-demo/artifacts/dedup_report.md` |
| Config-change investigation demo | `demos/config-change-investigation-demo/artifacts/change_events_normalized.json`, `demos/config-change-investigation-demo/artifacts/investigation_hits.json`, `demos/config-change-investigation-demo/artifacts/investigation_summary.json`, `demos/config-change-investigation-demo/artifacts/investigation_report.md` |

## Fast Verification

From the repository root:
Expand Down
37 changes: 37 additions & 0 deletions tests/test_reviewer_docs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,34 @@
),
]

STABLE_REVIEWER_ARTIFACTS = [
"data/processed/features.csv",
"data/processed/alerts.csv",
"data/processed/summary.json",
"data/processed/event_count_timeline.png",
"data/processed/error_rate_timeline.png",
"data/processed/alerts_timeline.png",
"data/processed/richer_sample/features.csv",
"data/processed/richer_sample/alerts.csv",
"data/processed/richer_sample/summary.json",
"data/processed/richer_sample/event_count_timeline.png",
"data/processed/richer_sample/error_rate_timeline.png",
"data/processed/richer_sample/alerts_timeline.png",
"demos/ai-assisted-detection-demo/artifacts/rule_hits.json",
"demos/ai-assisted-detection-demo/artifacts/case_bundles.json",
"demos/ai-assisted-detection-demo/artifacts/case_summaries.json",
"demos/ai-assisted-detection-demo/artifacts/case_report.md",
"demos/ai-assisted-detection-demo/artifacts/audit_traces.jsonl",
"demos/rule-evaluation-and-dedup-demo/artifacts/rule_hits_before_dedup.json",
"demos/rule-evaluation-and-dedup-demo/artifacts/rule_hits_after_dedup.json",
"demos/rule-evaluation-and-dedup-demo/artifacts/dedup_explanations.json",
"demos/rule-evaluation-and-dedup-demo/artifacts/dedup_report.md",
"demos/config-change-investigation-demo/artifacts/change_events_normalized.json",
"demos/config-change-investigation-demo/artifacts/investigation_hits.json",
"demos/config-change-investigation-demo/artifacts/investigation_summary.json",
"demos/config-change-investigation-demo/artifacts/investigation_report.md",
]


def _read_repo_file(relative_path: str) -> str:
return (REPO_ROOT / relative_path).read_text(encoding="utf-8")
Expand Down Expand Up @@ -94,6 +122,15 @@ def test_top_level_reviewer_pack_covers_matrix_and_artifact_contract() -> None:
assert f"`{artifact_path}`" in reviewer_pack


def test_reviewer_pack_freezes_stable_artifact_names() -> None:
reviewer_pack = _read_repo_file("docs/reviewer-pack.md")

assert "Stable Reviewer-Visible Artifacts" in reviewer_pack
for artifact_path in STABLE_REVIEWER_ARTIFACTS:
assert f"`{artifact_path}`" in reviewer_pack
assert (REPO_ROOT / artifact_path).is_file(), artifact_path


def test_architecture_doc_keeps_local_file_based_boundaries() -> None:
architecture = _read_repo_file("docs/architecture.md")

Expand Down
Loading