Skip to content

ci(traceability): gate PRs on rivet validate + commits (REQ-212, part of REQ-051)#513

Open
avrabe wants to merge 1 commit into
mainfrom
ci/traceability-gate
Open

ci(traceability): gate PRs on rivet validate + commits (REQ-212, part of REQ-051)#513
avrabe wants to merge 1 commit into
mainfrom
ci/traceability-gate

Conversation

@avrabe

@avrabe avrabe commented Jun 6, 2026

Copy link
Copy Markdown
Contributor

Why

Surfaced by the bootstrap-verification audit of rivet-as-tool: the project that builds the traceability tool did not gate its own PRs on traceability. ci.yml ran rivet docs check but neither rivet validate nor rivet commitsrivet validate ran only at release (release.yml), so a graph error or an untraced code commit could land on main and surface only at release. This is the in-CI mechanism REQ-051 has long called for.

Change — new traceability job

  • Gate 1 — rivet validate: exits 1 on errors (broken links, dup ids, bad link targets, cardinality, schema-rule inconsistencies). Coverage/lint warnings don't fail (default --fail-on error). rivet's own tree PASSes today (0 errors, 269 warnings).
  • Gate 2 — rivet commits (PRs only): --range <base.sha>..HEAD --format json, fails if orphans or broken_refs is non-empty — a non-exempt code commit missing trailers, or a trailer pointing at an unknown id.
    • Deliberately NOT --strict: I calibrated it — --strict promotes whole-store "artifact has no commit coverage" findings to errors, so it exits 1 even on the 3 clean recent merges (3 commits can't cover 918 artifacts). The scoped orphan/broken-ref check is the correct per-PR gate.

Verification (REQ-212 acceptance)

  • actionlint .github/workflows/ci.yml — clean apart from the pre-existing custom self-hosted runner-label false positives.
  • Gate bash tested locally: passes on a clean range (orphans=0, broken=0 → exit 0); fails on a range containing the reverted REQ-209 trailer (broken_refs=2 → exit 1).
  • rivet validate PASS; new artifacts validate.

Status honesty

REQ-212 (this job) → implemented, traces to REQ-051. REQ-051 stays draft on purpose: it also requires (1) the job be a branch-protection REQUIRED check so it actually blocks merges — an operator/repo-settings action that can't live in this file (empty required-checks set tracked in #436), and (2) a rivet validate --check-hooks flag (not yet implemented). A running-but-non-blocking gate is exactly the advisory-gate trap #436 describes, so the parent isn't "implemented" yet.

⚠️ Operator follow-ups: (a) add Traceability (rivet validate + commits) to main's required status checks once green; (b) the self-hosted runner pool is currently offline (#509), so this job — like all CI — won't execute until runners are back.

🤖 Generated with Claude Code

… of REQ-051)

The project that builds the traceability tool did not gate its own PRs on
traceability: ci.yml ran `rivet docs check` but neither `rivet validate` nor
`rivet commits`. `rivet validate` ran only at release time (release.yml), so a
graph error or an untraced code commit could land on main and surface only at
release. Surfaced by the bootstrap-verification audit of rivet-as-tool.

New `traceability` job in ci.yml:
- Gate 1: `rivet validate` — exits 1 on ERRORs (broken links, dup ids, bad
  targets, cardinality); coverage/lint WARNINGS don't fail (default
  --fail-on error). rivet's own tree PASSes (0 errors, 269 warnings).
- Gate 2 (pull_request only): `rivet commits --range <base.sha>..HEAD
  --format json`, fail if `orphans` or `broken_refs` is non-empty. NOT
  --strict: --strict promotes whole-store "artifact has no commit coverage"
  to errors and so can never pass on a narrow PR range (calibrated: --strict
  over the 3 clean recent merges still exits 1 on uncovered-artifact
  findings). The scoped orphan/broken check is the right per-PR gate.

REQ-212 (this job) is implemented and traces to REQ-051. REQ-051 stays draft:
it additionally needs the job marked a branch-protection REQUIRED check
(operator action, empty required set tracked in #436) and a `validate
--check-hooks` flag (not yet implemented) — a running-but-non-blocking gate is
the advisory-gate trap #436 describes, so the parent isn't "implemented" yet.

Confirmed with: actionlint (clean apart from the pre-existing custom
self-hosted runner-label false positives); the gate bash tested locally —
passes on a clean range (orphans=0, broken=0) and fails on a range containing
the reverted REQ-209 trailer (broken_refs=2); `rivet validate` PASS.

Implements: REQ-212
Refs: REQ-051, #436

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Jun 6, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Jun 6, 2026

Copy link
Copy Markdown

📐 Rivet artifact delta

Change Count
Added 1
Removed 0
Modified 1
Downstream impacted (depth ≤ 5) 0

Graph

graph LR
  REQ_051["REQ-051"]:::modified
  REQ_212["REQ-212"]:::added
  classDef added fill:#d4edda,stroke:#28a745,color:#155724
  classDef removed fill:#f8d7da,stroke:#dc3545,color:#721c24
  classDef modified fill:#fff3cd,stroke:#ffc107,color:#856404
  classDef overflow fill:#e2e3e5,stroke:#6c757d,color:#495057,stroke-dasharray: 3 3
Loading
Added
  • REQ-212
Modified
ID Changes
REQ-051

📎 Full HTML dashboard attached as workflow artifact rivet-delta-pr-513download from the workflow run.

Posted by rivet-delta workflow. The graph shows only changed artifacts; open the HTML dashboard (above) for full context.

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Rivet Criterion Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite Current: c478f13 Previous: 2871c97 Ratio
store_insert/10000 17983409 ns/iter (± 920969) 13357302 ns/iter (± 564846) 1.35

This comment was automatically generated by workflow using github-action-benchmark.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant