ci(traceability): gate PRs on rivet validate + commits (REQ-212, part of REQ-051)#513
Open
avrabe wants to merge 1 commit into
Open
ci(traceability): gate PRs on rivet validate + commits (REQ-212, part of REQ-051)#513avrabe wants to merge 1 commit into
avrabe wants to merge 1 commit into
Conversation
… of REQ-051) The project that builds the traceability tool did not gate its own PRs on traceability: ci.yml ran `rivet docs check` but neither `rivet validate` nor `rivet commits`. `rivet validate` ran only at release time (release.yml), so a graph error or an untraced code commit could land on main and surface only at release. Surfaced by the bootstrap-verification audit of rivet-as-tool. New `traceability` job in ci.yml: - Gate 1: `rivet validate` — exits 1 on ERRORs (broken links, dup ids, bad targets, cardinality); coverage/lint WARNINGS don't fail (default --fail-on error). rivet's own tree PASSes (0 errors, 269 warnings). - Gate 2 (pull_request only): `rivet commits --range <base.sha>..HEAD --format json`, fail if `orphans` or `broken_refs` is non-empty. NOT --strict: --strict promotes whole-store "artifact has no commit coverage" to errors and so can never pass on a narrow PR range (calibrated: --strict over the 3 clean recent merges still exits 1 on uncovered-artifact findings). The scoped orphan/broken check is the right per-PR gate. REQ-212 (this job) is implemented and traces to REQ-051. REQ-051 stays draft: it additionally needs the job marked a branch-protection REQUIRED check (operator action, empty required set tracked in #436) and a `validate --check-hooks` flag (not yet implemented) — a running-but-non-blocking gate is the advisory-gate trap #436 describes, so the parent isn't "implemented" yet. Confirmed with: actionlint (clean apart from the pre-existing custom self-hosted runner-label false positives); the gate bash tested locally — passes on a clean range (orphans=0, broken=0) and fails on a range containing the reverted REQ-209 trailer (broken_refs=2); `rivet validate` PASS. Implements: REQ-212 Refs: REQ-051, #436 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
📐 Rivet artifact delta
Graphgraph LR
REQ_051["REQ-051"]:::modified
REQ_212["REQ-212"]:::added
classDef added fill:#d4edda,stroke:#28a745,color:#155724
classDef removed fill:#f8d7da,stroke:#dc3545,color:#721c24
classDef modified fill:#fff3cd,stroke:#ffc107,color:#856404
classDef overflow fill:#e2e3e5,stroke:#6c757d,color:#495057,stroke-dasharray: 3 3
Added
Modified
Posted by |
There was a problem hiding this comment.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Rivet Criterion Benchmarks'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.
| Benchmark suite | Current: c478f13 | Previous: 2871c97 | Ratio |
|---|---|---|---|
store_insert/10000 |
17983409 ns/iter (± 920969) |
13357302 ns/iter (± 564846) |
1.35 |
This comment was automatically generated by workflow using github-action-benchmark.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Surfaced by the bootstrap-verification audit of rivet-as-tool: the project that builds the traceability tool did not gate its own PRs on traceability.
ci.ymlranrivet docs checkbut neitherrivet validatenorrivet commits—rivet validateran only at release (release.yml), so a graph error or an untraced code commit could land onmainand surface only at release. This is the in-CI mechanism REQ-051 has long called for.Change — new
traceabilityjobrivet validate: exits 1 on errors (broken links, dup ids, bad link targets, cardinality, schema-rule inconsistencies). Coverage/lint warnings don't fail (default--fail-on error). rivet's own tree PASSes today (0 errors, 269 warnings).rivet commits(PRs only):--range <base.sha>..HEAD --format json, fails iforphansorbroken_refsis non-empty — a non-exempt code commit missing trailers, or a trailer pointing at an unknown id.--strict: I calibrated it —--strictpromotes whole-store "artifact has no commit coverage" findings to errors, so it exits 1 even on the 3 clean recent merges (3 commits can't cover 918 artifacts). The scoped orphan/broken-ref check is the correct per-PR gate.Verification (REQ-212 acceptance)
actionlint .github/workflows/ci.yml— clean apart from the pre-existing custom self-hosted runner-label false positives.orphans=0, broken=0→ exit 0); fails on a range containing the reverted REQ-209 trailer (broken_refs=2→ exit 1).rivet validatePASS; new artifacts validate.Status honesty
REQ-212 (this job) →
implemented, traces to REQ-051. REQ-051 staysdrafton purpose: it also requires (1) the job be a branch-protection REQUIRED check so it actually blocks merges — an operator/repo-settings action that can't live in this file (empty required-checks set tracked in #436), and (2) arivet validate --check-hooksflag (not yet implemented). A running-but-non-blocking gate is exactly the advisory-gate trap #436 describes, so the parent isn't "implemented" yet.🤖 Generated with Claude Code