feat: add JSON Schema, SLSA mapping, and positioning clarification#49
Conversation
- Add schema/devops-maturity.schema.json — supports both boolean and structured evidence format for criteria entries - Add MAPPING-SLSA.md — maps D401-D405, D603 to SLSA requirements; clarifies that DevOps Maturity does not replace SLSA - Update README.md with schema reference, structured YAML example, and positioning statement - Update content/_index.md with SLSA FAQ and positioning note
✅ Deploy Preview for devops-maturity-spec ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
📝 WalkthroughWalkthroughThis PR establishes the assessment format specification and framework relationships by introducing a JSON Schema for assessment files, creating a SLSA mapping document that aligns DevOps Maturity criteria with SLSA concepts, and updating README and website documentation to clarify the role and scope of the DevOps Maturity baseline assessment. ChangesFramework Documentation and Schema
Estimated code review effort🎯 2 (Simple) | ⏱️ ~10 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (2)
README.md (1)
6-6: ⚡ Quick winImprove wording for clarity.
The phrase "is standardized to assess" is awkward. Consider rewording to: "is a standardized framework to assess" or "provides a standardized approach to assess".
📝 Proposed wording improvement
-The DevOps Maturity specification is standardized to assess the maturity of DevOps practices. It is a set of criteria to help you measure and improve your DevOps maturity. +The DevOps Maturity specification provides a standardized approach to assess the maturity of DevOps practices. It is a set of criteria to help you measure and improve your DevOps maturity.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@README.md` at line 6, Replace the awkward phrase "is standardized to assess" in the README sentence with clearer wording; for example change the sentence "The DevOps Maturity specification is standardized to assess the maturity of DevOps practices." to use "is a standardized framework to assess" or "provides a standardized approach to assess" so it reads either "The DevOps Maturity specification is a standardized framework to assess the maturity of DevOps practices." or "The DevOps Maturity specification provides a standardized approach to assess the maturity of DevOps practices."schema/devops-maturity.schema.json (1)
38-62: 💤 Low valueConsider restricting
additionalPropertiesfor evidence items.Evidence items currently allow arbitrary additional properties (
additionalProperties: trueon line 61). While this provides flexibility, it may allow typos to pass validation silently (e.g.,tpyeinstead oftype). Consider whether evidence items should have a stricter schema (additionalProperties: false) or define an explicit set of allowed additional fields.🔒 Proposed change to restrict additional properties
} }, - "additionalProperties": true + "additionalProperties": false } },🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@schema/devops-maturity.schema.json` around lines 38 - 62, The evidence item schema currently allows arbitrary extra fields via "additionalProperties": true which can hide typos like "tpye"; change the schema for the array items (the object under "items") to restrict unexpected properties by setting "additionalProperties": false or, if certain extra fields are intentional, replace true with an explicit object listing allowed extra property names and their schemas; update the object under the items block (the properties: { type, path, tool, url } and required: ["type"]) accordingly to enforce the stricter rule while keeping any sanctioned extra fields documented.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@MAPPING-SLSA.md`:
- Around line 11-20: Update the criteria mappings to align with SLSA
Build/provenance semantics: adjust D401 and D402 to describe that SLSA Build
requires provenance describing how artifacts were produced
(buildDefinition/runDetails) rather than asserting version-controlled build
steps or CI/CD-as-code; change D404 to remove the absolute requirement of "all
dependencies pinned to exact versions" and instead note that dependency
information in provenance is best-effort and more stringent at higher SLSA
levels; change D405 to remove the claim that SBOM generation is required by SLSA
Build and state SBOMs are complementary but not mandated; and tone down D603 to
reflect that SLSA provides verifiable attestations and evidence for automated
verification but does not itself mandate producing control-to-standard mappings
or audit-ready reports.
---
Nitpick comments:
In `@README.md`:
- Line 6: Replace the awkward phrase "is standardized to assess" in the README
sentence with clearer wording; for example change the sentence "The DevOps
Maturity specification is standardized to assess the maturity of DevOps
practices." to use "is a standardized framework to assess" or "provides a
standardized approach to assess" so it reads either "The DevOps Maturity
specification is a standardized framework to assess the maturity of DevOps
practices." or "The DevOps Maturity specification provides a standardized
approach to assess the maturity of DevOps practices."
In `@schema/devops-maturity.schema.json`:
- Around line 38-62: The evidence item schema currently allows arbitrary extra
fields via "additionalProperties": true which can hide typos like "tpye"; change
the schema for the array items (the object under "items") to restrict unexpected
properties by setting "additionalProperties": false or, if certain extra fields
are intentional, replace true with an explicit object listing allowed extra
property names and their schemas; update the object under the items block (the
properties: { type, path, tool, url } and required: ["type"]) accordingly to
enforce the stricter rule while keeping any sanctioned extra fields documented.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 8069715d-3983-4efe-a46f-d23bdf029382
📒 Files selected for processing (4)
MAPPING-SLSA.mdREADME.mdcontent/_index.mdschema/devops-maturity.schema.json
| ## Criteria Mapping | ||
|
|
||
| | DevOps Maturity | Description | SLSA Direction | | ||
| |---|---|---| | ||
| | **D401** Documented Build Process | Build steps are version-controlled and documented | Build process transparency — a prerequisite for Build Track reproducibility | | ||
| | **D402** CI/CD as Code | Pipelines and infrastructure defined as code | Build process definition — SLSA requires the build process to be defined and verifiable | | ||
| | **D403** Artifact Signing | Build artifacts are cryptographically signed | Artifact integrity / provenance distribution — aligns with SLSA's requirement for signed attestations | | ||
| | **D404** Dependency Pinning | All dependencies pinned to exact versions | Reproducibility support — reduces supply-chain attack surface | | ||
| | **D405** SBOM Generation | Automatically generate Software Bill of Materials | Supply-chain transparency — complements SLSA provenance with dependency inventory | | ||
| | **D603** Compliance Mapping & Auditability | Map controls to standards, provide audit-ready reports | Verified properties / audit evidence — supports the consumption and verification side | |
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
What are the specific requirements in SLSA for build process documentation, signed attestations, and provenance?
💡 Result:
In SLSA (Supply-chain Levels for Software Artifacts), requirements for (1) build process documentation, (2) signed attestations, and (3) provenance are captured primarily in the “Build” and “Producing artifacts” requirements plus the attestation/provenance specifications. 1) Build process documentation (what must be documented) - The build process MUST generate provenance that “unambiguously identifies the output package … and describes how that package was produced” [1]. This description is represented in the provenance attestation via required build inputs such as build definition (buildType, parameters, dependencies) and run details (builder, invocation metadata, byproducts) [2]. - For SLSA Build level L1, the Provenance predicate requires at least a buildDefinition and runDetails [3][2]. 2) Signed attestations (how signing is required/structured) - SLSA defines software attestations as authenticated, machine-readable metadata containing at least a message (statement) and a signature that denotes the attester [4]. - Provenance (as a predicate type) is carried inside an attestation statement (in-toto Statement) and is expected to be signed so verifiers can validate integrity and authenticity (i.e., that the digital signature is valid and the provenance was not tampered with after the build) [1]. 3) Provenance requirements (what provenance must contain and how it must be protected) - Provenance MUST be generated by the build system to: (a) unambiguously identify the output package by cryptographic digest, and (b) describe how it was produced [1]. - Consumers MUST be able to validate authenticity by verifying the digital signature of the provenance attestation (integrity) and being able to define trust (identify build system and other entities necessary to trust the artifact) [1]. - Authenticity / tamper-resistance in generation: - Secret signing material (e.g., signing key) MUST be stored in a secure management system appropriate for such material and accessible only to the build service account [1]. - The build service MUST generate/verify every field in the provenance in a trusted control plane; user-controlled build steps MUST NOT be able to inject or alter provenance fields (except as noted in the spec) [1]. - Provenance MUST be generated by the build system within the trust boundary identified in the provenance; it MUST not be generated by a tenant outside that trust boundary [5]. - Provenance predicate structure (what’s included): - Provenance is an attestation that a particular build platform produced a set of software artifacts through execution of the buildDefinition [2]. - The provenance example shows required elements including subject artifacts, predicateType, buildDefinition (buildType, externalParameters, internalParameters, resolvedDependencies) and runDetails (builder with id and optional dependencies/version; metadata with invocationId/startedOn/finishedOn; and byproducts) [2]. Bottom line - Documentation requirement in SLSA Build is effectively met by provenance containing an unambiguous “how produced” description tied to the output package [1] (with buildDefinition + runDetails required at least at L1) [3][2]. - Signed attestations are required so that consumers can verify the attestation signature and ensure provenance integrity/authenticity [1][4]. - Provenance must be generated inside the build platform’s trusted control plane, protect signing keys, and prevent user build steps from injecting/modifying provenance fields [1][5].
Citations:
- 1: https://slsa.dev/spec/v1.2/build-requirements
- 2: https://slsa.dev/spec/v1.0/provenance
- 3: https://slsa.dev/spec/draft/build-provenance
- 4: https://slsa.dev/spec/v1.1/attestation-model
- 5: https://slsa.dev/spec/v1.0-rc1/requirements
🌐 Web query:
SLSA build requirements reproducibility hermetic dependency pinning version controlled build definition
💡 Result:
SLSA doesn’t define “reproducible/hermetic” as a single universal checklist in SLSA v1.0; instead it defines Build levels (L1–L3 in v1.0) around provenance integrity and build isolation, while “Hermetic” and “Reproducible” were explicitly present in the earlier SLSA v0.1 draft but are deferred for v1.0 (future work). [1][2] 1) Reproducibility (bit-for-bit) - In SLSA v0.1, reproducibility is listed as a build requirement option (“Build - Reproducible”). It is explicitly marked as not fully required for some levels (shown as “○” rather than a “✓”). [3] - A later SLSA v0.1 “Build L4” discussion includes “Reproducible builds, which enable other build platforms to corroborate the provenance.” [4] - SLSA v1.0 explicitly defers “hermetic builds (SLSA L4)” and “common requirements” to a future version, so SLSA v1.0 Build levels L1–L3 are not the direct “bit-for-bit reproducibility” story. [2] 2) Hermeticity (network isolation / no extraneous dependencies) - SLSA v0.1’s requirements table includes “Build - Hermetic” as a requirement (“✓”). [3] - In the SLSA v1.0 build requirements text, the note distinguishes “Hermetic” (no network access) as distinct from the v1.0 build isolation concept, and states hermetic requirements “require substantial changes … and is considered in the future directions.” [5][6] 3) Dependency pinning and “version-controlled build definition” A) Version-controlled build definitions - SLSA v0.1 requires the build definition/config executed by the build service be “verifiably derived from text file definitions stored in a version control system.” [3] B) Declared dependencies with immutable references - SLSA v0.1 requires that “all transitive build steps, sources, and dependencies were fully declared up front with immutable references.” [3] - It also requires: “MUST declare all dependencies, including sources and other build steps, using immutable references in a format that the build service understands.” [3] C) Completeness of dependencies in provenance - The SLSA v1.0 provenance predicate/documentation describes that resolved dependencies should be an “unordered collection of artifacts needed at build time” and that completeness is “best effort, at least through SLSA Build L3,” including recursively listing artifacts when build steps fetch and execute scripts/archives. [7] 4) What SLSA v1.0 Build levels require instead (isolation + provenance integrity) - SLSA v1.0 production requirements split responsibility: the producer must choose a build platform capable of achieving the desired build level, while the build platform must strengthen controls to achieve that level and provide provenance generation and isolation between builds. [6] - The v1.0 producing-artifacts requirements state the build process must generate provenance that “unambiguously identifies the output package by cryptographic digest and describes how that package was produced.” [6] - Isolation requirements include preventing builds from influencing one another and ensuring “It MUST NOT be possible for one build to persist or influence the build environment of a subsequent build,” and preventing builds from accessing sensitive build-service secrets (e.g., signing keys). [6] 5) Practical synthesis: mapping your terms to SLSA language - “Version-controlled build definition” maps directly to SLSA v0.1’s “verifiably derived from text file definitions stored in a version control system.” [3] - “Dependency pinning” maps directly (in SLSA v0.1) to “immutable references” for declared transitive dependencies/sources/build steps. [3] - “Hermetic dependency pinning” (no extraneous dependencies, no network) is explicitly part of the v0.1 hermetic requirement, but in v1.0 the spec distinguishes it and defers it as future direction (i.e., not part of Build L1–L3 isolation/provenance requirements). [5][3] - “Reproducibility” (bit-for-bit) is explicitly tracked as a v0.1 capability/requirement but not guaranteed by the v1.0 Build levels. [3][2] If you tell me which artifact ecosystem you care about (e.g., container images, npm packages, Java, Go) and which SLSA level you’re targeting, I can translate the above into concrete checklist items (lockfile/immutable refs strategy, provenance completeness expectations, and what “hermetic” means operationally vs. just isolation).
Citations:
- 1: https://slsa.dev/spec/v1.0-rc1/requirements
- 2: https://slsa.dev/spec/v1.0/whats-new
- 3: https://slsa.dev/spec/v0.1/requirements
- 4: https://slsa.dev/spec/v1.0/future-directions
- 5: https://slsa.dev/spec/draft/build-requirements
- 6: https://slsa.dev/spec/v1.0/requirements
- 7: https://github.com/slsa-framework/slsa/blob/releases/v1.0/spec/provenance.md
🌐 Web query:
Does SLSA require generating SBOMs Software Bill of Materials in any build level
💡 Result:
No. SLSA does not require generating SBOMs (Software Bill of Materials) at any build level. SLSA’s Build track requirements are centered on producing tamper-evident, verifiable build provenance (attestations about how the artifact was built, build platform properties like isolation, and—at higher levels—more complete dependency information for the provenance predicate). In the SLSA FAQ, the SLSA Provenance/Build track explicitly explains that SBOMs and SLSA provenance are different and that existing SBOM formats were “not a good fit for the requirements of the Build track,” while noting SBOMs are good practice but not mandated by SLSA Build. [1] Additionally, the SLSA spec for producing artifacts defines build-level requirements around provenance generation/distribution and related provenance fields; it does not state SBOM generation as a requirement for any build level. [2]
Citations:
🌐 Web query:
SLSA audit evidence compliance mapping requirements auditability
💡 Result:
SLSA “auditability” and “audit evidence compliance mapping” are achieved by producing SLSA attestations (especially provenance and verification summary attestations) that external parties (auditors, package ecosystems, consumers, or monitors) can verify automatically. 1) What SLSA requires for auditable evidence (evidence = attestations + verification records) - For SLSA Source, the SCS (Source Control System) MUST create Source Provenance attestations contemporaneously with each branch update, so they provide a credible, auditable record of changes, including timing, who made the changes, and which technical controls were enforced. [1] - SLSA requires that the SCS document the format and intent of all Source Provenance attestations it produces. This is necessary so others can reason about the claimed properties in the related summary attestation. [1] - SLSA distinguishes between: a) Source Provenance attestations (tamper-proof metadata used to determine what SLSA Source Level / properties a revision meets) [1] b) Source VSAs (Source Verification Summary Attestations) used to communicate high-level security properties of a source revision to downstream users. [1] - In SLSA, attestations are the mechanism that make explicit security claims about artifacts; they are primarily meant to feed automated policy engines and verification workflows. [2] 2) How “compliance mapping” to audit requirements becomes audit-ready - SLSA itself provides a control/evidence model that can be mapped into compliance processes by treating “evidence items” as verifiable attestations that correspond to SLSA claims. - In SLSA Source L3+ (and similar levels in other tracks), SLSA expects organizations to be able to verify that processes were followed for the consumed revision by examining the related summary or provenance attestations; e.g., consumers can look for organization-specific properties recorded in verifiedLevels in the summary attestation. [3] - SLSA verification is explicitly part of the model: attestations don’t “do anything” unless someone inspects them via verification, and SLSA provides the mechanics for verifying properties of source revisions using the attestations associated with those revisions. [4] 3) What makes the evidence “auditable” in practice (properties needed for auditors) - Tamper-evident, contemporaneous provenance: SCS-generated Source Provenance attestations must be created contemporaneously and provide an auditable record. [1] - Verifiable semantics: SLSA provenance/attestation formats must be documented so claims can be interpreted reliably by verifiers and downstream users. [1] - Efficiency for audits via summary attestations: when you need auditability at scale, VSAs provide a compact, high-level result that downstream parties can use instead of re-deriving everything from raw history each time. [1] 4) Evidence-to-claim mapping: where verification should happen SLSA describes multiple valid verification placements—e.g., verification at source fetch time, at package upload time, at consumer download/use time, or via continuous monitoring—so the organization can choose audit workflows that match its architecture while keeping the evidence verifiable. [4] 5) Common “auditability gaps” organizations must avoid - Relying only on attestations without verification: SLSA emphasizes that attestations require inspection (verification) to be meaningful. [4] - Not documenting how attestations map to SLSA properties: SLSA Source requires the SCS to document provenance format/intent so that summary attestations’ claims can be reasoned about. [1] If you tell me which SLSA track/level you’re mapping (e.g., SLSA Source L3 vs Build L3) and what compliance framework you’re targeting (SOC 2, ISO 27001, SSDF, etc.), I can produce a concrete “audit evidence checklist” that points to the exact SLSA attestation artifacts you should store and what properties an auditor/verifier should be able to confirm.
Citations:
- 1: https://slsa.dev/spec/v1.2/source-requirements
- 2: https://github.com/slsa-framework/slsa/blob/releases/v1.2/spec/attestation-model.md
- 3: https://github.com/slsa-framework/slsa/blob/a2a41dab72fcd05b1078fbec92f221407d52dd94/spec/source-example-controls.md
- 4: https://slsa.dev/spec/v1.2-rc2/verifying-source
Fix the D401–D405 mappings to match SLSA Build/provenance requirements.
- D403 (Artifact Signing): Accurate—SLSA relies on signed attestations/provenance so verifiers can validate integrity/authenticity.
- D401 (Documented Build Process) / D402 (CI/CD as Code): Overstated—SLSA Build requires provenance that describes how the artifact was produced (via buildDefinition/runDetails), not an explicit requirement that build steps are “version-controlled” or that CI/CD is “defined as code.”
- D404 (Dependency Pinning): Incorrect as written—SLSA does not require “all dependencies pinned to exact versions” for Build L1–L3; dependency info in provenance is best-effort and more complete at higher levels.
- D405 (SBOM Generation): Incorrect—SLSA does not require SBOM generation in the Build track (SBOMs can complement provenance but are not mandated).
- D603 (Compliance Mapping & Auditability): Directionally fine but overstated—SLSA provides verifiable, auditable attestations/evidence for automated verification; it doesn’t require control-to-standard mapping content or producing “audit-ready reports” directly.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@MAPPING-SLSA.md` around lines 11 - 20, Update the criteria mappings to align
with SLSA Build/provenance semantics: adjust D401 and D402 to describe that SLSA
Build requires provenance describing how artifacts were produced
(buildDefinition/runDetails) rather than asserting version-controlled build
steps or CI/CD-as-code; change D404 to remove the absolute requirement of "all
dependencies pinned to exact versions" and instead note that dependency
information in provenance is best-effort and more stringent at higher SLSA
levels; change D405 to remove the claim that SBOM generation is required by SLSA
Build and state SBOMs are complementary but not mandated; and tone down D603 to
reflect that SLSA provides verifiable attestations and evidence for automated
verification but does not itself mandate producing control-to-standard mappings
or audit-ready reports.
There was a problem hiding this comment.
@coderabbitai so please make suggestions if you think you are right
There was a problem hiding this comment.
@shenxianpeng, here are the suggested corrections based on the SLSA specification research. The key issues are:
- D401/D402 – SLSA Build L1+ requires provenance (buildDefinition + runDetails) describing how the artifact was produced; it does not explicitly mandate that build steps be "version-controlled" or pipelines be "defined as code."
- D404 – SLSA does not mandate pinning all dependencies to exact versions; dependency completeness in provenance is best-effort through Build L3 (SLSA spec).
- D405 – SLSA explicitly states SBOMs are not a Build track requirement (SLSA FAQ).
- D603 – SLSA provides verifiable attestations for automated verification, but does not itself mandate producing control-to-standard mappings or audit-ready reports.
Here is the corrected table:
## Criteria Mapping
| DevOps Maturity | Description | SLSA Direction |
|---|---|---|
| **D401** Documented Build Process | Build steps are version-controlled and documented | Provenance completeness — SLSA Build requires provenance (buildDefinition + runDetails) that describes how the artifact was produced; documented build steps support this but are not directly mandated |
| **D402** CI/CD as Code | Pipelines and infrastructure defined as code | Verifiable build definition — supports SLSA's provenance requirement that build inputs be identifiable and reproducible, though SLSA does not explicitly mandate CI/CD-as-code |
| **D403** Artifact Signing | Build artifacts are cryptographically signed | Artifact integrity / provenance distribution — aligns directly with SLSA's requirement for signed attestations so consumers can verify authenticity |
| **D404** Dependency Pinning | All dependencies pinned to exact versions | Provenance dependency information — SLSA records resolved dependencies in provenance on a best-effort basis; completeness expectations increase at higher Build levels but exact pinning is not explicitly mandated |
| **D405** SBOM Generation | Automatically generate Software Bill of Materials | Complementary to SLSA provenance — SLSA Build track does **not** require SBOM generation; SBOMs complement provenance by providing dependency inventory but are not mandated |
| **D603** Compliance Mapping & Auditability | Map controls to standards, provide audit-ready reports | Verifiable attestations — SLSA provides tamper-evident, machine-verifiable attestations as audit evidence; SLSA does not itself mandate control-to-standard mapping or human-readable audit reports |The D403 row remains essentially unchanged as it was accurate. The others are toned down to reflect what SLSA actually requires vs. what merely aligns with or supports SLSA goals.
1 participant
Summary
Adds the structural foundation the spec was missing. Three key additions:
1. JSON Schema (
schema/devops-maturity.schema.json){status, evidence, verified_by, verified_at, rationale}2. SLSA Mapping (
MAPPING-SLSA.md)3. Positioning Clarification
README.mdandcontent/_index.mdupdated with:Summary by CodeRabbit
Documentation
New Features