Skip to content

PEP 770: Remove mechanism for statically defining SBOM files #4321

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
brettcannon merged 2 commits into from
Mar 25, 2025
Merged

PEP 770: Remove mechanism for statically defining SBOM files #4321

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7a9b241
PEP 770: Remove mechanism for statically defining SBOM files
sethmlarson Mar 25, 2025
cc3373d
dist-info.files table isn't an open issue
sethmlarson Mar 25, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 21 additions & 151 deletions peps/pep-0770.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -153,26 +153,19 @@ can projects begin using the new features. Even with this careful approach it's
not guaranteed that tools won't break on new metadata versions and fields.

To avoid this delay, simplify overall how to include SBOMs, and to give
flexibility to build backends and tools, this PEP proposes a new top-level table
in ``pyproject.toml``, ``[dist-info.files]``, to safely add data to a Python
package through a registry of reserved names that
avoids the need for new metadata fields and versions. This mechanism allows
build backends and tools to begin using the features described in this PEP
immediately after acceptance without the head-of-line blocking on other
projects adopting the PEP.

A new top-level table was chosen over using the ``[project]`` table because,
as described in :pep:`621`, the ``[project]`` table is used for storing core
metadata and this mechanism doesn't use core metadata.
flexibility to build backends and tools, this PEP proposes using a subdirectory
under ``.dist-info`` to safely add data to a Python package while avoiding the
need for new metadata fields and versions. This mechanism allows build backends
and tools to begin using the feature described in this PEP immediately after
acceptance without the head-of-line blocking on other projects adopting the PEP.

Storing files in the ``.dist-info`` or ``.data`` directory
----------------------------------------------------------

There are two top-level directories in binary distributions where files beyond
the software itself can be stored: ``.dist-info`` and ``.data``.
This specification chose to use the ``.dist-info`` directory for storing
subdirectories and files from the new ``[dist-info.files]`` top-level table
for two reasons:
subdirectories and files.

Firstly, the ``.data`` directory has no corresponding location in the installed
package, compared to ``.dist-info`` which does preserve the link between the
Expand Down Expand Up @@ -231,37 +224,18 @@ Specification

The changes necessary to implement this PEP include:

* Explicitly reserving all subdirectory names in the ``.dist-info`` directory.
* A new registry of reserved subdirectory names in the ``.dist-info`` directory.
* An optional top-level table, ``[dist-info.files]``, added to
`project source metadata <770-spec-project-source-metadata_>`_,
* An optional ``sboms`` key in the new ``[dist-info.files]`` table,
* Explicitly reserving the subdirectory ``.dist-info/sboms``.
* `Additions <770-spec-project-formats_>`_ to the built distribution (wheel),
and installed project specifications

In addition to the above, an informational PEP will be created for tools
consuming included SBOM documents and other Python package metadata to
generate complete SBOM documents for Python packages.

.. _770-spec-dist-info-subdirs:

Reserving all subdirectory names in ``.dist-info``
--------------------------------------------------

This PEP explicitly reserves all subdirectory names in the ``.dist-info``
directory for future usage.

Build backends MUST NOT create subdirectories in the ``.dist-info`` directory
beyond the names in the `registry <770-spec-dist-info-subdirs-registry>`_
to avoid collisions with future reserved names.

Build frontends and publishing tools MAY warn users if any ``.dist-info``
subdirectories aren't in the registry.

.. _770-spec-dist-info-subdirs-registry:

Registry of reserved ``.dist-info`` subdirectory names
------------------------------------------------------
Reserving the ``.dist-info/sboms`` directory
--------------------------------------------

This PEP introduces a new registry of reserved subdirectory names allowed in
the ``.dist-info`` directory for the :term:`distribution archive`
Expand All @@ -278,88 +252,7 @@ Subdirectory name PEP / Standard
================= ==============

See :ref:`770-backwards-compat` for a complete methodology for
creating this initial set of values to avoid backwards incompatibility issues.

.. _770-spec-project-source-metadata:

Project source metadata
-----------------------

This PEP specifies changes to the project's source metadata
in the ``pyproject.toml`` file:

Add new ``[dist-info.files]`` table
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

A new optional ``[dist-info.files]`` table is added for specifying paths
in the project source tree relative to ``pyproject.toml`` to file(s) which
should be included in the built project to a subdirectory of ``.dist-info``.

This new table has only one defined optional key: ``sboms``. The value of the
``sboms`` key MUST be an array of valid glob patterns, as specified below:

* Alphanumeric characters, underscores (``_``), hyphens (``-``) and dots (``.``)
MUST be matched verbatim.
* Special glob characters: ``*``, ``?``, ``**`` and character ranges: ``[]``
containing only the verbatim matched characters MUST be supported. Within
``[...]``, the hyphen indicates a locale-agnostic range (e.g. a-z, order based
on Unicode code points). Hyphens at the start or end are matched literally.
* Path delimiters MUST be the forward slash character (``/``). Patterns are
relative to the directory containing ``pyproject.toml``, therefore the leading
slash character MUST NOT be used.
* Parent directory indicators (``..``) MUST NOT be used.

Any characters or character sequences not covered by this specification are
invalid. Projects MUST NOT use such values. Tools consuming this field SHOULD
reject invalid values with an error.

Literal paths (e.g. ``bom.cdx.json``) are treated as valid globs which means
they can also be defined.

Build tools:

* MUST treat each value in the array as a glob pattern, and MUST raise an error
if the pattern contains invalid glob syntax.
* MUST include all files matched by a listed pattern in all distribution
archives under the ``.dist-info/sboms`` directory.
* MUST raise an error if any individual user-specified pattern does not match
at least one file.

If the ``sboms`` key is present and is set to a value of an empty array,
then tools MUST NOT include any SBOM files and MUST NOT raise an error.

Examples of valid SBOM files declarations:

.. code-block:: toml

[dist-info.files]
sboms = ["bom.json"]

[dist-info.files]
sboms = ["sboms/openssl.cdx.json", "sboms/openssl.spdx.json"]

[dist-info.files]
sboms = ["sboms/*"]

[dist-info.files]
sboms = []

Examples of invalid SBOM files declarations:

.. code-block:: toml

[dist-info.files]
sboms = ["..\bom.json"]

Reason: ``..`` must not be used. ``\\`` is an invalid path delimiter, ``/``
must be used.

.. code-block:: toml

[dist-info.files]
sboms = ["bom{.json*"]

Reason: ``bom{.json*`` is not a valid glob.
avoiding backwards incompatibilities with selecting this directory name.

.. _770-spec-project-formats:

Expand All @@ -368,12 +261,6 @@ SBOM files in project formats

A few additions will be made to the existing specifications.

:term:`Project source trees <Project source tree>`
Per :ref:`770-spec-project-source-metadata` section, the
`Declaring Project Metadata specification <pyprojecttoml_>`__
will be updated to add the ``[dist-info.files]`` table
and optional ``sboms`` key.

:term:`Built distributions <Built distribution>` (:term:`wheels <wheel>`)
The wheel specification will be updated to add the new registry of reserved
directory names and to reflect that if the ``.dist-info/sboms`` subdirectory
Expand Down Expand Up @@ -420,10 +307,10 @@ SBOM standards, versions, or fields.
Backwards Compatibility
=======================

Reserved ``.dist-info`` subdirectories registry
-----------------------------------------------
Reserved ``.dist-info/sboms`` subdirectory
------------------------------------------

The new registry of reserved ``.dist-info`` subdirectories represents
The new reserved ``.dist-info/sboms`` subdirectory represents
a new reservation that wasn't previously documented, thus has the potential to
break assumptions being made by already existing tools.

Expand Down Expand Up @@ -521,23 +408,6 @@ Typically tools can automatically annotate bundled dependencies when those
dependencies come from a "packaging ecosystem" (such as PyPI, Linux distros,
Crates.io, NPM, etc).

For packages which cannot be automatically annotated and if the package author
wishes to provide an SBOM the approach will be to generate or author SBOM files
and then include those files using ``pyproject.toml``:

.. code-block:: toml

[dist-info.files]
sboms = [
"sboms/bom.cdx.json"
]

For projects manually specifying an SBOM document the challenge will be
keeping the document up-to-date. The CPython project has some
`customized tooling <https://github.com/python/cpython/blob/main/Tools/build/generate_sbom.py>`__
for this task, but it can likely be generalized into a tool reusable by other
projects.

What do SBOM tool authors need to know?
---------------------------------------

Expand Down Expand Up @@ -644,11 +514,12 @@ user-involvement or knowledge. Compare this situation to license files which
are almost always static.

The 639-style approach was ultimately dropped in favor of defining SBOMs simply
by their presence in the ``.dist-info/sboms`` directory and using a new table in
``pyproject.toml`` called ``[dist-info.files]`` to define SBOMs in source
distributions. This approach allows users to specify static SBOM files while
still empowering build backends and tools to add their own SBOM data without the
static/dynamic conflict.
by their presence in the ``.dist-info/sboms`` directory. This approach allows
build backends and tools to add their own SBOM data without the static/dynamic
conflict.

A future PEP will define the process for statically defining SBOM files to be
added to the ``.dist-info/sboms`` directory.

Open Issues
===========
Expand Down Expand Up @@ -686,9 +557,8 @@ Acknowledgements
================

Thanks to Karolina Surma for authoring and leading :pep:`639` to acceptance.
This PEP's initial design was heavily inspired by :pep:`639` and the new
"dist-info.files" mechanism generalizes 639's approach of using a subdirectory
under ``.dist-info``.
This PEP's initial design was heavily inspired by :pep:`639` and adopts a
similar approach of using a subdirectory under ``.dist-info`` to store files.

Copyright
=========
Expand Down