From 2275852478c4a5068c0f2c484f41c363feae058a Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 17 Mar 2025 15:39:29 -0700 Subject: [PATCH 1/2] PEP 751: address feedback - Make `packages.*.name` optional - `packages.directory.editable` can be ignored at install-time - Clarify the marker syntax changes - Introduce `default-group` - Clarify how this PEP does not fully replace requirements files - Clarify how different installers can be used to install from a lock file, but different lockers could lead to different outcomes --- peps/pep-0751.rst | 128 +++++++++++++++++++++++++++++++++++++--------- 1 file changed, 105 insertions(+), 23 deletions(-) diff --git a/peps/pep-0751.rst b/peps/pep-0751.rst index 6e9951b0c19..2519addb2fa 100644 --- a/peps/pep-0751.rst +++ b/peps/pep-0751.rst @@ -210,7 +210,7 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Type**: Array of strings - **Required?**: no; defaults to ``[]`` -- **Inspiration**: :ref:`packaging:pyproject-tool-table` +- **Inspiration**: :ref:`packaging:core-metadata-provides-extra` - The list of :ref:`extras ` supported by this lock file. - Lockers MAY choose to not support writing lock files that support extras and @@ -240,6 +240,22 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. is, in effect, multi-use even if it only looks to be single-use. +``default-group`` +================= + +- **Type**: String +- **Required?**: no +- **Inspiration**: Poetry_, PDM_ +- The name of a synthetic dependency group to represent what should be installed + by default (e.g. what ``project.dependencies`` implicitly represents). +- Meant to be used in situations where ``packages.marker`` necessitates such a + group to exist. +- The group listed by this key SHOULD NOT be listed in ``dependency-groups`` as + the group is not meant to be directly exposed to users by name but instead via + an installer's UI. +- This group can only be specified when ``dependency-groups`` has values. + + ``created-by`` ============== @@ -455,7 +471,12 @@ consistent order. Usage of inline tables SHOULD also be kept consistent. - **Type**: boolean - **Required?**: no; defaults to ``false`` - **Inspiration**: :ref:`packaging:direct-url-data-structure-local-directory` -- A flag representing whether the source tree should be installed as editable. +- A flag representing whether the source tree was an editable install at lock + time. +- Installer MAY choose to ignore this flag if user actions or context would make + an editable install unnecessary or undesirable (e.g. a container image that + will not be mounted for development purposes but instead deployed to + production where it would be treated at read-only). ``packages.directory.subdirectory`` @@ -567,7 +588,8 @@ See ``packages.vcs.subdirectory``. ''''''''''''''''''''''' - **Type**: string -- **Required?**: yes +- **Required?**: no, not when the last component of ``path``/ ``url`` would be + the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:`packaging:source-distribution-format-sdist` file. @@ -620,7 +642,8 @@ See ``packages.archive.hashes``. '''''''''''''''''''''''' - **Type**: string -- **Required?**: yes +- **Required?**: no, not when the last component of ``path``/ ``url`` would be + the same value - **Inspiration**: PDM_, Poetry_, uv_ - The file name of the :ref:`packaging:binary-distribution-format` file. @@ -715,22 +738,67 @@ contexts where marker syntax is used (e.g. ``METADATA``, ``pyproject.toml``). First, two new markers will be introduced: ``extras`` and ``dependency_groups``. They represent the extras and dependency groups that have -been requested to be installed, respectively. - -Second, the marker specification will be changed to allow for containers and not -just strings for values. ONLY the new markers introduced in this PEP MAY and -MUST be used with a :py:class:`collections.abc.Container` type for their -value (which default to empty containers). An assumption of the type of -container used MUST NOT be made (e.g. could be a set, list, or tuple). +been requested to be installed, respectively: + +.. code-block:: diff + diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst + index 06897da2..c9ab247f 100644 + --- a/source/specifications/dependency-specifiers.rst + +++ b/source/specifications/dependency-specifiers.rst + @@ -87,7 +87,7 @@ environments:: + 'platform_system' | 'platform_version' | + 'platform_machine' | 'platform_python_implementation' | + 'implementation_name' | 'implementation_version' | + - 'extra' # ONLY when defined by a containing layer + + 'extra' | 'extras' | 'dependency_groups' # ONLY when defined by a containing layer + ) + marker_var = wsp* (env_var | python_str) + marker_expr = marker_var marker_op marker_var + +This does NOT preclude using the same syntax parser in other contexts, only +where the new markers happen to be considered valid based on context. + +Second, the marker specification will be changed to allow sets for values (on +top of the current support for strings and versions). ONLY the new markers +introduced in this PEP will allow for a set for their value (which defaults to +an empty set). This specifically does NOT update the spec to allow for set +literals. + +Third, the marker expression syntax specification will be updated to allow for +operations involving sets: + +.. code-block:: diff + diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst + index 06897da2..ac29d796 100644 + --- a/source/specifications/dependency-specifiers.rst + +++ b/source/specifications/dependency-specifiers.rst + @@ -196,15 +196,16 @@ safely evaluate it without running arbitrary code that could become a security + vulnerability. Markers were first standardised in :pep:`345`. This document + fixes some issues that were observed in the design described in :pep:`426`. + + -Comparisons in marker expressions are typed by the comparison operator. The + - operators that are not in perform the same as they + -do for strings in Python. The operators use the version comparison + -rules of the :ref:`Version specifier specification ` + -when those are defined (that is when both sides have a valid + -version specifier). If there is no defined behaviour of this specification + -and the operator exists in Python, then the operator falls back to + -the Python behaviour. Otherwise an error should be raised. e.g. the following + -will result in errors:: + +Comparisons in marker expressions are typed by the comparison operator and the + +type of the marker value. The operators that are not in + + perform the same as they do for strings or sets in Python based on + +whether the marker value is a string or set itself. The operators + +use the version comparison rules of the + +:ref:`Version specifier specification ` when those are + +defined (that is when both sides have a valid version specifier). If there is no + +defined behaviour of this specification and the operator exists in Python, then + +the operator falls back to the Python behaviour for the types involved. + +Otherwise an error should be raised. e.g. the following will result in errors:: + + "dog" ~= "fred" + python_version ~= "surprise" -Third, the " operators that are not in " will be changed -from operating "the same as they do for *strings*" to "the same as they -do for *containers*". There are no backwards-compatibility concerns as strings -are containers themselves. - -Fourth, a tool MUST raise an error if an extra or dependency group is specified -in a marker expression that does not exist in ``extras`` or -``dependency-groups``, respectively. These changes, along with ``packages.extras``/ ``packages.dependency-groups`` and marker expressions' Boolean logic support, allow for expressing arbitrary, @@ -806,10 +874,6 @@ Example run-on = 2025-03-06T12:28:57.760769 - - - - ------------ Installation ------------ @@ -923,6 +987,17 @@ file. Recording the sdist file name is for the same reason. This PEP supports multi-use lock files while requirements files are single-use. +This PEP does NOT fully replace requirements files because: + +- They support specifying installation + `options `__ + at install-time (e.g. ``--index-url``, ``--constrants``). +- They can + `reference other requirements files `__. + ======================= Backwards Compatibility @@ -997,6 +1072,13 @@ lock files at least support single-use lock files. Neither type of lock file is better or worse than the other, it just changes how much can be written down in a single file (which can influence how manageable). +Lock files that follow this PEP can be installed by any installer that +implements the specification. This allows users of a lock file to not not be +tied to the locker used by the person who produced the lock file. But it is not +the case that using a different locker will lead to the same result. This could +be for various reasons, including using different algorithms to determine what +to lock. + ======================== Reference Implementation From 0395a5e9224aefa5827adb6a3b1040cca4fc21db Mon Sep 17 00:00:00 2001 From: Brett Cannon Date: Mon, 17 Mar 2025 15:47:19 -0700 Subject: [PATCH 2/2] Address Sphinx errors --- peps/pep-0751.rst | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/peps/pep-0751.rst b/peps/pep-0751.rst index 2519addb2fa..f5cac51d8fb 100644 --- a/peps/pep-0751.rst +++ b/peps/pep-0751.rst @@ -741,6 +741,7 @@ First, two new markers will be introduced: ``extras`` and been requested to be installed, respectively: .. code-block:: diff + diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 06897da2..c9ab247f 100644 --- a/source/specifications/dependency-specifiers.rst @@ -768,6 +769,7 @@ Third, the marker expression syntax specification will be updated to allow for operations involving sets: .. code-block:: diff + diff --git a/source/specifications/dependency-specifiers.rst b/source/specifications/dependency-specifiers.rst index 06897da2..ac29d796 100644 --- a/source/specifications/dependency-specifiers.rst @@ -993,10 +995,10 @@ This PEP does NOT fully replace requirements files because: `options `__ at install-time (e.g. ``--index-url``, ``--constrants``). - They can - `reference other requirements files `__ via ``-r``. - They can - `use environment variables `__. + `use environment variables `__ . =======================