From 1315ffa15a4ae802c17c2ac5e2cb0b558c23a8ac Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Wed, 17 Sep 2025 09:49:26 +0100 Subject: [PATCH 1/2] PEP 803: Updates based on Discourse discussion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Use the more explicit tag `abi3.abi3t` to allow possibly expanding compatibility to 3.14 - Turn `Py_SIZE` & `Py_SET_SIZE` to functions, don't remove them - Add a table of compatibiity between extensions & interpreters - 2 more of the steps are now implemented. (They make sense on their own and are small enough to not need a PEP, but it does make sense to “collect” them in the PEP.) --- peps/pep-0803.rst | 228 ++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 190 insertions(+), 38 deletions(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index 102f56d6c03..7efefb7787b 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -7,6 +7,7 @@ Type: Standards Track Requires: 703, 793, 697 Created: 19-Aug-2025 Python-Version: 3.15 +Post-History: `8-Sep-2025 `__ Abstract @@ -18,6 +19,9 @@ To allow this, the :c:type:`PyObject` internal structure and related APIs will be removed from version 3.15 of the Limited API, requiring migration to new API for common tasks like defining modules and most classes. +Binary distributions (wheels) built with Limited API version 3.15 and above +should use the ABI tag ``abi3.abi3t``. + Terminology =========== @@ -77,11 +81,14 @@ One ABI A single compiled extension module should support both free-threaded and GIL-enabled builds. -No backwards compatibility +No backwards compatibility now The new limited API will not support CPython 3.14 and below. Projects that need this support can build separate extensions specifically for the 3.14 free-threaded interpreter, and for older stable ABI versions. + However, we won't block the possibility of extending compatibility to + CPython 3.14 and below. + API changes are OK The new Limited API may require extension authors to make significant changes to their code. @@ -116,12 +123,12 @@ Version 3.15 of the Limited API will: - :c:macro:`!_PyObject_EXTRA_INIT` - :c:macro:`PyObject_HEAD_INIT` - :c:macro:`PyObject_VAR_HEAD` + - :c:func:`Py_SET_TYPE` -- no longer include these function-like macros: +- export the following as functions in the ABI, rather than macros: - :c:func:`Py_SIZE` - :c:func:`Py_SET_SIZE` - - :c:func:`Py_SET_TYPE` Implications @@ -164,7 +171,7 @@ they will not yet be removed from the Limited API: New Export Hook (PEP 793) ------------------------- -Implementation of this PEP requires :pep:`793` (``PyModExport``: +Implementation of this PEP requires :pep:`793` (``PyModExport``): A new entry point for C extension modules) to be accepted, providing a new “export hook” for defining extension modules. Using the new hook will become mandatory in Limited API 3.15. @@ -207,17 +214,19 @@ removed in future CPython versions, if the internal object layout needs to change. -Wheel tags ----------- +The ``abi3t`` wheel tag +----------------------- -PyPA build tools should not need changes: if they allow the user to set the -limited API version, setting it to 3.15 should define :c:macro:`Py_LIMITED_API` -to ``0x030f0000``. -The resulting wheel should be tagged with the Python-ABI tag ``cp315-abi3``. +Wheels that use a stable ABI compatible with free-threading CPython builds +should use a new ABI tag: ``abi3t``. +The name is chosen to reflect the fact that this ABI is similar to ``abi3``, +except changes necessary to support free-threading (which uses the letter ``t`` +in existing, version-specific ABI tags like ``cp314t``). -Installers and other tools should assume that wheels with a *python tag* -``cp315`` and above and ABI tag ``abi3`` are compatible with free-threaded -builds. +Since wheels built using Limited API 3.15 will be compatible with both +GIL-enabled builds and free-threaded ones, they should use the +`compressed ABI tag set `__ +``abi3.abi3t``. New API @@ -231,11 +240,11 @@ Limited API to allow thread-safety without a GIL -- presumably ``PyMutex``, ``Py similar -- will be added via the C API working group, or in a follow-up PEP. -Backwards Compatibility -======================= +Backwards and Forwards Compatibility +==================================== Limited API 3.15 will not be backwards-compatible with older CPython releases, -due to removed structs and functions. +due to the need to use new ``PyModExport`` API added in :pep:`793`. Extension authors who cannot switch may continue to use Limited API 3.14 and below. @@ -243,6 +252,128 @@ For compatibility with free-threaded builds, they can compile using version-specific ABI -- for example, compile on CPython 3.15 without defining ``Py_LIMITED_API``. +Limited API 3.15 will be forward-compatible with future versions of CPython 3.x. +Older versions of the Limited API (that is, 3.14 and below) will continue +to be forward-compatible with GIL-enabled builds of CPython 3.x, starting with +the version that introduced the given Limited API. + + +Compatibility Overview +---------------------- + +The following table summarizes compatibility of wheel tags with CPython +interpreters. “GIL” stands for GIL-enabled interpreter; “FT” stands for a +free-threaded one. + +.. list-table:: + :widths: auto + :header-rows: 1 + + * * Wheel tag + * 3.14 (GIL) + * 3.14 (FT) + * 3.15 (GIL) + * 3.15 (FT) + * 3.16+ (GIL) + * 3.16+ (FT) + * * ``cp314-cp314`` + * ✅ + * ❌ + * ❌ + * ❌ + * ❌ + * ❌ + * * ``cp314-cp314t`` + * ❌ + * ✅ + * ❌ + * ❌ + * ❌ + * ❌ + * * ``cp314-abi3`` + * ✅ + * ❌ + * ✅ + * ❌ + * ✅ + * ❌ + * * ``cp315-cp315`` + * ❌ + * ❌ + * ✅ + * ❌ + * ❌ + * ❌ + * * ``cp315-cp315t`` + * ❌ + * ❌ + * ❌ + * ✅ + * ❌ + * ❌ + * * ``cp315-abi3`` + * ❌ + * ❌ + * ✅ + * ❌ + * ✅ + * ❌ + * * ``cp315-abi3.abi3t`` + * ❌ + * ❌ + * ✅ + * ✅ + * ✅ + * ✅ + +The following table summarizes which wheel tag should be used for an extension +built with a given interpreter and ``Py_LIMITED_API`` macro: + +.. list-table:: + :widths: auto + :header-rows: 1 + + * * To get the wheel tag… + * Compile on… + * with ``Py_LIMITED_API`` set to… + * Note + * * ``cp314-cp314`` + * 3.14 (GIL) + * (unset) + * existing + * * ``cp314-cp314t`` + * 3.14 (FT) + * (unset) + * existing + * * ``cp314-abi3`` + * 3.14+ (GIL) + * ``PY_PACK_VERSION(3, 14)`` + * existing + * * ``cp315-cp315`` + * 3.15 (GIL) + * (unset) + * continued + * * ``cp315-cp315t`` + * 3.15 (FT) + * (unset) + * continued + * * ``cp315-abi3`` + * 3.15+ (GIL) + * ``PY_PACK_VERSION(3, 15)`` + * discontinued + * * ``cp315-abi3.abi3t`` + * 3.15+ (any) + * ``PY_PACK_VERSION(3, 15)`` + * new + +Values in the *Note* column: + +* *existing*: The wheel tag is currently in use +* *continued*: The wheel tag continues the existing scheme +* *discontinued*: The wheel tag continues the existing scheme, but it will + be discouraged. Older tools may still generate it. +* *new*: Proposed in this PEP. + Security Implications ===================== @@ -268,10 +399,10 @@ This PEP combines several pieces, implemented individually: Implemented in GitHub pull request `python/cpython#136505 `__. - For ``PyModExport``, see :pep:`793`. -- For a version-checking slot, see GitHub pull request `python/cpython#137212 - `__. -- For a check for older ``abi3``, see GitHub pull request `python/cpython#137957 - `__. +- A version-checking slot was implemented in GitHub pull request + `python/cpython#137212 `__. +- A check for older ``abi3`` was implemented in GitHub pull request + `python/cpython#137957 `__. - For wheel tags, there is no implementation yet. - A porting guide is not yet written. @@ -285,26 +416,24 @@ Add an alternative stable ABI for free-threading It would be possible to: -- Keep the current stable ABI (“``abi3``”) unchanged (except additions, as done - in each release). Extensions would need no code changes and builds would be - compatible with old and new GIL-enabled CPython versions. -- Add a new stable ABI (“``abi3t``”) specifically for free-threading. - Extensions would need no code changes and builds would be +- Add a new stable ABI (“``abi3t``”) specifically for free-threading, + which would be incompatible with the existing ``abi3``. + Extensions would need no code changes to target ``abi3t`` and builds would be compatible with free-threaded CPython (3.14 and above). -- Defining an additional macro (“``Py_OPAQUE_PYOBJECT``”) would make +- Define an additional macro (“``Py_OPAQUE_PYOBJECT``”), which would make ``PyObject`` opaque as in this PEP. Extensions would need code changes as in - this PEP, and compiled extensions (“``abi3.abi3t``”) would be compatible with - all builds of CPython 3.15+. + this PEP, and as in this PEP, compiled extensions (“``abi3.abi3t``”) would + be compatible with all builds of CPython 3.15+. This scheme was rejected as too complex. It would also make the free-threading memory layout of ``PyObject`` part of the stable ABI, preventing future adjustments. -Shim for compatibility with CPython 3.14 ----------------------------------------- +Shims for compatibility with CPython 3.14 +----------------------------------------- -The issue that prevents compatibility with Python 3.14 is that with +The main issue that prevents compatibility with Python 3.14 is that with opaque ``PyObject`` and ``PyModuleDef``, it is not feasible to initialize an extension module. The solution, :pep:`793`, is only being added in Python 3.15. @@ -314,16 +443,39 @@ free-threading and GIL-enabled) are “frozen”, so it is possible for an extension to query the running interpreter, and for 3.14, use a ``struct`` definition corresponding to the detected build's ``PyModuleDef``. -This is too onerous to support and test in CPython's Limited API. -It would also require adding a new wheel tag (e.g. ``abi3t``) that all install -tools would need to recognize. (This PEP's ``cp315-abi3`` is incompatible -with Python 3.14.) +This is too onerous to support and test in CPython's Limited API at this point, +but it may be allowed in the future. -Open Issues -=========== +Using the Python wheel tag to determine compatibility +----------------------------------------------------- + +A previous version of this PEP avoided adding a new wheel tag (``abi3t``), +and specified that wheels tagged ``abi3`` would be compatible with +free-threading if the *Python tag* is ``cp315`` or higher. + +Such a scheme would work for this PEP, but it cannot express that an extension +is compatible with both GIL-enabled and free-threaded builds +of CPython 3.14 or lower. +Adding a new explicit tag means that *if* we allow building such wheels in the +future, packaging tools should not need additional changes to support them. +They would be tagged ``cp314-abi3.abi3t``. + + +Adding an ``abi4`` wheel tag +---------------------------- + +Instead of ``abi3t``, we could “bump the version” and use ``abi4`` instead +as the wheel ABI tag. +In the wheel tag, the difference is largely cosmetic. + +However, one thing this PEP does not propose is changing the *filename* +tag: extensions will be named with the extensions like ``.abi3.so``. +Changing this while keeping compatibility with GIL-enabled builds would be +an unnecessary technical change. -[See discussion for now.] +Using ``abi3.abi4`` in wheel tags but only ``.abi3`` in filenames would +look more inconsistent than ``abi3.abi3t`` and ``.abi3``. Copyright From 0ea5d3252d7b8286a9c9556c527053578361d60a Mon Sep 17 00:00:00 2001 From: Petr Viktorin Date: Thu, 18 Sep 2025 14:40:30 +0100 Subject: [PATCH 2/2] Proper date format --- peps/pep-0803.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0803.rst b/peps/pep-0803.rst index 7efefb7787b..edece9a2c31 100644 --- a/peps/pep-0803.rst +++ b/peps/pep-0803.rst @@ -7,7 +7,7 @@ Type: Standards Track Requires: 703, 793, 697 Created: 19-Aug-2025 Python-Version: 3.15 -Post-History: `8-Sep-2025 `__ +Post-History: `08-Sep-2025 `__ Abstract