Skip to content

PEP 803: Updates based on Discourse discussion #4594

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
encukou merged 2 commits into from
Sep 19, 2025
Merged

PEP 803: Updates based on Discourse discussion #4594

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1315ffa
PEP 803: Updates based on Discourse discussion
encukou Sep 17, 2025
0ea5d32
Proper date format
encukou Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
228 changes: 190 additions & 38 deletions peps/pep-0803.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ Type: Standards Track
Requires: 703, 793, 697
Created: 19-Aug-2025
Python-Version: 3.15
Post-History: `08-Sep-2025 <https://discuss.python.org/t/103628>`__


Abstract
Expand All @@ -18,6 +19,9 @@ To allow this, the :c:type:`PyObject` internal structure and related APIs
will be removed from version 3.15 of the Limited API, requiring migration to
new API for common tasks like defining modules and most classes.

Binary distributions (wheels) built with Limited API version 3.15 and above
should use the ABI tag ``abi3.abi3t``.


Terminology
===========
Expand Down Expand Up @@ -77,11 +81,14 @@ One ABI
A single compiled extension module should support both
free-threaded and GIL-enabled builds.

No backwards compatibility
No backwards compatibility now
The new limited API will not support CPython 3.14 and below.
Projects that need this support can build separate extensions specifically
for the 3.14 free-threaded interpreter, and for older stable ABI versions.

However, we won't block the possibility of extending compatibility to
CPython 3.14 and below.

API changes are OK
The new Limited API may require extension authors to make significant
changes to their code.
Expand Down Expand Up @@ -116,12 +123,12 @@ Version 3.15 of the Limited API will:
- :c:macro:`!_PyObject_EXTRA_INIT`
- :c:macro:`PyObject_HEAD_INIT`
- :c:macro:`PyObject_VAR_HEAD`
- :c:func:`Py_SET_TYPE`

- no longer include these function-like macros:
- export the following as functions in the ABI, rather than macros:

- :c:func:`Py_SIZE`
- :c:func:`Py_SET_SIZE`
- :c:func:`Py_SET_TYPE`


Implications
Expand Down Expand Up @@ -164,7 +171,7 @@ they will not yet be removed from the Limited API:
New Export Hook (PEP 793)
-------------------------

Implementation of this PEP requires :pep:`793` (``PyModExport``:
Implementation of this PEP requires :pep:`793` (``PyModExport``):
A new entry point for C extension modules) to be
accepted, providing a new “export hook” for defining extension modules.
Using the new hook will become mandatory in Limited API 3.15.
Expand Down Expand Up @@ -207,17 +214,19 @@ removed in future CPython versions, if the internal object layout needs
to change.


Wheel tags
----------
The ``abi3t`` wheel tag
-----------------------

PyPA build tools should not need changes: if they allow the user to set the
limited API version, setting it to 3.15 should define :c:macro:`Py_LIMITED_API`
to ``0x030f0000``.
The resulting wheel should be tagged with the Python-ABI tag ``cp315-abi3``.
Wheels that use a stable ABI compatible with free-threading CPython builds
should use a new ABI tag: ``abi3t``.
The name is chosen to reflect the fact that this ABI is similar to ``abi3``,
except changes necessary to support free-threading (which uses the letter ``t``
in existing, version-specific ABI tags like ``cp314t``).

Installers and other tools should assume that wheels with a *python tag*
``cp315`` and above and ABI tag ``abi3`` are compatible with free-threaded
builds.
Since wheels built using Limited API 3.15 will be compatible with both
GIL-enabled builds and free-threaded ones, they should use the
`compressed ABI tag set <https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets>`__
``abi3.abi3t``.


New API
Expand All @@ -231,18 +240,140 @@ Limited API to allow thread-safety without a GIL -- presumably ``PyMutex``, ``Py
similar -- will be added via the C API working group, or in a follow-up PEP.


Backwards Compatibility
=======================
Backwards and Forwards Compatibility
====================================

Limited API 3.15 will not be backwards-compatible with older CPython releases,
due to removed structs and functions.
due to the need to use new ``PyModExport`` API added in :pep:`793`.

Extension authors who cannot switch may continue to use Limited API 3.14
and below.
For compatibility with free-threaded builds, they can compile using
version-specific ABI -- for example, compile on CPython 3.15 without defining
``Py_LIMITED_API``.

Limited API 3.15 will be forward-compatible with future versions of CPython 3.x.
Older versions of the Limited API (that is, 3.14 and below) will continue
to be forward-compatible with GIL-enabled builds of CPython 3.x, starting with
the version that introduced the given Limited API.


Compatibility Overview
----------------------

The following table summarizes compatibility of wheel tags with CPython
interpreters. “GIL” stands for GIL-enabled interpreter; “FT” stands for a
free-threaded one.

.. list-table::
:widths: auto
:header-rows: 1

* * Wheel tag
* 3.14 (GIL)
* 3.14 (FT)
* 3.15 (GIL)
* 3.15 (FT)
* 3.16+ (GIL)
* 3.16+ (FT)
* * ``cp314-cp314``
* ✅
* ❌
* ❌
* ❌
* ❌
* ❌
* * ``cp314-cp314t``
* ❌
* ✅
* ❌
* ❌
* ❌
* ❌
* * ``cp314-abi3``
* ✅
* ❌
* ✅
* ❌
* ✅
* ❌
* * ``cp315-cp315``
* ❌
* ❌
* ✅
* ❌
* ❌
* ❌
* * ``cp315-cp315t``
* ❌
* ❌
* ❌
* ✅
* ❌
* ❌
* * ``cp315-abi3``
* ❌
* ❌
* ✅
* ❌
* ✅
* ❌
* * ``cp315-abi3.abi3t``
* ❌
* ❌
* ✅
* ✅
* ✅
* ✅

The following table summarizes which wheel tag should be used for an extension
built with a given interpreter and ``Py_LIMITED_API`` macro:

.. list-table::
:widths: auto
:header-rows: 1

* * To get the wheel tag…
* Compile on…
* with ``Py_LIMITED_API`` set to…
* Note
* * ``cp314-cp314``
* 3.14 (GIL)
* (unset)
* existing
* * ``cp314-cp314t``
* 3.14 (FT)
* (unset)
* existing
* * ``cp314-abi3``
* 3.14+ (GIL)
* ``PY_PACK_VERSION(3, 14)``
* existing
* * ``cp315-cp315``
* 3.15 (GIL)
* (unset)
* continued
* * ``cp315-cp315t``
* 3.15 (FT)
* (unset)
* continued
* * ``cp315-abi3``
* 3.15+ (GIL)
* ``PY_PACK_VERSION(3, 15)``
* discontinued
* * ``cp315-abi3.abi3t``
* 3.15+ (any)
* ``PY_PACK_VERSION(3, 15)``
* new

Values in the *Note* column:

* *existing*: The wheel tag is currently in use
* *continued*: The wheel tag continues the existing scheme
* *discontinued*: The wheel tag continues the existing scheme, but it will
be discouraged. Older tools may still generate it.
* *new*: Proposed in this PEP.


Security Implications
=====================
Expand All @@ -268,10 +399,10 @@ This PEP combines several pieces, implemented individually:
Implemented in GitHub pull request `python/cpython#136505
<https://github.com/python/cpython/pull/136505>`__.
- For ``PyModExport``, see :pep:`793`.
- For a version-checking slot, see GitHub pull request `python/cpython#137212
<https://github.com/python/cpython/pull/137212>`__.
- For a check for older ``abi3``, see GitHub pull request `python/cpython#137957
<https://github.com/python/cpython/pull/137957>`__.
- A version-checking slot was implemented in GitHub pull request
`python/cpython#137212 <https://github.com/python/cpython/pull/137212>`__.
- A check for older ``abi3`` was implemented in GitHub pull request
`python/cpython#137957 <https://github.com/python/cpython/pull/137957>`__.
- For wheel tags, there is no implementation yet.
- A porting guide is not yet written.

Expand All @@ -285,26 +416,24 @@ Add an alternative stable ABI for free-threading

It would be possible to:

- Keep the current stable ABI (“``abi3``”) unchanged (except additions, as done
in each release). Extensions would need no code changes and builds would be
compatible with old and new GIL-enabled CPython versions.
- Add a new stable ABI (“``abi3t``”) specifically for free-threading.
Extensions would need no code changes and builds would be
- Add a new stable ABI (“``abi3t``”) specifically for free-threading,
which would be incompatible with the existing ``abi3``.
Extensions would need no code changes to target ``abi3t`` and builds would be
compatible with free-threaded CPython (3.14 and above).
- Defining an additional macro (“``Py_OPAQUE_PYOBJECT``”) would make
- Define an additional macro (“``Py_OPAQUE_PYOBJECT``”), which would make
``PyObject`` opaque as in this PEP. Extensions would need code changes as in
this PEP, and compiled extensions (“``abi3.abi3t``”) would be compatible with
all builds of CPython 3.15+.
this PEP, and as in this PEP, compiled extensions (“``abi3.abi3t``”) would
be compatible with all builds of CPython 3.15+.

This scheme was rejected as too complex.
It would also make the free-threading memory layout of ``PyObject`` part
of the stable ABI, preventing future adjustments.


Shim for compatibility with CPython 3.14
----------------------------------------
Shims for compatibility with CPython 3.14
-----------------------------------------

The issue that prevents compatibility with Python 3.14 is that with
The main issue that prevents compatibility with Python 3.14 is that with
opaque ``PyObject`` and ``PyModuleDef``, it is not feasible to initialize
an extension module.
The solution, :pep:`793`, is only being added in Python 3.15.
Expand All @@ -314,16 +443,39 @@ free-threading and GIL-enabled) are “frozen”, so it is possible for an
extension to query the running interpreter, and for 3.14, use
a ``struct`` definition corresponding to the detected build's ``PyModuleDef``.

This is too onerous to support and test in CPython's Limited API.
It would also require adding a new wheel tag (e.g. ``abi3t``) that all install
tools would need to recognize. (This PEP's ``cp315-abi3`` is incompatible
with Python 3.14.)
This is too onerous to support and test in CPython's Limited API at this point,
but it may be allowed in the future.


Open Issues
===========
Using the Python wheel tag to determine compatibility
-----------------------------------------------------

A previous version of this PEP avoided adding a new wheel tag (``abi3t``),
and specified that wheels tagged ``abi3`` would be compatible with
free-threading if the *Python tag* is ``cp315`` or higher.

Such a scheme would work for this PEP, but it cannot express that an extension
is compatible with both GIL-enabled and free-threaded builds
of CPython 3.14 or lower.
Adding a new explicit tag means that *if* we allow building such wheels in the
future, packaging tools should not need additional changes to support them.
They would be tagged ``cp314-abi3.abi3t``.


Adding an ``abi4`` wheel tag
----------------------------

Instead of ``abi3t``, we could “bump the version” and use ``abi4`` instead
as the wheel ABI tag.
In the wheel tag, the difference is largely cosmetic.

However, one thing this PEP does not propose is changing the *filename*
tag: extensions will be named with the extensions like ``.abi3.so``.
Changing this while keeping compatibility with GIL-enabled builds would be
an unnecessary technical change.

[See discussion for now.]
Using ``abi3.abi4`` in wheel tags but only ``.abi3`` in filenames would
look more inconsistent than ``abi3.abi3t`` and ``.abi3``.


Copyright
Expand Down