PEP 803: Updates based on Discourse discussion (GH-4594)

- Use the more explicit tag `abi3.abi3t` to allow possibly
  expanding compatibility to 3.14
- Turn `Py_SIZE` & `Py_SET_SIZE` to functions, don't remove them
- Add a table of compatibiity between extensions & interpreters
- 2 more of the steps are now implemented. (They make sense on
  their own and are small enough to not need a PEP, but it does
  make sense to “collect” them in the PEP.)

Author: encukou

peps/pep-0803.rst

@@ -7,6 +7,7 @@ Type: Standards Track
 Requires: 703, 793, 697
 Created: 19-Aug-2025
 Python-Version: 3.15
+Post-History: `08-Sep-2025 <https://discuss.python.org/t/103628>`__
 
 
 Abstract
@@ -18,6 +19,9 @@ To allow this, the :c:type:`PyObject` internal structure and related APIs
 will be removed from version 3.15 of the Limited API, requiring migration to
 new API for common tasks like defining modules and most classes.
 
+Binary distributions (wheels) built with Limited API version 3.15 and above
+should use the ABI tag ``abi3.abi3t``.
+
 
 Terminology
 ===========
@@ -77,11 +81,14 @@ One ABI
    A single compiled extension module should support both
    free-threaded and GIL-enabled builds.
 
-No backwards compatibility
+No backwards compatibility now
    The new limited API will not support CPython 3.14 and below.
    Projects that need this support can build separate extensions specifically
    for the 3.14 free-threaded interpreter, and for older stable ABI versions.
 
+   However, we won't block the possibility of extending compatibility to
+   CPython 3.14 and below.
+
 API changes are OK
    The new Limited API may require extension authors to make significant
    changes to their code.
@@ -116,12 +123,12 @@ Version 3.15 of the Limited API will:
   - :c:macro:`!_PyObject_EXTRA_INIT`
   - :c:macro:`PyObject_HEAD_INIT`
   - :c:macro:`PyObject_VAR_HEAD`
+  - :c:func:`Py_SET_TYPE`
 
-- no longer include these function-like macros:
+- export the following as functions in the ABI, rather than macros:
 
   - :c:func:`Py_SIZE`
   - :c:func:`Py_SET_SIZE`
-  - :c:func:`Py_SET_TYPE`
 
 
 Implications
@@ -164,7 +171,7 @@ they will not yet be removed from the Limited API:
 New Export Hook (PEP 793)
 -------------------------
 
-Implementation of this PEP requires :pep:`793` (``PyModExport``:
+Implementation of this PEP requires :pep:`793` (``PyModExport``):
 A new entry  point for C extension modules) to be
 accepted, providing a new “export hook” for defining extension modules.
 Using the new hook will become mandatory in Limited API 3.15.
@@ -207,17 +214,19 @@ removed in future CPython versions, if the internal object layout needs
 to change.
 
 
-Wheel tags
-----------
+The ``abi3t`` wheel tag
+-----------------------
 
-PyPA build tools should not need changes: if they allow the user to set the
-limited API version, setting it to 3.15 should define :c:macro:`Py_LIMITED_API`
-to ``0x030f0000``.
-The resulting wheel should be tagged with the Python-ABI tag ``cp315-abi3``.
+Wheels that use a stable ABI compatible with free-threading CPython builds
+should use a new ABI tag: ``abi3t``.
+The name is chosen to reflect the fact that this ABI is similar to ``abi3``,
+except changes necessary to support free-threading (which uses the letter ``t``
+in existing, version-specific ABI tags like ``cp314t``).
 
-Installers and other tools should assume that wheels with a *python tag*
-``cp315`` and above and ABI tag ``abi3`` are compatible with free-threaded
-builds.
+Since wheels built using Limited API 3.15 will be compatible with both
+GIL-enabled builds and free-threaded ones, they should use the
+`compressed ABI tag set <https://packaging.python.org/en/latest/specifications/platform-compatibility-tags/#compressed-tag-sets>`__
+``abi3.abi3t``.
 
 
 New API
@@ -231,18 +240,140 @@ Limited API to allow thread-safety without a GIL -- presumably ``PyMutex``, ``Py
 similar -- will be added via the C API working group, or in a follow-up PEP.
 
 
-Backwards Compatibility
-=======================
+Backwards and Forwards Compatibility
+====================================
 
 Limited API 3.15 will not be backwards-compatible with older CPython releases,
-due to removed structs and functions.
+due to the need to use new ``PyModExport`` API added in :pep:`793`.
 
 Extension authors who cannot switch may continue to use Limited API 3.14
 and below.
 For compatibility with free-threaded builds, they can compile using
 version-specific ABI -- for example, compile on CPython 3.15 without defining
 ``Py_LIMITED_API``.
 
+Limited API 3.15 will be forward-compatible with future versions of CPython 3.x.
+Older versions of the Limited API (that is, 3.14 and below) will continue
+to be forward-compatible with GIL-enabled builds of CPython 3.x, starting with
+the version that introduced the given Limited API.
+
+
+Compatibility Overview
+----------------------
+
+The following table summarizes compatibility of wheel tags with CPython
+interpreters. “GIL” stands for GIL-enabled interpreter; “FT” stands for a
+free-threaded one.
+
+.. list-table::
+   :widths: auto
+   :header-rows: 1
+
+   * * Wheel tag
+     * 3.14 (GIL)
+     * 3.14 (FT)
+     * 3.15 (GIL)
+     * 3.15 (FT)
+     * 3.16+ (GIL)
+     * 3.16+ (FT)
+   * * ``cp314-cp314``
+     * ✅
+     * ❌
+     * ❌
+     * ❌
+     * ❌
+     * ❌
+   * * ``cp314-cp314t``
+     * ❌
+     * ✅
+     * ❌
+     * ❌
+     * ❌
+     * ❌
+   * * ``cp314-abi3``
+     * ✅
+     * ❌
+     * ✅
+     * ❌
+     * ✅
+     * ❌
+   * * ``cp315-cp315``
+     * ❌
+     * ❌
+     * ✅
+     * ❌
+     * ❌
+     * ❌
+   * * ``cp315-cp315t``
+     * ❌
+     * ❌
+     * ❌
+     * ✅
+     * ❌
+     * ❌
+   * * ``cp315-abi3``
+     * ❌
+     * ❌
+     * ✅
+     * ❌
+     * ✅
+     * ❌
+   * * ``cp315-abi3.abi3t``
+     * ❌
+     * ❌
+     * ✅
+     * ✅
+     * ✅
+     * ✅
+
+The following table summarizes which wheel tag should be used for an extension
+built with a given interpreter and ``Py_LIMITED_API`` macro:
+
+.. list-table::
+   :widths: auto
+   :header-rows: 1
+
+   * * To get the wheel tag…
+     * Compile on…
+     * with ``Py_LIMITED_API`` set to…
+     * Note
+   * * ``cp314-cp314``
+     * 3.14 (GIL)
+     * (unset)
+     * existing
+   * * ``cp314-cp314t``
+     * 3.14 (FT)
+     * (unset)
+     * existing
+   * * ``cp314-abi3``
+     * 3.14+ (GIL)
+     * ``PY_PACK_VERSION(3, 14)``
+     * existing
+   * * ``cp315-cp315``
+     * 3.15 (GIL)
+     * (unset)
+     * continued
+   * * ``cp315-cp315t``
+     * 3.15 (FT)
+     * (unset)
+     * continued
+   * * ``cp315-abi3``
+     * 3.15+ (GIL)
+     * ``PY_PACK_VERSION(3, 15)``
+     * discontinued
+   * * ``cp315-abi3.abi3t``
+     * 3.15+ (any)
+     * ``PY_PACK_VERSION(3, 15)``
+     * new
+
+Values in the *Note* column:
+
+* *existing*: The wheel tag is currently in use
+* *continued*: The wheel tag continues the existing scheme
+* *discontinued*: The wheel tag continues the existing scheme, but it will
+  be discouraged. Older tools may still generate it.
+* *new*: Proposed in this PEP.
+
 
 Security Implications
 =====================
@@ -268,10 +399,10 @@ This PEP combines several pieces, implemented individually:
   Implemented in GitHub pull request `python/cpython#136505
   <https://github.com/python/cpython/pull/136505>`__.
 - For ``PyModExport``, see :pep:`793`.
-- For a version-checking slot, see GitHub pull request `python/cpython#137212
-  <https://github.com/python/cpython/pull/137212>`__.
-- For a check for older ``abi3``, see GitHub pull request `python/cpython#137957
-  <https://github.com/python/cpython/pull/137957>`__.
+- A version-checking slot was implemented in GitHub pull request
+  `python/cpython#137212 <https://github.com/python/cpython/pull/137212>`__.
+- A check for older ``abi3`` was implemented in GitHub pull request
+  `python/cpython#137957 <https://github.com/python/cpython/pull/137957>`__.
 - For wheel tags, there is no implementation yet.
 - A porting guide is not yet written.
 
@@ -285,26 +416,24 @@ Add an alternative stable ABI for free-threading
 
 It would be possible to:
 
-- Keep the current stable ABI (“``abi3``”) unchanged (except additions, as done
-  in each release). Extensions would need no code changes and builds would be
-  compatible with old and new GIL-enabled CPython versions.
-- Add a new stable ABI (“``abi3t``”) specifically for free-threading.
-  Extensions would need no code changes and builds would be
+- Add a new stable ABI (“``abi3t``”) specifically for free-threading,
+  which would be incompatible with the existing ``abi3``.
+  Extensions would need no code changes to target ``abi3t`` and builds would be
   compatible with free-threaded CPython (3.14 and above).
-- Defining an additional macro (“``Py_OPAQUE_PYOBJECT``”) would make
+- Define an additional macro (“``Py_OPAQUE_PYOBJECT``”), which would make
   ``PyObject`` opaque as in this PEP. Extensions would need code changes as in
-  this PEP, and compiled extensions (“``abi3.abi3t``”) would be compatible with
-  all builds of CPython 3.15+.
+  this PEP, and as in this PEP, compiled extensions (“``abi3.abi3t``”) would
+  be compatible with all builds of CPython 3.15+.
 
 This scheme was rejected as too complex.
 It would also make the free-threading memory layout of ``PyObject`` part
 of the stable ABI, preventing future adjustments.
 
 
-Shim for compatibility with CPython 3.14
-----------------------------------------
+Shims for compatibility with CPython 3.14
+-----------------------------------------
 
-The issue that prevents compatibility with Python 3.14 is that with
+The main issue that prevents compatibility with Python 3.14 is that with
 opaque ``PyObject`` and ``PyModuleDef``, it is not feasible to initialize
 an extension module.
 The solution, :pep:`793`, is only being added in Python 3.15.
@@ -314,16 +443,39 @@ free-threading and GIL-enabled) are “frozen”, so it is possible for an
 extension to query the running interpreter, and for 3.14, use
 a ``struct`` definition corresponding to the detected build's ``PyModuleDef``.
 
-This is too onerous to support and test in CPython's Limited API.
-It would also require adding a new wheel tag (e.g. ``abi3t``) that all install
-tools would need to recognize. (This PEP's ``cp315-abi3`` is incompatible
-with Python 3.14.)
+This is too onerous to support and test in CPython's Limited API at this point,
+but it may be allowed in the future.
 
 
-Open Issues
-===========
+Using the Python wheel tag to determine compatibility
+-----------------------------------------------------
+
+A previous version of this PEP avoided adding a new wheel tag (``abi3t``),
+and specified that wheels tagged ``abi3`` would be compatible with
+free-threading if the *Python tag* is ``cp315`` or higher.
+
+Such a scheme would work for this PEP, but it cannot express that an extension
+is compatible with both GIL-enabled and free-threaded builds
+of CPython 3.14 or lower.
+Adding a new explicit tag means that *if* we allow building such wheels in the
+future, packaging tools should not need additional changes to support them.
+They would be tagged ``cp314-abi3.abi3t``.
+
+
+Adding an ``abi4`` wheel tag
+----------------------------
+
+Instead of ``abi3t``, we could “bump the version” and use ``abi4`` instead
+as the wheel ABI tag.
+In the wheel tag, the difference is largely cosmetic.
+
+However, one thing this PEP does not propose is changing the *filename*
+tag: extensions will be named with the extensions like ``.abi3.so``.
+Changing this while keeping compatibility with GIL-enabled builds would be
+an unnecessary technical change.
 
-[See discussion for now.]
+Using ``abi3.abi4`` in wheel tags but only ``.abi3`` in filenames would
+look more inconsistent than ``abi3.abi3t`` and ``.abi3``.
 
 
 Copyright