diff --git a/peps/pep-0776.rst b/peps/pep-0776.rst index 0d4450d1d14..4a382183242 100644 --- a/peps/pep-0776.rst +++ b/peps/pep-0776.rst @@ -2,13 +2,13 @@ PEP: 776 Title: Emscripten Support Author: Hood Chatham Sponsor: Łukasz Langa -Discussions-To: https://discuss.python.org/t/84996 +Discussions-To: https://discuss.python.org/t/86276 Status: Draft -Type: Standards Track -Topic: Packaging +Type: Informational Created: 18-Mar-2025 Python-Version: 3.14 -Post-History: `18-Mar-2025 `__ +Post-History: `18-Mar-2025 `__, + `28-Mar-2025 `__, Abstract ======== @@ -20,8 +20,17 @@ also maintains an Emscripten target. This PEP formalizes the addition of Tier 3 for Emscripten support in Python 3.14 which `was approved by the Steering Council on October 25, 2024 -`__. The goal is to allow -Pyodide wheels to be uploaded to PyPI. +`__. The goals are: + +1. To describe the current state of the CPython Emscripten runtime +2. To describe the current state of the Pyodide runtime +3. To identify minor features to be upstreamed from the Pyodide runtime into the + CPython Emscripten runtime + +The minor features identified here are all features that could be implemented +without a PEP. We discuss more significant runtime features that we would like +to implement but we defer decisions on those features to subsequent PEPs. + Motivation ========== @@ -41,33 +50,6 @@ Emscripten platform. Emscripten and WASI are also the only supported platforms that offer any meaningful sandboxing. -Goals -===== - -It is our long term goal to upstream the entire Pyodide runtime into CPython, -but this is out of scope for the present PEP. This PEP only attempts to -establish the foundation for future work. - -Runtime Goals -------------- - -1. To describe the current state of the CPython Emscripten runtime -2. To describe the current state of the Pyodide runtime -3. To identify minor features to be upstreamed from the Pyodide runtime into the - CPython Emscripten runtime - -The minor features identified here are all features that could be implemented -without a PEP. We discuss more significant runtime features that we would like -to implement but we defer decisions on those features to subsequent PEPs. - -Packaging Goals ---------------- - -1. To describe Pyodide's packaging tooling -2. To describe Pyodide's wheel abi to a similar level of detail as the manylinux - PEPs -3. For Pyodide wheels to be allowed for upload to PyPI - Emscripten Platform Information =============================== @@ -142,30 +124,6 @@ Because of these limitations, Pyodide standardizes a no-pthreads build of Python. If there is sufficient demand, a pthreads build with no dynamic loader could be added later. -Emscripten ABI Compatibility ----------------------------- - -The Emscripten compiler has no ABI stability guarantees between versions. Many -Emscripten updates are ABI compatible by chance, and the Rust Emscripten target -behaves as if the ABI were stable with only `occasional negative consequences -`__. - -There are several linker flags that adjust the Emscripten ABI, so Python -packages built to run with Emscripten must make sure to match the ABI-sensitive -linker flags used to compile the interpreter to avoid load-time or run-time -errors. The Emscripten compiler continuously fixes bugs and adds support for new -web platform features. Thus, there is significant benefit to being able to -update the ABI. - -In order to balance the ABI stability needs of package maintainers with the ABI -flexibility to allow the platform to move forward, Pyodide plans to adopt a new -ABI for each feature release of Python. - -The Pyodide team also coordinates the ABI flags that Pyodide uses with the -Emscripten ABI that Rust supports in order to ensure that we have support for -the many popular Rust packages. Historically, most of the work for this has -been related to unwinding ABIs. See for instance `this Rust Major Change -Proposal `__. Development Tools ----------------- @@ -520,7 +478,7 @@ The following modules can be imported, but are not functional: as well as any functionality that requires these. The following are present but cannot be imported due to a dependency on the -termios package which has been removed: +termios module which has been removed: - pty - tty @@ -638,110 +596,6 @@ buildbots, maintained by Russell Keith-Magee. CPython does not currently test Tier 3 platforms on GitHub Actions, but if this ever changes, their Linux runners are able to build and test Emscripten Python. -Packaging ---------- - -Existing Package Support -~~~~~~~~~~~~~~~~~~~~~~~~ - -Pyodide currently maintains ports of 255 different packages at the time of this -writing, including major scientific Python packages like NumPy, SciPy, pandas, -Polars, scikit-learn, OpenCV, PyArrow, and Pillow as well as general purpose -packages like aiohttp, Requests, Pydantic, cryptography, and orjson. - -About 60 packages are also testing against Pyodide in their CI, including NumPy, -pandas, awkward-cpp, scikit-image, statsmodels, PyArrow, Hypothesis, and PyO3. - -Pyodide Wheel Tags -~~~~~~~~~~~~~~~~~~ - -Pyodide wheels will use ``pyodide__wasm32`` as the platform tag. For -example, ``pyodide_2025_0_wasm32``. - -With a fixed version of Emscripten, it is possible to link dynamic libraries -that require a large number of distinct ABIs, depending on ABI-sensitive linker -options and what versions of what static libraries are linked. It is our intent -that the ``pyodide_2025_0`` specifies the particular ABI that will work with the -Pyodide CPython runtime. - -For example, wheels with the following tags are compatible with Python 3.13 -Pyodide: - -- ``cp13-cp13-pyodide_2025_0_wasm32`` -- ``abi3-cp10-pyodide_2025_0_wasm32`` - -As well as all non-platformed tags. To generate the list of compatible tags, one -can use the following code:: - - - from packaging.tags import cpython_tags, _generic_platforms - - def _emscripten_platforms() -> Iterator[str]: - pyodide_abi_version = sysconfig.get_config_var("PYODIDE_ABI_VERSION") - if pyodide_abi_version: - yield f"pyodide_{pyodide_abi_version}_wasm32" - yield from _generic_platforms() - - emscripten_tags = cpython_tags(platforms=_emscripten_platforms()) - -This code will be added to `pypa/packaging -`__. - -Emscripten Wheel ABI -~~~~~~~~~~~~~~~~~~~~ - -The specification of the ``pyodide_`` ABI includes: - -* Which version of the Emscripten compiler is used -* What libraries are statically linked with the interpreter -* What stack unwinding ABI is to be used -* Which runtime platform features are required to be present - -and a handful of other similar details that affect the ABI. - -The ABI is selected by choosing the appropriate version of the Emscripten -compiler and passing appropriate compiler and linker flags. It is possible for -other people to build their own Python interpreter that is compatible with the -Pyodide ABI, it is not necessary to use the Pyodide distribution itself. - -The ``pyodide build`` tool knows how to create wheels that match our ABI. Unlike -with manylinux wheels, there is no need for a Docker container to build the -``pyodide_`` wheels. All that is needed is a Linux machine and appropriate -versions of Python, Node.js, and Emscripten. - - -Package Installers -~~~~~~~~~~~~~~~~~~ - -Installers should use the ``_emscripten_platforms()`` function shown above to -determine which platforms are compatible with an Emscripten build of CPython. In -particular, the Pyodide ABI version is exposed via -``sysconfig.get_config_var("PYODIDE_ABI_VERSION")``. - - -Package indexes -~~~~~~~~~~~~~~~ - -We recommend that package indexes accept any wheel whose platform tag matches -``pyodide_[0-9]+_[0-9]+_wasm32``. We recommend that package indexes continue not -accepting wheels that match ``emscripten_[0-9]+_[0-9]+_[0-9]+_wasm32``. - - -Dependency Specifier Markers -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -To check for the Emscripten platform in a dependency specifier, one can use -``sys_platform == 'emscripten'`` (or its negation). Such checks are already in use -in the wild and seem to be sufficient for the needs of the community. - - -Trove Classifier -~~~~~~~~~~~~~~~~ - -Packages that build and test Emscripten wheels can declare this by adding the -``Environment :: WebAssembly :: Emscripten``. PyPI already accepts uploads of -packages with this classifier. - PEP 11 ------ @@ -836,10 +690,6 @@ care to minimize backwards incompatibility. We will also need a way to disable partially-upstreamed features so that Pyodide can replace them with more complete versions downstream. -These backwards compatibility concerns impact not just the runtime but also the -packaging system. Adding new platform tags should not affect existing packaging -tools because tools ignore wheels with an unknown package tag. - Security Implications ===================== @@ -861,12 +711,6 @@ to use them all at runtime. The documentation will cover this in a similar form to the existing Windows embeddable package. In the short term, we will encourage developers to use Pyodide if at all possible. -Second, maintainers of packages with binary components need to know how to -build, test, label, and deploy them for Emscripten (see Packaging). The Pyodide -documentation on `building and testing packages -`__ -is the best reference for this. - Reference Implementation ========================