feat(toolchains): support runtime registration from manifest (#3802)

Currently, all supported Python runtime versions and their
platform-specific metadata (URLs, SHA256s, strip_prefix) must be
hardcoded in `python/versions.bzl`. This makes it slow and difficult
to adopt new Python versions or custom builds without updating
`rules_python` itself.

This PR introduces the ability to dynamically fetch and register
Python runtimes from a remote python-build-standalone (PBS) manifest
file (e.g., `SHA256SUMS`).

This is supported via two new attributes in `python.override`:
- `add_runtime_manifest_urls`: A list of URLs pointing to manifest
  files  to parse and register.
- `runtime_manifest_sha`: The SHA256 hash of the manifest file.

The manifest file format is the python-build-standalone SHA256SUMS
format (`SHA FILENAME`), extended to allow arbitrary URLs for the
filename (to allow more arbitrary locations).

Author: rickeylev

.bazelrc.deleted_packages

@@ -38,6 +38,7 @@ common --deleted_packages=tests/integration/pip_parse
 common --deleted_packages=tests/integration/pip_parse/empty
 common --deleted_packages=tests/integration/pip_parse_isolated
 common --deleted_packages=tests/integration/py_cc_toolchain_registered
+common --deleted_packages=tests/integration/runtime_manifests
 common --deleted_packages=tests/integration/toolchain_target_settings
 common --deleted_packages=tests/integration/uv_lock
 common --deleted_packages=tests/modules/another_module

CHANGELOG.md

@@ -114,6 +114,9 @@ END_UNRELEASED_TEMPLATE
 
 {#v0-0-0-added}
 ### Added
+* (toolchains) Support dynamically fetching and registering Python runtimes
+  from a python-build-standalone manifest file using
+  `python.override(add_runtime_manifest_urls = ..., runtime_manifest_sha = ...)`.
 * (toolchain) Added {obj}`python.override.toolchain_target_settings` to allow
   adding `config_setting` labels to all registered toolchains.
 * (windows) Full venv support for Windows is available. Set

docs/toolchains.md

@@ -242,6 +242,9 @@ existing attributes:
   {attr}`python.single_version_platform_override.coverage_tool`.
 * Adding additional Python versions via {bzl:obj}`python.single_version_override` or
   {bzl:obj}`python.single_version_platform_override`.
+* Adding additional Python versions dynamically from a manifest file or URL
+  via {attr}`python.override.add_runtime_manifest_files` or
+  {attr}`python.override.add_runtime_manifest_urls`.
 
 ### Registering custom runtimes
 
@@ -310,6 +313,73 @@ Added support for custom platform names, `target_compatible_with`, and
 `target_settings` with `single_version_platform_override`.
 :::
 
+### Registering runtimes from a manifest
+
+If you want to register multiple custom runtimes or versions at once, you can
+use a python-build-standalone manifest file. This is useful if you want to
+adopt new versions that are not yet built into `rules_python` without having
+to manually define each one using `single_version_platform_override`.
+
+To do this, specify the `add_runtime_manifest_files` or
+`add_runtime_manifest_urls` (and `runtime_manifest_sha`) attributes in
+`python.override` in your `MODULE.bazel`.
+
+In the example below, we register all runtimes available in a specific local
+or remote PBS release manifest:
+
+```starlark
+# File: MODULE.bazel
+python = use_extension("@rules_python//python/extensions:python.bzl", "python")
+python.override(
+    add_runtime_manifest_files = [
+        "@//:SHA256SUMS",
+    ],
+    add_runtime_manifest_urls = [
+        "https://github.com/astral-sh/python-build-standalone/releases/download/20260414/SHA256SUMS",
+    ],
+    base_url = "https://example.com/downloads",
+    runtime_manifest_sha = "ce18fdfd47c66830a40ea9b9e314a14b1636bbfd684501bc5ca1fc6d55a7933f",
+)
+```
+
+#### Manifest file format
+
+The manifest must be a plain text file where each line contains the SHA256 hash
+and the location of a runtime archive, separated by whitespace:
+
+```
+<sha256> <location>
+```
+
+The `<location>` can be either:
+- A relative filename (e.g.,
+  `cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`).
+  In this case, the download URL is constructed by appending the filename to
+  the `base_url` attribute (if using `add_runtime_manifest_files`) or to the
+  parent directory of each URL in `add_runtime_manifest_urls` (treating them
+  as mirrors).
+- An absolute URL (e.g.,
+  `https://example.com/downloads/cpython-3.10.20+20260414-x86_64-unknown-linux-gnu-install_only.tar.zst`).
+  In this case, the URL is used directly to download the archive.
+
+In both cases, the filename or the last path segment of the URL must follow
+the standard python-build-standalone naming convention. `rules_python` parses
+this name to extract runtime metadata (such as Python version, target
+architecture, operating system, and libc).
+
+Notes:
+- `rules_python` will read or download the manifest, parse it, and
+  automatically register toolchains for all valid Python runtimes found in it
+  that match supported platforms.
+- Only runtimes matching known platforms in `rules_python` will be registered.
+
+:::{versionadded} VERSION_NEXT_FEATURE
+Added support for registering runtimes from a manifest using
+`add_runtime_manifest_files`, `add_runtime_manifest_urls`, and
+`runtime_manifest_sha` in `python.override`.
+:::
+
+
 ### Using defined toolchains from WORKSPACE
 
 It is possible to use toolchains defined in `MODULE.bazel` in `WORKSPACE`. For example,

python/private/BUILD.bazel

@@ -252,6 +252,11 @@ bzl_library(
     srcs = ["normalize_name.bzl"],
 )
 
+bzl_library(
+    name = "pbs_manifest_bzl",
+    srcs = ["pbs_manifest.bzl"],
+)
+
 bzl_library(
     name = "precompile_bzl",
     srcs = ["precompile.bzl"],
@@ -274,6 +279,7 @@ bzl_library(
     srcs = ["python.bzl"],
     deps = [
         ":full_version_bzl",
+        ":pbs_manifest_bzl",
         ":platform_info_bzl",
         ":python_register_toolchains_bzl",
         ":pythons_hub_bzl",

python/private/pbs_manifest.bzl

@@ -0,0 +1,153 @@
+"""Helper functions to parse python-build-standalone manifests."""
+
+def parse_filename(filename):
+    """Parses a python-build-standalone filename (or URL) into its components.
+
+    See https://gregoryszorc.com/docs/python-build-standalone/main/running.html
+
+    Example: cpython-3.10.20+20260414-x86_64_v2-unknown-linux-musl-lto-full.tar.zst
+
+    Args:
+      filename: The filename or URL of the python-build-standalone release asset.
+
+    Returns:
+      A dictionary of parsed components if parsed successfully, else None.
+    """
+    basename = filename.rpartition("/")[-1]
+    if basename.endswith(".tar.zst"):
+        name = basename.removesuffix(".tar.zst")
+    elif basename.endswith(".tar.gz"):
+        name = basename.removesuffix(".tar.gz")
+    else:
+        return None
+
+    if not name.startswith("cpython-"):
+        return None
+    name = name.removeprefix("cpython-")
+
+    left, plus, tail = name.partition("+")
+    if plus:
+        python_version = left
+        build_version, sep, rest = tail.partition("-")
+        if not sep:
+            return None
+    else:
+        python_version, sep, rest = left.partition("-")
+        if not sep:
+            return None
+        build_version = ""
+
+    arch, sep, rest = rest.partition("-")
+    if not sep:
+        return None
+
+    microarch = ""
+    arch_base, sep_v, microarch_num = arch.partition("_v")
+    if sep_v:
+        arch = arch_base
+        microarch = "v" + microarch_num
+
+    vendor, sep, rest = rest.partition("-")
+    if not sep:
+        return None
+
+    os, sep, rest = rest.partition("-")
+    if not sep:
+        return None
+
+    libc = ""
+    next_part, _, remaining = rest.partition("-")
+    if os == "linux" and next_part in ["gnu", "musl"]:
+        libc = next_part
+        flavor = remaining
+    elif os == "windows" and next_part == "msvc":
+        libc = next_part
+        flavor = remaining
+    else:
+        libc = ""
+        flavor = rest
+
+    freethreaded = False
+    if flavor.startswith("freethreaded+"):
+        freethreaded = True
+        flavor = flavor.removeprefix("freethreaded+")
+    elif flavor.startswith("freethreaded-"):
+        freethreaded = True
+        flavor = flavor.removeprefix("freethreaded-")
+    elif flavor == "freethreaded":
+        freethreaded = True
+        flavor = ""
+
+    archive_flavor = ""
+    if flavor.endswith("-full"):
+        archive_flavor = "full"
+        flavor = flavor.removesuffix("-full")
+    elif flavor == "full":
+        archive_flavor = "full"
+        flavor = ""
+    elif flavor.endswith("-install_only_stripped"):
+        archive_flavor = "install_only_stripped"
+        flavor = flavor.removesuffix("-install_only_stripped")
+    elif flavor == "install_only_stripped":
+        archive_flavor = "install_only_stripped"
+        flavor = ""
+    elif flavor.endswith("-install_only"):
+        archive_flavor = "install_only"
+        flavor = flavor.removesuffix("-install_only")
+    elif flavor == "install_only":
+        archive_flavor = "install_only"
+        flavor = ""
+
+    return {
+        "arch": arch,
+        "archive_flavor": archive_flavor,
+        "build_version": build_version,
+        "flavor": flavor,
+        "freethreaded": freethreaded,
+        "libc": libc,
+        "location": filename,
+        "microarch": microarch,
+        "os": os,
+        "python_version": python_version,
+        "vendor": vendor,
+    }
+
+def parse_sha_manifest(content):
+    """Parses the SHA256SUMS file content into a list of structs.
+
+    Args:
+      content: The raw content of the manifest file.
+
+    Returns:
+      A list of structs capturing the parsed components of each valid entry.
+      Each struct contains the following fields:
+        - arch: CPU architecture (e.g., "x86_64").
+        - archive_flavor: Release asset archive type (e.g., "full", "install_only").
+        - build_version: Standalone release date (e.g., "20260414").
+        - location: Full package filename or URL (e.g., "cpython-3.11.15..." or "https://...").
+        - flavor: Build configuration flavor (e.g., "install_only").
+        - freethreaded: Whether the build is free-threaded (boolean).
+        - libc: C library type (e.g., "gnu", "musl", "msvc", or "").
+        - microarch: Microarchitecture level (e.g., "v2", "v3", or "").
+        - os: Operating system (e.g., "linux", "darwin", "windows").
+        - python_version: Python semver version (e.g., "3.11.15").
+        - sha256: SHA256 integrity hash of the release asset.
+        - vendor: Platform vendor (e.g., "unknown", "apple").
+    """
+    results = []
+    for line in content.split("\n"):
+        line = line.strip()
+        if not line:
+            continue
+        parts = [p for p in line.split(" ") if p]
+        if len(parts) != 2:
+            continue
+        sha256, filename = parts
+
+        parsed = parse_filename(filename)
+        if parsed:
+            results.append(struct(
+                sha256 = sha256,
+                **parsed
+            ))
+    return results