Merge remote-tracking branch 'upstream/main' into gh-144278

Author: zooba

.gitattributes

@@ -106,6 +106,7 @@ Python/executor_cases.c.h                           generated
 Python/generated_cases.c.h                          generated
 Python/optimizer_cases.c.h                          generated
 Python/opcode_targets.h                             generated
+Python/record_functions.c.h                         generated
 Python/stdlib_module_names.h                        generated
 Tools/peg_generator/pegen/grammar_parser.py         generated
 aclocal.m4                                          generated

.github/ISSUE_TEMPLATE/documentation.md

@@ -0,0 +1,15 @@
+name: Documentation
+description: Report a problem with the documentation
+labels: ["docs"]
+body:
+  - type: markdown
+    attributes:
+      value: |
+        > [!NOTE]
+        > Trivial changes (for example typos) don’t require an issue before opening a PR.
+  - type: textarea
+    attributes:
+      label: "Documentation"
+      description: "A clear and concise description of the issue."
+    validations:
+      required: true

.github/ISSUE_TEMPLATE/documentation.yml

@@ -655,11 +655,14 @@ jobs:
       matrix:
         sanitizer:
         - address
-        - undefined
-        - memory
         oss-fuzz-project-name:
         - cpython3
         - python3-libraries
+        include:
+        - sanitizer: undefined
+          oss-fuzz-project-name: cpython3
+        - sanitizer: memory
+          oss-fuzz-project-name: cpython3
         exclude:
         # Note that the 'no-exclude' sentinel below is to prevent
         # an empty string value from excluding all jobs and causing

.github/workflows/build.yml

@@ -711,10 +711,10 @@ Object Protocol
 
    :c:func:`PyUnstable_EnableTryIncRef` must have been called
    earlier on *obj* or this function may spuriously return ``0`` in the
-   :term:`free threading` build.
+   :term:`free-threaded build`.
 
    This function is logically equivalent to the following C code, except that
-   it behaves atomically in the :term:`free threading` build::
+   it behaves atomically in the :term:`free-threaded build`::
 
       if (Py_REFCNT(op) > 0) {
          Py_INCREF(op);
@@ -791,10 +791,10 @@ Object Protocol
    On GIL-enabled builds, this function is equivalent to
    :c:expr:`Py_REFCNT(op) == 1`.
 
-   On a :term:`free threaded <free threading>` build, this checks if *op*'s
+   On a :term:`free-threaded build`, this checks if *op*'s
    :term:`reference count` is equal to one and additionally checks if *op*
    is only used by this thread. :c:expr:`Py_REFCNT(op) == 1` is **not**
-   thread-safe on free threaded builds; prefer this function.
+   thread-safe on free-threaded builds; prefer this function.
 
    The caller must hold an :term:`attached thread state`, despite the fact
    that this function doesn't call into the Python interpreter. This function

Doc/c-api/object.rst

@@ -25,7 +25,7 @@ of Python objects.
 
    .. note::
 
-      On :term:`free threaded <free threading>` builds of Python, returning 1
+      On :term:`free-threaded builds <free-threaded build>` of Python, returning 1
       isn't sufficient to determine if it's safe to treat *o* as having no
       access by other threads. Use :c:func:`PyUnstable_Object_IsUniquelyReferenced`
       for that instead.

Doc/c-api/refcounting.rst

@@ -171,7 +171,10 @@ Now, build install the *project in the current directory* (``.``) via ``pip``:
 
 .. code-block:: sh
 
-   python -m pip install .
+   python -m pip -v install .
+
+The ``-v`` (``--verbose``) option causes ``pip`` to show the output from
+the compiler, which is often useful during development.
 
 .. tip::
 
@@ -460,7 +463,7 @@ So, we'll need to *encode* the data, and we'll use the UTF-8 encoding for it.
 and the C API has special support for it.)
 
 The function to encode a Python string into a UTF-8 buffer is named
-:c:func:`PyUnicode_AsUTF8` [#why-pyunicodeasutf8]_.
+:c:func:`PyUnicode_AsUTF8AndSize` [#why-pyunicodeasutf8]_.
 Call it like this:
 
 .. code-block:: c
@@ -469,31 +472,31 @@ Call it like this:
    static PyObject *
    spam_system(PyObject *self, PyObject *arg)
    {
-      const char *command = PyUnicode_AsUTF8(arg);
+      const char *command = PyUnicode_AsUTF8AndSize(arg, NULL);
       int status = 3;
       PyObject *result = PyLong_FromLong(status);
       return result;
    }
 
-If :c:func:`PyUnicode_AsUTF8` is successful, *command* will point to the
-resulting array of bytes.
+If :c:func:`PyUnicode_AsUTF8AndSize` is successful, *command* will point to the
+resulting C string -- a zero-terminated array of bytes [#embedded-nul]_.
 This buffer is managed by the *arg* object, which means we don't need to free
 it, but we must follow some rules:
 
 * We should only use the buffer inside the ``spam_system`` function.
-  When ``spam_system`` returns, *arg* and the buffer it manages might be
+  After ``spam_system`` returns, *arg* and the buffer it manages might be
   garbage-collected.
 * We must not modify it. This is why we use ``const``.
 
-If :c:func:`PyUnicode_AsUTF8` was *not* successful, it returns a ``NULL``
+If :c:func:`PyUnicode_AsUTF8AndSize` was *not* successful, it returns a ``NULL``
 pointer.
 When calling *any* Python C API, we always need to handle such error cases.
 The way to do this in general is left for later chapters of this documentation.
 For now, be assured that we are already handling errors from
 :c:func:`PyLong_FromLong` correctly.
 
-For the :c:func:`PyUnicode_AsUTF8` call, the correct way to handle errors is
-returning ``NULL`` from ``spam_system``.
+For the :c:func:`PyUnicode_AsUTF8AndSize` call, the correct way to handle
+errors is returning ``NULL`` from ``spam_system``.
 Add an ``if`` block for this:
 
 
@@ -503,7 +506,7 @@ Add an ``if`` block for this:
    static PyObject *
    spam_system(PyObject *self, PyObject *arg)
    {
-      const char *command = PyUnicode_AsUTF8(arg);
+      const char *command = PyUnicode_AsUTF8AndSize(arg);
       if (command == NULL) {
          return NULL;
       }
@@ -512,7 +515,18 @@ Add an ``if`` block for this:
       return result;
    }
 
-That's it for the setup.
+To test that error handling works, compile again, restart Python so that
+``import spam`` picks up the new version of your module, and try passing
+a non-string value to your function:
+
+.. code-block:: pycon
+
+   >>> import spam
+   >>> spam.system(3)
+   Traceback (most recent call last):
+      ...
+   TypeError: bad argument type for built-in operation
+
 Now, all that is left is calling the C library function :c:func:`system` with
 the ``char *`` buffer, and using its result instead of the ``3``:
 
@@ -522,7 +536,7 @@ the ``char *`` buffer, and using its result instead of the ``3``:
    static PyObject *
    spam_system(PyObject *self, PyObject *arg)
    {
-      const char *command = PyUnicode_AsUTF8(arg);
+      const char *command = PyUnicode_AsUTF8AndSize(arg);
       if (command == NULL) {
          return NULL;
       }
@@ -543,7 +557,8 @@ system command:
    >>> result
    0
 
-You might also want to test error cases:
+You can also test with other commands, like ``ls``, ``dir``, or one
+that doesn't exist:
 
 .. code-block:: pycon
 
@@ -553,11 +568,6 @@ You might also want to test error cases:
    >>> result
    32512
 
-   >>> spam.system(3)
-   Traceback (most recent call last):
-      ...
-   TypeError: bad argument type for built-in operation
-
 
 The result
 ==========
@@ -665,3 +675,13 @@ on :py:attr:`sys.path`.
    type.
 .. [#why-pyunicodeasutf8] Here, ``PyUnicode`` refers to the original name of
    the Python :py:class:`str` class: ``unicode``.
+
+   The ``AndSize`` part of the name refers to the fact that this function can
+   also retrieve the size of the buffer, using an output argument.
+   We don't need this, so we set the second argument to NULL.
+.. [#embedded-nul] We're ignoring the fact that Python strings can also
+   contain NUL bytes, which terminate a C string.
+   In other words, our function will treat ``spam.system("foo\0bar")`` as
+   ``spam.system("foo")``.
+   This possibility can lead to security issues, so the real ``os.system``
+   function size checks for this case and raises an error.

Doc/extending/first-extension-module.rst

@@ -160,9 +160,9 @@ Glossary
       On most builds of Python, having an attached thread state implies that the
       caller holds the :term:`GIL` for the current interpreter, so only
       one OS thread can have an attached thread state at a given moment. In
-      :term:`free-threaded <free threading>` builds of Python, threads can concurrently
-      hold an attached thread state, allowing for true parallelism of the bytecode
-      interpreter.
+      :term:`free-threaded builds <free-threaded build>` of Python, threads can
+      concurrently hold an attached thread state, allowing for true parallelism of
+      the bytecode interpreter.
 
    attribute
       A value associated with an object which is usually referenced by name
@@ -580,6 +580,13 @@ Glossary
       the :term:`global interpreter lock` which allows only one thread to
       execute Python bytecode at a time.  See :pep:`703`.
 
+   free-threaded build
+
+      A build of :term:`CPython` that supports :term:`free threading`,
+      configured using the :option:`--disable-gil` option before compilation.
+
+      See :ref:`freethreading-python-howto`.
+
    free variable
       Formally, as defined in the :ref:`language execution model <bind_names>`, a free
       variable is any variable used in a namespace which is not a local variable in that

Doc/glossary.rst

@@ -12,7 +12,7 @@
 static PyObject *
 spam_system(PyObject *self, PyObject *arg)
 {
-   const char *command = PyUnicode_AsUTF8(arg);
+   const char *command = PyUnicode_AsUTF8AndSize(arg, NULL);
    if (command == NULL) {
       return NULL;
    }

Doc/includes/capi-extension/spammodule-01.c

@@ -91,7 +91,8 @@ POST request.
    ``False`` otherwise.
 
    If *validate* is false, characters that are neither
-   in the normal base-64 alphabet nor the alternative alphabet are
+   in the normal base-64 alphabet nor (if *ignorechars* is not specified)
+   the alternative alphabet are
    discarded prior to the padding check, but the ``+`` and ``/`` characters
    keep their meaning if they are not in *altchars* (they will be discarded
    in future Python versions).
@@ -101,15 +102,14 @@ POST request.
 
    For more information about the strict base64 check, see :func:`binascii.a2b_base64`
 
+   .. versionchanged:: next
+      Added the *ignorechars* parameter.
+
    .. deprecated:: next
       Accepting the ``+`` and ``/`` characters with an alternative alphabet
       is now deprecated.
 
 
-   .. versionchanged:: next
-      Added the *ignorechars* parameter.
-
-
 .. function:: standard_b64encode(s)
 
    Encode :term:`bytes-like object` *s* using the standard Base64 alphabet