Merge branch 'main' into jit/gc

Author: ashm-dev

Doc/c-api/import.rst

@@ -129,8 +129,7 @@ Importing Modules
    of :class:`~importlib.machinery.SourceFileLoader` otherwise.
 
    The module's :attr:`~module.__file__` attribute will be set to the code
-   object's :attr:`~codeobject.co_filename`.  If applicable,
-   :attr:`~module.__cached__` will also be set.
+   object's :attr:`~codeobject.co_filename`.
 
    This function will reload the module if it was already imported.  See
    :c:func:`PyImport_ReloadModule` for the intended way to reload a module.
@@ -142,10 +141,13 @@ Importing Modules
    :c:func:`PyImport_ExecCodeModuleWithPathnames`.
 
    .. versionchanged:: 3.12
-      The setting of :attr:`~module.__cached__` and :attr:`~module.__loader__`
+      The setting of ``__cached__`` and :attr:`~module.__loader__`
       is deprecated. See :class:`~importlib.machinery.ModuleSpec` for
       alternatives.
 
+   .. versionchanged:: 3.15
+      ``__cached__`` is no longer set.
+
 
 .. c:function:: PyObject* PyImport_ExecCodeModuleEx(const char *name, PyObject *co, const char *pathname)
 
@@ -157,16 +159,19 @@ Importing Modules
 
 .. c:function:: PyObject* PyImport_ExecCodeModuleObject(PyObject *name, PyObject *co, PyObject *pathname, PyObject *cpathname)
 
-   Like :c:func:`PyImport_ExecCodeModuleEx`, but the :attr:`~module.__cached__`
-   attribute of the module object is set to *cpathname* if it is
-   non-``NULL``.  Of the three functions, this is the preferred one to use.
+   Like :c:func:`PyImport_ExecCodeModuleEx`, but the path to any compiled file
+   via *cpathname* is used appropriately when non-``NULL``.  Of the three
+   functions, this is the preferred one to use.
 
    .. versionadded:: 3.3
 
    .. versionchanged:: 3.12
-      Setting :attr:`~module.__cached__` is deprecated. See
+      Setting ``__cached__`` is deprecated. See
       :class:`~importlib.machinery.ModuleSpec` for alternatives.
 
+   .. versionchanged:: 3.15
+      ``__cached__`` no longer set.
+
 
 .. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, const char *pathname, const char *cpathname)
 

Doc/deprecations/c-api-pending-removal-in-3.20.rst

@@ -1,6 +1,13 @@
 Pending removal in Python 3.20
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
+* :c:func:`!_PyObject_CallMethodId`, :c:func:`!_PyObject_GetAttrId` and
+  :c:func:`!_PyUnicode_FromId` are deprecated since 3.15 and will be removed in
+  3.20. Instead, use :c:func:`PyUnicode_FromString()` and cache the result in
+  the module state, then call :c:func:`PyObject_CallMethod` or
+  :c:func:`PyObject_GetAttr`.
+  (Contributed by Victor Stinner in :gh:`141049`.)
+
 * The ``cval`` field in :c:type:`PyComplexObject` (:gh:`128813`).
   Use :c:func:`PyComplex_AsCComplex` and :c:func:`PyComplex_FromCComplex`
   to convert a Python complex number to/from the C :c:type:`Py_complex`

Doc/deprecations/pending-removal-in-3.15.rst

@@ -3,9 +3,9 @@ Pending removal in Python 3.15
 
 * The import system:
 
-  * Setting :attr:`~module.__cached__` on a module while
+  * Setting ``__cached__`` on a module while
     failing to set :attr:`__spec__.cached <importlib.machinery.ModuleSpec.cached>`
-    is deprecated. In Python 3.15, :attr:`!__cached__` will cease to be set or
+    is deprecated. In Python 3.15, ``__cached__`` will cease to be set or
     take into consideration by the import system or standard library. (:gh:`97879`)
 
   * Setting :attr:`~module.__package__` on a module while

Doc/howto/gdb_helpers.rst

@@ -136,7 +136,7 @@ enabled::
        at Objects/unicodeobject.c:551
    #7  0x0000000000440d94 in PyUnicodeUCS2_FromString (u=0x5c2b8d "__lltrace__") at Objects/unicodeobject.c:569
    #8  0x0000000000584abd in PyDict_GetItemString (v=
-       {'Yuck': <type at remote 0xad4730>, '__builtins__': <module at remote 0x7ffff7fd5ee8>, '__file__': 'Lib/test/crashers/nasty_eq_vs_dict.py', '__package__': None, 'y': <Yuck(i=0) at remote 0xaacd80>, 'dict': {0: 0, 1: 1, 2: 2, 3: 3}, '__cached__': None, '__name__': '__main__', 'z': <Yuck(i=0) at remote 0xaace60>, '__doc__': None}, key=
+       {'Yuck': <type at remote 0xad4730>, '__builtins__': <module at remote 0x7ffff7fd5ee8>, '__file__': 'Lib/test/crashers/nasty_eq_vs_dict.py', '__package__': None, 'y': <Yuck(i=0) at remote 0xaacd80>, 'dict': {0: 0, 1: 1, 2: 2, 3: 3}, '__name__': '__main__', 'z': <Yuck(i=0) at remote 0xaace60>, '__doc__': None}, key=
        0x5c2b8d "__lltrace__") at Objects/dictobject.c:2171
 
 Notice how the dictionary argument to ``PyDict_GetItemString`` is displayed

Doc/library/argparse.rst

@@ -1679,7 +1679,7 @@ The Namespace object
 Other utilities
 ---------------
 
-Sub-commands
+Subcommands
 ^^^^^^^^^^^^
 
 .. method:: ArgumentParser.add_subparsers(*, [title], [description], [prog], \
@@ -1708,7 +1708,7 @@ Sub-commands
    * *description* - description for the sub-parser group in help output, by
      default ``None``
 
-   * *prog* - usage information that will be displayed with sub-command help,
+   * *prog* - usage information that will be displayed with subcommand help,
      by default the name of the program and any positional arguments before the
      subparser argument
 
@@ -1718,7 +1718,7 @@ Sub-commands
    * action_ - the basic type of action to be taken when this argument is
      encountered at the command line
 
-   * dest_ - name of the attribute under which sub-command name will be
+   * dest_ - name of the attribute under which subcommand name will be
      stored; by default ``None`` and no value is stored
 
    * required_ - Whether or not a subcommand must be provided, by default

Doc/library/functions.rst

@@ -526,7 +526,7 @@ are always available.  They are listed here in alphabetical order.
       >>> dir()   # show the names in the module namespace  # doctest: +SKIP
       ['__builtins__', '__name__', 'struct']
       >>> dir(struct)   # show the names in the struct module # doctest: +SKIP
-      ['Struct', '__all__', '__builtins__', '__cached__', '__doc__', '__file__',
+      ['Struct', '__all__', '__builtins__', '__doc__', '__file__',
        '__initializing__', '__loader__', '__name__', '__package__',
        '_clearcache', 'calcsize', 'error', 'pack', 'pack_into',
        'unpack', 'unpack_from']

Doc/library/importlib.rst

@@ -1197,8 +1197,7 @@ find and load modules.
 
    .. attribute:: cached
 
-      The filename of a compiled version of the module's code
-      (see :attr:`module.__cached__`).
+      The filename of a compiled version of the module's code.
       The :term:`finder` should always set this attribute but it may be ``None``
       for modules that do not need compiled code stored.
 
@@ -1300,7 +1299,7 @@ an :term:`importer`.
 
    .. versionadded:: 3.4
 
-.. function:: cache_from_source(path, debug_override=None, *, optimization=None)
+.. function:: cache_from_source(path, *, optimization=None)
 
    Return the :pep:`3147`/:pep:`488` path to the byte-compiled file associated
    with the source *path*.  For example, if *path* is ``/foo/bar/baz.py`` the return
@@ -1319,12 +1318,6 @@ an :term:`importer`.
    ``/foo/bar/__pycache__/baz.cpython-32.opt-2.pyc``. The string representation
    of *optimization* can only be alphanumeric, else :exc:`ValueError` is raised.
 
-   The *debug_override* parameter is deprecated and can be used to override
-   the system's value for ``__debug__``. A ``True`` value is the equivalent of
-   setting *optimization* to the empty string. A ``False`` value is the same as
-   setting *optimization* to ``1``. If both *debug_override* an *optimization*
-   are not ``None`` then :exc:`TypeError` is raised.
-
    .. versionadded:: 3.4
 
    .. versionchanged:: 3.5
@@ -1334,6 +1327,9 @@ an :term:`importer`.
    .. versionchanged:: 3.6
       Accepts a :term:`path-like object`.
 
+   .. versionchanged:: 3.15
+      The *debug_override* parameter was removed.
+
 
 .. function:: source_from_cache(path)
 

Doc/library/profiling.sampling.rst

@@ -191,6 +191,11 @@ production systems. The target process requires no modification and need not
 be restarted. The profiler attaches, collects samples for the specified
 duration, then detaches and produces output.
 
+::
+
+   python -m profiling.sampling attach --live 12345
+   python -m profiling.sampling attach --flamegraph -d 30 -o profile.html 12345
+
 On most systems, attaching to another process requires appropriate permissions.
 See :ref:`profiling-permissions` for platform-specific requirements.
 
@@ -470,9 +475,10 @@ which you can use to judge whether the data is sufficient for your analysis.
 Profiling modes
 ===============
 
-The sampling profiler supports three modes that control which samples are
+The sampling profiler supports four modes that control which samples are
 recorded. The mode determines what the profile measures: total elapsed time,
-CPU execution time, or time spent holding the global interpreter lock.
+CPU execution time, time spent holding the global interpreter lock, or
+exception handling.
 
 
 Wall-clock mode
@@ -528,6 +534,25 @@ I/O-bound or waiting. The function spends most of its time waiting for network,
 disk, locks, or sleep. CPU optimization won't help here; consider async I/O,
 connection pooling, or reducing wait time instead.
 
+.. code-block:: python
+
+   import time
+
+   def do_sleep():
+       time.sleep(2)
+
+   def do_compute():
+       sum(i**2 for i in range(1000000))
+
+   if __name__ == "__main__":
+       do_sleep()
+       do_compute()
+
+::
+
+   python -m profiling.sampling run --mode=wall script.py  # do_sleep ~98%, do_compute ~1%
+   python -m profiling.sampling run --mode=cpu script.py   # do_sleep absent, do_compute dominates
+
 
 GIL mode
 --------
@@ -552,6 +577,90 @@ GIL?" and "why are my other threads starving?" It can also be useful in
 single-threaded programs to distinguish Python execution time from time spent
 in C extensions or I/O.
 
+.. code-block:: python
+
+   import hashlib
+
+   def hash_work():
+       # C extension - releases GIL during computation
+       for _ in range(200):
+           hashlib.sha256(b"data" * 250000).hexdigest()
+
+   def python_work():
+       # Pure Python - holds GIL during computation
+       for _ in range(3):
+           sum(i**2 for i in range(1000000))
+
+   if __name__ == "__main__":
+       hash_work()
+       python_work()
+
+::
+
+   python -m profiling.sampling run --mode=cpu script.py  # hash_work ~42%, python_work ~38%
+   python -m profiling.sampling run --mode=gil script.py  # hash_work ~5%, python_work ~60%
+
+
+Exception mode
+--------------
+
+Exception mode (``--mode=exception``) records samples only when a thread has
+an active exception::
+
+   python -m profiling.sampling run --mode=exception script.py
+
+Samples are recorded in two situations: when an exception is being propagated
+up the call stack (after ``raise`` but before being caught), or when code is
+executing inside an ``except`` block where exception information is still
+present in the thread state.
+
+The following example illustrates which code regions are captured:
+
+.. code-block:: python
+
+   def example():
+       try:
+           raise ValueError("error")    # Captured: exception being raised
+       except ValueError:
+           process_error()              # Captured: inside except block
+       finally:
+           cleanup()                    # NOT captured: exception already handled
+
+   def example_propagating():
+       try:
+           try:
+               raise ValueError("error")
+           finally:
+               cleanup()                # Captured: exception propagating through
+       except ValueError:
+           pass
+
+   def example_no_exception():
+       try:
+           do_work()
+       finally:
+           cleanup()                    # NOT captured: no exception involved
+
+Note that ``finally`` blocks are only captured when an exception is actively
+propagating through them. Once an ``except`` block finishes executing, Python
+clears the exception information before running any subsequent ``finally``
+block. Similarly, ``finally`` blocks that run during normal execution (when no
+exception was raised) are not captured because no exception state is present.
+
+This mode is useful for understanding where your program spends time handling
+errors. Exception handling can be a significant source of overhead in code
+that uses exceptions for flow control (such as ``StopIteration`` in iterators)
+or in applications that process many error conditions (such as network servers
+handling connection failures).
+
+Exception mode helps answer questions like "how much time is spent handling
+exceptions?" and "which exception handlers are the most expensive?" It can
+reveal hidden performance costs in code that catches and processes many
+exceptions, even when those exceptions are handled gracefully. For example,
+if a parsing library uses exceptions internally to signal format errors, this
+mode will capture time spent in those handlers even if the calling code never
+sees the exceptions.
+
 
 Output formats
 ==============
@@ -890,6 +999,25 @@ stack often shows event loop internals rather than the logical flow of your
 coroutines. Async-aware mode addresses this by tracking which task is running
 and presenting stacks that reflect the ``await`` chain.
 
+.. code-block:: python
+
+   import asyncio
+
+   async def fetch(url):
+       await asyncio.sleep(0.1)
+       return url
+
+   async def main():
+       for _ in range(50):
+           await asyncio.gather(fetch("a"), fetch("b"), fetch("c"))
+
+   if __name__ == "__main__":
+       asyncio.run(main())
+
+::
+
+   python -m profiling.sampling run --async-aware --flamegraph -o out.html script.py
+
 .. note::
 
    Async-aware profiling requires the target process to have the :mod:`asyncio`
@@ -1006,8 +1134,9 @@ Mode options
 
 .. option:: --mode <mode>
 
-   Sampling mode: ``wall`` (default), ``cpu``, or ``gil``.
-   The ``cpu`` and ``gil`` modes are incompatible with ``--async-aware``.
+   Sampling mode: ``wall`` (default), ``cpu``, ``gil``, or ``exception``.
+   The ``cpu``, ``gil``, and ``exception`` modes are incompatible with
+   ``--async-aware``.
 
 .. option:: --async-mode <mode>