PEP 810: Add lazy objects to the spec. (#4643)

Add lazy objects to the spec, clarify what happens when reification fails,
tweak some of the language.

Author: Yhg1s

peps/pep-0810.rst

@@ -335,19 +335,29 @@ An import remains lazy only if the filter function returns ``True``.
 
 If no lazy import filter is set, all *potentially lazy* imports are lazy.
 
+Lazy objects
+------------
+
+Lazy modules, as well as names lazy imported from modules, are represented
+by :class:`!types.LazyImportType` instances, which are resolved to the real
+object (reified) before they can be used. This reification is usually done
+automatically (see below), but can also be done by calling the lazy object's
+``get`` method.
+
 Lazy import mechanism
 ---------------------
 
 When an import is lazy, ``__lazy_import__`` is called instead of
 ``__import__``. ``__lazy_import__`` has the same function signature as
 ``__import__``. It adds the module name to ``sys.lazy_modules``, a set of
 fully-qualified module names which have been lazily imported at some point
-(primarily for diagnostics and introspection), and returns a "lazy module
-object."
+(primarily for diagnostics and introspection), and returns a
+:class:`!types.LazyImportType`` object for the module.
 
 The implementation of ``from ... import`` (the ``IMPORT_FROM`` bytecode
 implementation) checks if the module it's fetching from is a lazy module
-object, and if so, returns a lazy object for each name instead.
+object, and if so, returns a :class:`!types.LazyImportType` for each name
+instead.
 
 The end result of this process is that lazy imports (regardless of how they
 are enabled) result in lazy objects being assigned to global variables.
@@ -356,34 +366,36 @@ Lazy module objects do not appear in ``sys.modules``, they're just listed in
 the ``sys.lazy_modules`` set. Under normal operation lazy objects should only
 end up stored in global variables, and the common ways to access those
 variables (regular variable access, module attributes) will resolve lazy
-imports ("reify") and replace them when they're accessed.
+imports (reify) and replace them when they're accessed.
 
 It is still possible to expose lazy objects through other means, like
 debuggers. This is not considered a problem.
 
 Reification
 -----------
 
-When a lazy object is first used, it needs to be reified. This means resolving
-the import at that point in the program and replacing the lazy object with the
-concrete one. Reification imports the module in the same way as it would have
-been if it had been imported eagerly. Notably, reification still calls
-``__import__`` to resolve the import, which uses the state of the import system
-(e.g. ``sys.path``, ``sys.meta_path``, ``sys.path_hooks`` and ``__import__``)
-at **reification** time, **not** the state when the ``lazy import`` statement
-was evaluated.
-
-When the module is first reified, it's removed from ``sys.lazy_modules`` (even
-if there are still other unreified lazy references to it). When a package is
+When a lazy object is used, it needs to be reified. This means resolving the
+import at that point in the program and replacing the lazy object with the
+concrete one. Reification imports the module at that point in the program.
+Notably, reification still calls ``__import__`` to resolve the import, which
+uses the state of the import system (e.g. ``sys.path``, ``sys.meta_path``,
+``sys.path_hooks`` and ``__import__``) at **reification** time, **not** the
+state when the ``lazy import`` statement was evaluated.
+
+When the module is reified, it's removed from ``sys.lazy_modules`` (even if
+there are still other unreified lazy references to it). When a package is
 reified and submodules in the package were also previously lazily imported,
 those submodules are *not* automatically reified but they *are* added to the
-reified package's globals (unless the package already assigned something else
-to the name of the submodule).
+reified package's globals (unless the package already assigned something
+else to the name of the submodule).
 
-If reification fails (e.g., due to an ``ImportError``), the exception is
-enhanced with chaining to show both where the lazy import was defined and
-where it was first accessed (even though it propagates from the code that
-triggered reification). This provides clear debugging information:
+If reification fails (e.g., due to an ``ImportError``), the lazy object is
+*not* reified or replaced. Subsequent uses of the lazy object will re-try
+the reification. Exceptions that happen during reification are raised as
+normal, but the exception is enhanced with chaining to show both where the
+lazy import was defined and where it was accessed (even though it propagates
+from the code that triggered reification). This provides clear debugging
+information:
 
 .. code-block:: python
 
@@ -469,7 +481,7 @@ Example using ``globals()``:
   g = globals()
 
   print('json' in sys.modules)  # False - still lazy
-  print(type(g['json']))  # <class 'lazy_import'>
+  print(type(g['json']))  # <class 'LazyImport'>
 
   # Explicitly reify using the get() method
   resolved = g['json'].get()
@@ -704,7 +716,7 @@ These changes are limited to bindings explicitly made lazy:
 
 * **Error timing.** Exceptions that would have occurred during an eager import
   (for example ``ImportError`` or ``AttributeError`` for a missing member) now
-  occur at the first *use* of the lazy name.
+  occur at the *use* of the lazy name.
 
   .. code-block:: python
 
@@ -714,7 +726,7 @@ These changes are limited to bindings explicitly made lazy:
     # With lazy import - error deferred
     lazy import broken_module
     print("Import succeeded")
-    broken_module.foo()  # ImportError raised here on first use
+    broken_module.foo()  # ImportError raised here on use
 
 * **Side-effect timing.** Import-time side effects in lazily imported modules
   occur at first use of the binding, not at module import time.
@@ -727,16 +739,17 @@ These changes are limited to bindings explicitly made lazy:
   when it is first used.
 * **Proxy visibility.** Before first use, the bound name refers to a lazy
   proxy. Indirect introspection that touches the value may observe a proxy
-  lazy object representation. After first use, the name is rebound to the real
-  object and becomes indistinguishable from an eager import.
+  lazy object representation. After first use (provied the module was
+  imported succesfully), the name is rebound to the real object and becomes
+  indistinguishable from an eager import.
 
 Thread-safety and reification
 -----------------------------
 
-First use of a lazy binding follows the existing import-lock discipline.
-Exactly one thread performs the import and **atomically rebinds** the
-importing module's global to the resolved object. Concurrent readers
-thereafter observe the real object.
+Reification follows the existing import-lock discipline. Exactly one thread
+performs the import and **atomically rebinds** the importing module's global
+to the resolved object. Concurrent readers thereafter observe the real
+object.
 
 Lazy imports are thread-safe and have no special considerations for
 free-threading. A module that would normally be imported in the main thread
@@ -758,11 +771,11 @@ code that doesn't.
 Runtime performance
 ~~~~~~~~~~~~~~~~~~~
 
-After reification (first use), lazy imports have **zero overhead**. The
-adaptive interpreter specializes the bytecode (typically after 2-3 accesses),
-eliminating any checks. For example, ``LOAD_GLOBAL`` becomes
-``LOAD_GLOBAL_MODULE``, which directly accesses the module identically to
-normal imports.
+After reification (provided the import was succesful), lazy imports have
+**zero overhead**. The adaptive interpreter specializes the bytecode
+(typically after 2-3 accesses), eliminating any checks. For example,
+``LOAD_GLOBAL`` becomes ``LOAD_GLOBAL_MODULE``, which directly accesses the
+module identically to normal imports.
 
 The `pyperformance suite`_ confirms the implementation is performance-neutral.
 
@@ -1033,6 +1046,9 @@ it was first used:
       1/0
   ZeroDivisionError: division by zero
 
+Exceptions during reification prevent the replacement of the lazy object,
+and subsequent uses of the lazy object will retry the whole reification.
+
 How do lazy imports affect modules with import-time side effects?
 -----------------------------------------------------------------
 
@@ -1118,8 +1134,8 @@ What's the performance overhead of lazy imports?
 
 The overhead is minimal:
 
-- Zero overhead after first use thanks to the adaptive interpreter optimizing
-  the slow path away.
+- Zero overhead after first use (provided the import doesn't fail) thanks to
+  the adaptive interpreter optimizing the slow path away.
 - Small one-time cost to create the proxy object.
 - Reification (first use) has the same cost as a regular import.
 - No ongoing performance penalty.
@@ -1159,7 +1175,7 @@ error. If lazy imports are globally enabled, star imports will still be eager.
 How do lazy imports interact with import hooks and custom loaders?
 ------------------------------------------------------------------
 
-Import hooks and loaders work normally. When a lazy object is first used,
+Import hooks and loaders work normally. When a lazy object is used,
 the standard import protocol runs, including any custom hooks or loaders that
 were in place at reification time.
 
@@ -1191,7 +1207,7 @@ reifies all lazy imports in that module first. This design ensures:
   lazy import json
 
   g = globals()
-  print(type(g['json']))  # <class 'lazy_import'> - your problem
+  print(type(g['json']))  # <class 'LazyImport'> - your problem
 
   # From external code:
   import sys
@@ -1390,11 +1406,11 @@ The best practice is still to avoid circular imports in your code design.
 Will lazy imports affect the performance of my hot paths?
 ---------------------------------------------------------
 
-After first use, lazy imports have **zero overhead** thanks to the adaptive
-interpreter. The interpreter specializes the bytecode (e.g., ``LOAD_GLOBAL``
-becomes ``LOAD_GLOBAL_MODULE``) which eliminates the lazy check on subsequent
-accesses. This means once a lazy import is reified, accessing it is just as
-fast as a normal import.
+After first use (provided the import succeed), lazy imports have **zero
+overhead** thanks to the adaptive interpreter. The interpreter specializes
+the bytecode (e.g., ``LOAD_GLOBAL`` becomes ``LOAD_GLOBAL_MODULE``) which
+eliminates the lazy check on subsequent accesses. This means once a lazy
+import is reified, accessing it is just as fast as a normal import.
 
 .. code-block:: python