diff --git a/peps/pep-0810.rst b/peps/pep-0810.rst index 04c256585db..e67b5d19d5c 100644 --- a/peps/pep-0810.rst +++ b/peps/pep-0810.rst @@ -101,7 +101,7 @@ binding is indistinguishable from one created by a normal import. This clarity reduces surprises and makes the feature accessible to developers who may not be deeply familiar with Python’s import machinery. -Lazy imports are **controlled**, in the sense that deferred loading is only +Lazy imports are **controlled**, in the sense that lazy loading is only triggered by the importing code itself. In the general case, a library will only experience lazy imports if its own authors choose to mark them as such. This avoids shifting responsibility onto downstream users and prevents @@ -206,9 +206,11 @@ Other design decisions context. * In addition, it is useful to provide a mechanism to activate or deactivate - lazy imports at a global level. While the primary design centers on explicit - syntax, there are scenarios -- such as large applications, testing - environments, or frameworks -- where enabling laziness consistently across + lazy imports for all code running in the interpreter + (referred to in this PEP as the 'global lazy imports flag'). + While the primary design centers the explicit ``lazy import`` syntax, + there are scenarios -- such as large applications, testing environments, + or frameworks -- where enabling laziness consistently across many modules provides the most benefit. A global switch makes it easy to experiment with or enforce consistent behavior, while still working in combination with the filtering API to respect exclusions or tool-specific @@ -320,11 +322,14 @@ If the global lazy imports flag is set to "disabled", no *potentially lazy* import is ever imported lazily, and the behavior is equivalent to a regular import statement: the import is *eager* (as if the lazy keyword was not used). -For a *potentially lazy* import, the lazy imports filter (if set) is called -with the name of the module doing the import, the name of the module being -imported, and (if applicable) the fromlist. If the lazy import filter returns -``True``, the *potentially lazy* import becomes a lazy import. Otherwise, the -import is *not* lazy, and the normal (eager) import continues. +Finally, the application may use a custom filter function on all *potentially +lazy* imports to determine if they should be lazy or not. +If a filter function is set, it will be called with the name of the module +doing the import, the name of the module being imported, and (if applicable) +the fromlist. +An import remains lazy only if the filter function returns ``True``. + +If no lazy import filter is set, all *potentially lazy* imports are lazy. Lazy import mechanism --------------------- @@ -395,7 +400,7 @@ The traceback shows both locations: Traceback (most recent call last): File "app.py", line 2, in lazy from json import dumsp - ImportError: deferred import of 'json.dumsp' raised an exception during resolution + ImportError: lazy import of 'json.dumsp' raised an exception during resolution The above exception was the direct cause of the following exception: @@ -405,9 +410,11 @@ The traceback shows both locations: ^^^^^ ImportError: cannot import name 'dumsp' from 'json'. Did you mean: 'dump'? -This exception chaining clearly shows: (1) where the lazy import was defined, -(2) that it was deferred, and (3) where the actual access happened that -triggered the error. +This exception chaining clearly shows: + +(1) where the lazy import was defined, +(2) that the module was not eagerly imported, and +(3) where the actual access happened that triggered the error. Reification does **not** automatically occur when a module that was previously lazily imported is subsequently eagerly imported. Reification does **not** @@ -652,6 +659,11 @@ lazy* import is ever imported lazily, the import filter is never called, and the behavior is equivalent to a regular ``import`` statement: the import is *eager* (as if the lazy keyword was not used). +Python code can run the :func:`!sys.set_lazy_imports` function to override +the state of the global lazy imports flag inherited from the environment or CLI. +This is especially useful if an application needs to ensure that all imports +are evaluated eagerly, via ``sys.set_lazy_imports('disabled')``. + Backwards Compatibility ======================= @@ -817,9 +829,10 @@ avoid incompatibilities: FAQ === -**Q: How does this differ from the rejected PEP 690?** +How does this differ from the rejected PEP 690? +----------------------------------------------- -A: PEP 810 takes an explicit, opt-in approach instead of :pep:`690`'s implicit +PEP 810 takes an explicit, opt-in approach instead of :pep:`690`'s implicit global approach. The key differences are: - **Explicit syntax**: ``lazy import foo`` clearly marks which imports are @@ -829,9 +842,10 @@ global approach. The key differences are: - **Simpler implementation**: Uses proxy objects instead of modifying core dictionary behavior. -**Q: What happens when lazy imports encounter errors?** +What happens when lazy imports encounter errors? +------------------------------------------------ -A: Import errors (``ImportError``, ``ModuleNotFoundError``, syntax errors) are +Import errors (``ImportError``, ``ModuleNotFoundError``, syntax errors) are deferred until first use of the lazy name. This is similar to moving an import into a function. The error will occur with a clear traceback pointing to the first access of the lazy object. @@ -846,7 +860,7 @@ it was first used: Traceback (most recent call last): File "test.py", line 1, in lazy import broken_module - ImportError: deferred import of 'broken_module' raised an exception during resolution + ImportError: lazy import of 'broken_module' raised an exception during resolution The above exception was the direct cause of the following exception: @@ -858,9 +872,10 @@ it was first used: 1/0 ZeroDivisionError: division by zero -**Q: How do lazy imports affect modules with import-time side effects?** +How do lazy imports affect modules with import-time side effects? +----------------------------------------------------------------- -A: Side effects are deferred until first use. This is generally desirable for +Side effects are deferred until first use. This is generally desirable for performance, but may require code changes for modules that rely on import-time registration patterns. We recommend: @@ -868,16 +883,17 @@ registration patterns. We recommend: - Call initialization functions explicitly when needed - Avoid relying on import order for side effects -**Q: Can I use lazy imports with** ``from ... import ...`` **statements?** +Can I use lazy imports with ``from ... import ...`` statements? +--------------------------------------------------------------- -A: Yes, as long as you don't use ``from ... import *``. Both ``lazy import +Yes, as long as you don't use ``from ... import *``. Both ``lazy import foo`` and ``lazy from foo import bar`` are supported. The ``bar`` name will be bound to a lazy object that resolves to ``foo.bar`` on first use. -**Q: Does** ``lazy from module import Class`` **load the entire module or just -the class?** +Does ``lazy from module import Class`` load the entire module or just the class? +-------------------------------------------------------------------------------- -A: It loads the **entire module**, not just the class. This is because +It loads the **entire module**, not just the class. This is because Python's import system always executes the complete module file -- there's no mechanism to execute only part of a ``.py`` file. When you first access ``Class``, Python: @@ -912,9 +928,10 @@ at the import statement. loaded. You cannot selectively load only parts of a module -- Python's import system doesn't support partial module execution. -**Q: What about type annotations and** ``TYPE_CHECKING`` **imports?** +What about type annotations and ``TYPE_CHECKING`` imports? +---------------------------------------------------------- -A: Lazy imports eliminate the common need for ``TYPE_CHECKING`` guards. You +Lazy imports eliminate the common need for ``TYPE_CHECKING`` guards. You can write: .. code-block:: python @@ -935,9 +952,10 @@ Instead of: def process(items: Sequence[str]) -> Mapping[str, int]: ... -**Q: What's the performance overhead of lazy imports?** +What's the performance overhead of lazy imports? +------------------------------------------------ -A: The overhead is minimal: +The overhead is minimal: - Zero overhead after first use thanks to the adaptive interpreter optimizing the slow path away. @@ -948,51 +966,60 @@ A: The overhead is minimal: Benchmarking with the `pyperformance suite`_ shows the implementation is performance neutral when lazy imports are not used. -.. _pyperformance suite: https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20250922-3.15.0a0-27836e5/bm-20250922-vultr-x86_64-DinoV-lazy_imports-3.15.0a0-27836e5-vs-base.svg +.. _pyperformance suite: https://github.com/facebookexperimental/ + free-threading-benchmarking/blob/main/results/bm-20250922-3.15.0a0-27836e5/ + bm-20250922-vultr-x86_64-DinoV-lazy_imports-3.15.0a0-27836e5-vs-base.svg -**Q: Can I mix lazy and eager imports of the same module?** +Can I mix lazy and eager imports of the same module? +---------------------------------------------------- -A: Yes. If module ``foo`` is imported both lazily and eagerly in the same +Yes. If module ``foo`` is imported both lazily and eagerly in the same program, the eager import takes precedence and both bindings resolve to the same module object. -**Q: How do I migrate existing code to use lazy imports?** +How do I migrate existing code to use lazy imports? +--------------------------------------------------- -A: Migration is incremental: +Migration is incremental: 1. Identify slow-loading modules using profiling tools. 2. Add ``lazy`` keyword to imports that aren't needed immediately. 3. Test that side-effect timing changes don't break functionality. 4. Use :data:`!__lazy_modules__` for compatibility with older Python versions. -**Q: What about star imports** (``from module import *``)? +What about star imports (``from module import *``)? +--------------------------------------------------- -A: Wild card (star) imports cannot be lazy - they remain eager. This is +Wild card (star) imports cannot be lazy - they remain eager. This is because the set of names being imported cannot be determined without loading the module. Using the ``lazy`` keyword with star imports will be a syntax error. If lazy imports are globally enabled, star imports will still be eager. -**Q: How do lazy imports interact with import hooks and custom loaders?** +How do lazy imports interact with import hooks and custom loaders? +------------------------------------------------------------------ -A: Import hooks and loaders work normally. When a lazy object is first used, +Import hooks and loaders work normally. When a lazy object is first used, the standard import protocol runs, including any custom hooks or loaders that were in place at reification time. -**Q: What happens in multi-threaded environments?** +What happens in multi-threaded environments? +-------------------------------------------- -A: Lazy import reification is thread-safe. Only one thread will perform the +Lazy import reification is thread-safe. Only one thread will perform the actual import, and the binding is atomically updated. Other threads will see either the lazy proxy or the final resolved object. -**Q: Can I force reification of a lazy import without using it?** +Can I force reification of a lazy import without using it? +---------------------------------------------------------- -A: Yes, accessing a module's ``__dict__`` will reify all lazy objects in that +Yes, accessing a module's ``__dict__`` will reify all lazy objects in that module. Individual lazy objects can be resolved by calling their ``get()`` method. -**Q: What's the difference between** ``globals()`` **and** ``mod.__dict__`` **for lazy imports?** +What's the difference between ``globals()`` and ``mod.__dict__`` for lazy imports? +---------------------------------------------------------------------------------- -A: Calling ``globals()`` returns the module's dictionary without reifying lazy +Calling ``globals()`` returns the module's dictionary without reifying lazy imports -- you'll see lazy proxy objects when accessing them through the returned dictionary. However, accessing ``mod.__dict__`` from external code reifies all lazy imports in that module first. This design ensures: @@ -1015,24 +1042,28 @@ This distinction means adding lazy imports and calling ``globals()`` is your responsibility to manage, while external code accessing ``mod.__dict__`` always sees fully loaded modules. -**Q: Why not use** ``importlib.util.LazyLoader`` **instead?** +Why not use ``importlib.util.LazyLoader`` instead? +-------------------------------------------------- -A: ``LazyLoader`` has significant limitations: +``LazyLoader`` has significant limitations: - Requires verbose setup code for each lazy import. - Has ongoing performance overhead on every attribute access. - Doesn't work well with ``from ... import`` statements. - Less clear and standard than dedicated syntax. -**Q: Will this break tools like** ``isort`` **or** ``black``? +Will this break tools like ``isort`` or ``black``? +-------------------------------------------------- -A: Tools will need updates to recognize the ``lazy`` keyword, but the changes -should be minimal since the import structure remains the same. The keyword -appears at the beginning, making it easy to parse. +Linters, formatters, and other tools will need updates to recognize +the ``lazy`` keyword, but the changes should be minimal since the import +structure remains the same. The keyword appears at the beginning, +making it easy to parse. -**Q: How do I know if a library is compatible with lazy imports?** +How do I know if a library is compatible with lazy imports? +----------------------------------------------------------- -A: Most libraries should work fine with lazy imports. Libraries that might +Most libraries should work fine with lazy imports. Libraries that might have issues: - Those with essential import-time side effects (registration, @@ -1042,10 +1073,10 @@ have issues: When in doubt, test lazy imports with your specific use cases. -**Q: What happens if I globally enable lazy imports mode and a library doesn't -work correctly?** +What happens if I globally enable lazy imports mode and a library doesn't work correctly? +----------------------------------------------------------------------------------------- -A: *Note: This is an advanced feature.* You can use the lazy imports filter to +*Note: This is an advanced feature.* You can use the lazy imports filter to exclude specific modules that are known to have problematic side effects: .. code-block:: python @@ -1067,15 +1098,17 @@ forces an eager import. Alternatively, set the global mode to ``"disabled"`` via ``-X lazy_imports=disabled`` to turn off all lazy imports for debugging. -**Q: Can I use lazy imports inside functions?** +Can I use lazy imports inside functions? +---------------------------------------- -A: No, the ``lazy`` keyword is only allowed at module level. For +No, the ``lazy`` keyword is only allowed at module level. For function-level lazy loading, use traditional inline imports or move the import to module level with ``lazy``. -**Q: What about forwards compatibility with older Python versions?** +What about forwards compatibility with older Python versions? +------------------------------------------------------------- -A: Use the :data:`!__lazy_modules__` global for compatibility: +Use the :data:`!__lazy_modules__` global for compatibility: .. code-block:: python @@ -1097,9 +1130,12 @@ maximum predictability, it's recommended to define :data:`!__lazy_modules__` once, before any imports. But as it is checked on each import, it can be modified between ``import`` statements. -**Q: How do explicit lazy imports interact with PEP-649/PEP-749** +How do explicit lazy imports interact with PEP 649 and PEP 749? +--------------------------------------------------------------- -A: If an annotation is not stringified, it is an expression that is evaluated +Python 3.14 implemented deferred evaluation of annotations, +as specified by :pep:`649` and :pep:`749`. +If an annotation is not stringified, it is an expression that is evaluated at a later time. It will only be resolved if the annotation is accessed. In the example below, the ``fake_typing`` module is only loaded when the user inspects the ``__annotations__`` dictionary. The ``fake_typing`` module would @@ -1113,10 +1149,10 @@ also be loaded if the user uses ``annotationlib.get_annotations()`` or pass print(foo.__annotations__) # Triggers loading the fake_typing module -**Q: How do lazy imports interact with** ``dir()``, ``getattr()``, **and -module introspection?** +How do lazy imports interact with ``dir()``, ``getattr()``, and module introspection? +------------------------------------------------------------------------------------- -A: Accessing lazy imports through normal attribute access or ``getattr()`` +Accessing lazy imports through normal attribute access or ``getattr()`` will trigger reification. Calling ``dir()`` on a module will reify all lazy imports in that module to ensure the directory listing is complete. This is similar to accessing ``mod.__dict__``. @@ -1134,9 +1170,10 @@ similar to accessing ``mod.__dict__``. dir(json) # Now json is in sys.modules -**Q: Do lazy imports work with circular imports?** +Do lazy imports work with circular imports? +------------------------------------------- -A: Lazy imports don't automatically solve circular import problems. If two +Lazy imports don't automatically solve circular import problems. If two modules have a circular dependency, making the imports lazy might help **only if** the circular reference isn't accessed during module initialization. However, if either module accesses the other during import time, you'll still @@ -1190,9 +1227,10 @@ which then tries to access ``module_a`` before it's fully initialized. The best practice is still to avoid circular imports in your code design. -**Q: Will lazy imports affect the performance of my hot paths?** +Will lazy imports affect the performance of my hot paths? +--------------------------------------------------------- -A: After first use, lazy imports have **zero overhead** thanks to the adaptive +After first use, lazy imports have **zero overhead** thanks to the adaptive interpreter. The interpreter specializes the bytecode (e.g., ``LOAD_GLOBAL`` becomes ``LOAD_GLOBAL_MODULE``) which eliminates the lazy check on subsequent accesses. This means once a lazy import is reified, accessing it is just as @@ -1227,9 +1265,10 @@ You can observe the specialization using ``dis.dis(use_json, adaptive=True)``: The specialized ``LOAD_GLOBAL_MODULE`` and ``LOAD_ATTR_MODULE`` instructions are optimized fast paths with no overhead for checking lazy imports. -**Q: What about** ``sys.modules``? **When does a lazy import appear there?** +What about ``sys.modules``? When does a lazy import appear there? +----------------------------------------------------------------- -A: A lazily imported module does **not** appear in ``sys.modules`` until it's +A lazily imported module does **not** appear in ``sys.modules`` until it's reified (first used). Once reified, it appears in ``sys.modules`` just like any eager import. @@ -1244,9 +1283,10 @@ any eager import. print('json' in sys.modules) # True -**Q: Why you chose ``lazy`` as the keyword name?** +Why you chose ``lazy`` as the keyword name? +------------------------------------------- -A: Not "why"... memorize! :) +Not "why"... memorize! :) Alternate Implementation Ideas ==============================