From eb7481fa866b2accc7427b005ddecfaef4668411 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 22 Sep 2025 11:57:23 +0100 Subject: [PATCH 01/39] PEP 805: Explicit lazy imports --- peps/pep-0805.rst | 88 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 peps/pep-0805.rst diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst new file mode 100644 index 00000000000..1682ca3a70c --- /dev/null +++ b/peps/pep-0805.rst @@ -0,0 +1,88 @@ +PEP: 805 +Title: Explicit lazy import +Author: +Discussions-To: Pending +Status: Draft +Type: Standards Track +Topic: Release +Created: 22-09-2025 +Python-Version: 3.15 + + +Abstract +======== + +[A short (~200 word) description of the technical issue being addressed.] + + +Motivation +========== + +[Clearly explain why the existing language specification is inadequate to address the problem that the PEP solves.] + + +Rationale +========= + +[Describe why particular design decisions were made.] + + +Specification +============= + +[Describe the syntax and semantics of any new language feature.] + + +Backwards Compatibility +======================= + +[Describe potential impact and severity on pre-existing code.] + + +Security Implications +===================== + +[How could a malicious user take advantage of this new feature?] + + +How to Teach This +================= + +[How to teach users, new and experienced, how to apply the PEP to their work.] + + +Reference Implementation +======================== + +[Link to any existing implementation and details about its state, e.g. proof-of-concept.] + + +Rejected Ideas +============== + +[Why certain ideas that were brought while discussing this PEP were not ultimately pursued.] + + +Open Issues +=========== + +[Any points that are still being decided/discussed.] + + +Acknowledgements +================ + +[Thank anyone who has helped with the PEP.] + + +Footnotes +========= + +[A collection of footnotes cited in the PEP, and a place to list non-inline hyperlink targets.] + + +Copyright +========= + +This document is placed in the public domain or under the +CC0-1.0-Universal license, whichever is more permissive. From bb2881eb559ac909b3978aa90e56bd5b7f4e7e59 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 22 Sep 2025 15:40:44 +0100 Subject: [PATCH 02/39] Add Motivation/Rationale --- peps/pep-0523.rst | 2 +- peps/pep-0805.rst | 146 ++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 141 insertions(+), 7 deletions(-) diff --git a/peps/pep-0523.rst b/peps/pep-0523.rst index d8def41b485..4d4c02b0013 100644 --- a/peps/pep-0523.rst +++ b/peps/pep-0523.rst @@ -1,7 +1,7 @@ PEP: 523 Title: Adding a frame evaluation API to CPython Author: Brett Cannon , - Dino Viehland + Thomas Wouters Status: Final Type: Standards Track Created: 16-May-2016 diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 1682ca3a70c..222be73be2d 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -1,6 +1,12 @@ PEP: 805 -Title: Explicit lazy import -Author: +Title: Explicit lazy imports +Author: Pablo Galindo , + Germán Méndez Bravo , + Thomas Wouters , + Dino Viehland , + Brittany Reynoso , + Noah Kim , + Tim Stumbaugh Discussions-To: Pending Status: Draft Type: Standards Track @@ -12,19 +18,147 @@ Python-Version: 3.15 Abstract ======== -[A short (~200 word) description of the technical issue being addressed.] +This PEP introduces lazy imports as an explicit language feature. Lazy imports +defer the loading and execution of a module until the first time the imported +name is used, rather than at the point of the import statement. +By allowing developers to mark imports as lazy with new syntax, Python programs +can reduce startup time, memory usage, and unnecessary work. This is +particularly beneficial for command-line tools, test suites, and applications +with large dependency graphs. + +The proposal preserves full backwards compatibility: normal import statements +remain unchanged, and lazy imports are enabled only where explicitly requested. Motivation ========== -[Clearly explain why the existing language specification is inadequate to address the problem that the PEP solves.] - +By convention, Python places imports at module level to avoid repetition and +extra overhead. The drawback is that importing a main module often triggers a +cascade of imports, loading many dependencies that may never be used. The +effect is especially costly for command-line tools with multiple subcommands, +where even a simple ``--help`` can load dozens of unnecessary modules. This is +particularly bad for big CLIs that grew organically. + +Some projects try to delay imports by moving them into functions, but this is +fragile and hard to maintain. Existing tools such as +``importlib.util.LazyLoader`` or third-party packages provide partial solutions +but either fail to cover all cases or add runtime overhead. Scientific Python +libraries have adopted similar patterns, formalized in `SPEC 1 +`_ and the ``lazy_loader`` +package, and typing imports are another common case where eager loading is +wasteful. + +This proposal introduces lazy imports with a design that is local, explicit, +controlled, and granular. Each of these qualities is essential to making the +feature predictable and safe to use in practice. + +The behavior is **local**: laziness applies only to the specific import marked +with the ``lazy`` keyword, and it does not cascade recursively into other +imports. This ensures that developers can reason about the effect of laziness +by looking only at the line of code in front of them, without worrying about +whether imported modules will themselves behave differently. A ``lazy import`` +is an isolated decision, not a global shift in semantics. + +The semantics are also **explicit**. When a name is imported lazily, the binding +is created in the importing module immediately, but the target module is not +loaded until the first time the name is accessed. After this point, the binding +is indistinguishable from a normal import. This clarity reduces surprises and +makes the feature accessible to developers who may not be deeply familiar with +Python’s import machinery. + +Lazy imports are **controlled**, in the sense that deferred loading is only +triggered by the importing code itself. In the general case, a library will only +experience lazy imports if its own authors choose to mark them as such. This +avoids shifting responsibility onto downstream users and prevents accidental +surprises in library behavior. Since library authors typically manage their own +import subgraphs, they retain predictable control over when and how laziness is +applied. + +The mechanism is also **granular**. It is introduced through explicit syntax on +individual imports, rather than a global flag or implicit setting. This allows +developers to adopt it incrementally, starting with the most +performance-sensitive areas of a codebase. The experience is similar to the way +typing was introduced into Python: optional, progressive, and adaptable to the +needs of each project. + +The design of lazy imports provides several concrete advantages: + +* Startup latency is critical for command-line tools, many of which are + short-lived processes. A CLI with multiple subcommands typically imports + every dependency up front, even if the user only requests ``tool --help``. + With lazy imports, only the code paths actually reached trigger module + loading. This can reduce startup time by 50–70% in practice, restoring + Python's competitiveness in domains where fast startup matters most. + +* Type annotations frequently require imports that are never used at runtime. + The common workaround is to wrap them in ``if TYPE_CHECKING:`` blocks, which + adds boilerplate and complexity. With lazy imports, annotation-only imports + impose no runtime penalty, eliminating the need for guards and making + annotated codebases cleaner. + +* Large applications often import thousands of modules, each adding objects and + state to memory. In long-lived processes this raises baseline memory usage + and reduces stability under load. Lazy imports defer this cost until a module + is needed, keeping unused subsystems unloaded. Memory savings of 30–40% have + been observed in real workloads. Rationale ========= -[Describe why particular design decisions were made.] +The design of this proposal is centered on clarity, predictability, and ease of +adoption. Each decision was made to ensure that lazy imports provide tangible +benefits without introducing unnecessary complexity into the language or its +runtime. + +It is also worth noting that while this PEP outlines one specific approach, we +list alternate implementation strategies for some of the core aspects and +semantics of the proposal. If the community expresses a strong preference for a +different technical path that still preserves the same core semantics or there +is fundamental disagreement over the specific option, the Steering Council can +easily chose to accept the propisal with what they thing is the best set of +options for the language. + +The choice to introduce a new ``lazy`` keyword reflects the need for explicit +syntax. Import behavior is too fundamental to be left implicit or hidden behind +global flags or environment variables. By marking laziness directly at the +import site, the intent is immediately visible to both readers and tools. This +avoids surprises, reduces the cognitive burden of reasoning about imports, and +keeps the semantics in line with Python's tradition of explicitness. + +Another important decision is to represent lazy imports with proxy objects in +the module's namespace, rather than by modifying dictionary lookup. Earlier +approaches experimented with embedding laziness into dictionaries, but this +blurred abstractions and risked affecting unrelated parts of the runtime. The +proxy approach is simpler: it behaves like a placeholder until first use, at +which point it resolves the import and rebinds the name. From then on, the +binding is indistinguishable from a normal import. This makes the mechanism +easy to explain and keeps the rest of the interpreter unchanged. + +The scope of laziness is deliberately local and non-recursive. A lazy import +only affects the specific statement where it appears; it does not cascade into +other modules or submodules. This choice is crucial for predictability. When +developers read code, they can reason about import behavior line by line, +without worrying about hidden laziness deeper in the dependency graph. The +result is a feature that is powerful but still easy to understand in context. + +Compatibility for library authors was also a key concern. Many maintainers need +a migration path that allows them to support both new and old versions of +Python at once. For this reason, the proposal includes the ``__lazy_modules__`` +global as a transitional mechanism. A module can declare which imports should +be treated as lazy, and on Python 3.15 or later those imports will become lazy +automatically. On earlier versions the declaration is ignored, leaving imports +eager. This gives authors a practical bridge until they can rely on the keyword +as the canonical syntax. + +Finally, the feature is designed to be adopted incrementally. Nothing changes +unless a developer explicitly opts in, and adoption can begin with just a few +imports in performance-sensitive areas. This mirrors the experience of gradual +typing in Python: a mechanism that can be introduced progressively, without +forcing projects to commit globally from day one. By combining explicit syntax, +a simple runtime model, a compatibility layer, and gradual adoption, this +proposal balances performance improvements with the clarity and stability that +Python users expect. Specification From 900d69c1140d09f152ae7458241b3bc8cadeb2b3 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 22 Sep 2025 15:42:17 +0100 Subject: [PATCH 03/39] Add FAQ --- peps/pep-0805.rst | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 222be73be2d..29a4fcc2244 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -185,6 +185,11 @@ How to Teach This [How to teach users, new and experienced, how to apply the PEP to their work.] +FAQ +=== + +[A list of frequently asked questions with answers] + Reference Implementation ======================== From dec56a66d48d787da6c51754ff777b8184daba82 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Mon, 22 Sep 2025 17:56:08 +0100 Subject: [PATCH 04/39] restructure sections --- peps/pep-0805.rst | 42 +++++++++++++++++++++++++++++++----------- 1 file changed, 31 insertions(+), 11 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 29a4fcc2244..1b3742440f0 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -35,9 +35,9 @@ Motivation By convention, Python places imports at module level to avoid repetition and extra overhead. The drawback is that importing a main module often triggers a -cascade of imports, loading many dependencies that may never be used. The -effect is especially costly for command-line tools with multiple subcommands, -where even a simple ``--help`` can load dozens of unnecessary modules. This is +cascade of imports, loading many dependencies that may never be used. The effect +is especially costly for command-line tools with multiple subcommands, where +even a simple ``--help`` can load dozens of unnecessary modules. This is particularly bad for big CLIs that grew organically. Some projects try to delay imports by moving them into functions, but this is @@ -135,13 +135,6 @@ which point it resolves the import and rebinds the name. From then on, the binding is indistinguishable from a normal import. This makes the mechanism easy to explain and keeps the rest of the interpreter unchanged. -The scope of laziness is deliberately local and non-recursive. A lazy import -only affects the specific statement where it appears; it does not cascade into -other modules or submodules. This choice is crucial for predictability. When -developers read code, they can reason about import behavior line by line, -without worrying about hidden laziness deeper in the dependency graph. The -result is a feature that is powerful but still easy to understand in context. - Compatibility for library authors was also a key concern. Many maintainers need a migration path that allows them to support both new and old versions of Python at once. For this reason, the proposal includes the ``__lazy_modules__`` @@ -161,6 +154,34 @@ proposal balances performance improvements with the clarity and stability that Python users expect. +Other design dessicions +----------------------- + +* The scope of laziness is deliberately local and non-recursive. A lazy import + only affects the specific statement where it appears; it does not cascade into + other modules or submodules. This choice is crucial for predictability. When + developers read code, they can reason about import behavior line by line, + without worrying about hidden laziness deeper in the dependency graph. The + result is a feature that is powerful but still easy to understand in context. + +* In addition, it is useful to provide a mechanism to activate or deactivate lazy + imports at a global level. While the primary design centers on explicit syntax, + there are scenarios—such as large applications, testing environments, or + frameworks—where enabling laziness consistently across many modules provides + the most benefit. A global switch makes it easy to experiment with or enforce + consistent behavior, while still working in combination with the filtering API + to respect exclusions or tool-specific configuration. This ensures that global + adoption can be practical without reducing flexibility or control. + +* In addition, it is useful to provide a mechanism to activate or deactivate lazy + imports at a global level. While the primary design centers on explicit syntax, + there are scenarios—such as large applications, testing environments, or + frameworks—where enabling laziness consistently across many modules provides + the most benefit. A global switch makes it easy to experiment with or enforce + consistent behavior, while still working in combination with the filtering API + to respect exclusions or tool-specific configuration. This ensures that global + adoption can be practical without reducing flexibility or control. + Specification ============= @@ -184,7 +205,6 @@ How to Teach This [How to teach users, new and experienced, how to apply the PEP to their work.] - FAQ === From f8bc20d64d6d6d503729ba28abd4e0aec593552e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20M=C3=A9ndez=20Bravo?= Date: Mon, 22 Sep 2025 13:12:45 -0700 Subject: [PATCH 05/39] Update pep-0805.rst Fixed a few typos and a duplicate point --- peps/pep-0805.rst | 16 ++++------------ 1 file changed, 4 insertions(+), 12 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 1b3742440f0..33919c30c18 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -116,7 +116,7 @@ list alternate implementation strategies for some of the core aspects and semantics of the proposal. If the community expresses a strong preference for a different technical path that still preserves the same core semantics or there is fundamental disagreement over the specific option, the Steering Council can -easily chose to accept the propisal with what they thing is the best set of +easily choose to accept the proposal with what they think is the best set of options for the language. The choice to introduce a new ``lazy`` keyword reflects the need for explicit @@ -154,8 +154,8 @@ proposal balances performance improvements with the clarity and stability that Python users expect. -Other design dessicions ------------------------ +Other design decisions +---------------------- * The scope of laziness is deliberately local and non-recursive. A lazy import only affects the specific statement where it appears; it does not cascade into @@ -173,15 +173,7 @@ Other design dessicions to respect exclusions or tool-specific configuration. This ensures that global adoption can be practical without reducing flexibility or control. -* In addition, it is useful to provide a mechanism to activate or deactivate lazy - imports at a global level. While the primary design centers on explicit syntax, - there are scenarios—such as large applications, testing environments, or - frameworks—where enabling laziness consistently across many modules provides - the most benefit. A global switch makes it easy to experiment with or enforce - consistent behavior, while still working in combination with the filtering API - to respect exclusions or tool-specific configuration. This ensures that global - adoption can be practical without reducing flexibility or control. - + Specification ============= From 7be3240a3ab8d90d265c130049e432c4dd4109f1 Mon Sep 17 00:00:00 2001 From: Thomas Wouters Date: Tue, 23 Sep 2025 12:56:28 +0200 Subject: [PATCH 06/39] Draft of the specification section. --- peps/pep-0805.rst | 79 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 78 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 33919c30c18..98c8c26a6de 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -177,7 +177,84 @@ Other design decisions Specification ============= -[Describe the syntax and semantics of any new language feature.] +A new soft keyword is added, ``lazy`` (other names have been suggested), which +can appear in front of both forms of import:: + + - ``lazy import spam`` + - ``lazy from spam import ham`` + +The soft keyword is only allowed at the global level, so not inside +functions. It's also not allowed in try blocks. Import statements that use +the soft keyword are potentially lazy. + +If the importing module has a ``__lazy_modules__`` attribute, it contains a +list of names the imports of which are potentially lazy (as if the lazy +keyword was used). + +The normal (non-lazy) import statement also checks the global lazy imports +flag. If it is ``"enabled"``, all imports at the global level that are not in +a try block are potentially lazy. + +If the global lazy imports flag is set to ``"disabled"``, the import is _not_ +lazy (as if the lazy keyword was not used). + +For potentially lazy imports, the lazy imports filter (if set) is called, +passing the name of the module doing the import, the module being imported, +and (if applicable) the fromlist. If the lazy import filter returns True the +lazy import continues. Otherwise, the import is _not_ lazy, and the normal +import continues. + +When an import is lazy, ``__lazy_import__`` is called instead of ``__import__``. +``__lazy_import__`` will: + + - Check if the module already exists in ``sys.modules``, and return that if + it is. + - Add the module to ``sys.lazy_modules``, a set of modules which have been + lazily imported. + - Return a "lazy module object". + +The implementation of ``from ... import`` (the ``IMPORT_FROM`` bytecode +implementation) checks if the module it's fetching from is a lazy module +object, and if so, returns lazy objects for each name instead. + +The end result of this process is that lazy imports (regardless of how they +are enabled) result in lazy objects being assigned to global variables. + +Lazy module objects do not appear in ``sys.modules``, just the +``sys.lazy_modules`` set. Lazy objects should only end up stored in global +variables, and the common ways to access those variables (regular variable +access, module attributes, ``globals()``) will resolve lazy imports ("reify") +and replace them when they're accessed. + +It is still possible to expose lazy objects through other means, like +debuggers. This is not considered a problem. + +Reification +----------- + +When a lazy object is first used, it needs to be reified. This means +resolving the import at that point in the program, and replacing the lazy +object with the concrete one. Reification imports the module the same as it +would have been if it had been imported eagerly, barring changes to the +import system (like changes to ``sys.path``, ``sys.meta_path``, ``sys.path_hooks`` +or ``__import__``). + +Reification calls ``__import__`` to resolve the import. Once the module is +reified it's removed from ``sys.lazy_modules``. + +Reification does _not_ automatically occur when a module that was lazily +imported before is eagerly imported. Reification does _not_ resolve all lazy +objects referencing the module. It only resolves the lazy object being +accessed. + +Accessing a lazy object (from a global variable or a module attribute) +reifies the object. Accessing a module's ``__dict__`` or calling ``globals()`` +(and ``locals()`` at the global level, where it means the same as ``globals()``) +reifies _all_ lazy objects in that module. + +More indirect ways of accessing arbitrary globals (e.g. inspecting +``frame.f_globals``) does _not_ reify all the objects. A lazy object can be +resolved explicitly by calling the ``get`` method. Backwards Compatibility From d40b989d379418e4d9176d9b5e134023eee1cd5d Mon Sep 17 00:00:00 2001 From: Thomas Wouters Date: Tue, 23 Sep 2025 14:12:39 +0100 Subject: [PATCH 07/39] Fix the damn reST markup. --- peps/pep-0805.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 98c8c26a6de..f57936a47e1 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -178,10 +178,11 @@ Specification ============= A new soft keyword is added, ``lazy`` (other names have been suggested), which -can appear in front of both forms of import:: +can appear in front of both forms of import: - - ``lazy import spam`` - - ``lazy from spam import ham`` +* ``lazy import spam`` + +* ``lazy from spam import ham`` The soft keyword is only allowed at the global level, so not inside functions. It's also not allowed in try blocks. Import statements that use @@ -207,11 +208,11 @@ import continues. When an import is lazy, ``__lazy_import__`` is called instead of ``__import__``. ``__lazy_import__`` will: - - Check if the module already exists in ``sys.modules``, and return that if - it is. - - Add the module to ``sys.lazy_modules``, a set of modules which have been - lazily imported. - - Return a "lazy module object". +* Check if the module already exists in ``sys.modules``, and return that if it is. + +* Add the module to ``sys.lazy_modules``, a set of modules which have been lazily imported. + +* Return a "lazy module object". The implementation of ``from ... import`` (the ``IMPORT_FROM`` bytecode implementation) checks if the module it's fetching from is a lazy module From e48febbcd437112cb006539bd591b04307d342a1 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Tue, 23 Sep 2025 14:28:48 -0400 Subject: [PATCH 08/39] Add 'Alternate Implementations', 'Rejected Ideas', & a few small nits --- peps/pep-0805.rst | 58 +++++++++++++++++++++++++++++++++++++++-------- 1 file changed, 49 insertions(+), 9 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index f57936a47e1..4ff17ab77e6 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -40,14 +40,14 @@ is especially costly for command-line tools with multiple subcommands, where even a simple ``--help`` can load dozens of unnecessary modules. This is particularly bad for big CLIs that grew organically. -Some projects try to delay imports by moving them into functions, but this is -fragile and hard to maintain. Existing tools such as -``importlib.util.LazyLoader`` or third-party packages provide partial solutions -but either fail to cover all cases or add runtime overhead. Scientific Python -libraries have adopted similar patterns, formalized in `SPEC 1 +Some projects try to delay imports by moving them into functions, but this practice +is fragile, hard to maintain, and obfuscates the full set of dependencies of a module. +Existing tools such as ``importlib.util.LazyLoader`` and existing third-party packages provide +partial solutions but either fail to cover all cases or add runtime overhead. Scientific +Python libraries have adopted similar patterns, formalized in `SPEC 1 `_ and the ``lazy_loader`` -package, and typing imports are another common case where eager loading is -wasteful. +package. In addition, typing imports are another common case where unneeded imports are +wastefully eagerly loaded. This proposal introduces lazy imports with a design that is local, explicit, controlled, and granular. Each of these qualities is essential to making the @@ -285,12 +285,52 @@ Reference Implementation [Link to any existing implementation and details about its state, e.g. proof-of-concept.] +Alternate Implementation +======================== + +Leveraging a Subclass of Dict +----------------------------- + +Instead of updating the internal dict object to directly add the fields needed to support lazy imports, +we could create a subclass of the dict object to be used specifically for Lazy Import enablement. + +Alternate Keyword Names +----------------------- + +We also considered a variety of other keywords to support explicit lazy imports. The most compelling +alternate options were ``defer`` and ``delay``. + Rejected Ideas ============== -[Why certain ideas that were brought while discussing this PEP were not ultimately pursued.] - +Modification of the Dict Object +------------------------------- + +The initial PEP for lazy imports (PEP 690) relied heavily on the modification of the internal dict +object to support lazy imports. We recognize that this data structure is highly tuned, heavily used +across the codebase, and very performance sensitive. Because of the importance of this data structure +and the desire to keep the implementation of lazy imports encapsulated from users who may have no +interest in the feature, we’ve decided to invest in an alternate approach. + +Support of Import Cycles +------------------------ + +The initial PEP for lazy imports (PEP 690) contained support for eliminating most import cycles, +but this meant that in some cases a codebase would no longer run successfully after disabling +lazy imports. For the explicit lazy imports PEP we’ve decided not to add support for this use case. +While codebases with heavily nested dependencies may benefit from this feature, supporting import +cycles runs the risk of creating a dependency on lazy imports. As the intention of this PEP is to +give users the power to opt in or opt out of lazy imports based on the specific needs within their +codebases, we’ve decided not to implement this feature and instead prioritize backwards compatibility. + +Placing the ``lazy`` Keyword at the End of Import Statements +------------------------------------------------------------ + +We discussed appending lazy to the end of import statements like such ``import foo lazy`` or +``from foo import bar, baz lazy`` but ultimately decided that this approach provided less clarity. +For example, if multiple modules are imported in a single statement, it is unclear if the lazy binding +applies to all of the imported objects or just a subset of the items. Open Issues =========== From efffe73e6f36ecb2df6fe6781c68c4fa867dafd7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20M=C3=A9ndez=20Bravo?= Date: Tue, 23 Sep 2025 11:34:31 -0700 Subject: [PATCH 09/39] No known security vulnerabilities --- peps/pep-0805.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 4ff17ab77e6..557be460abd 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -267,8 +267,7 @@ Backwards Compatibility Security Implications ===================== -[How could a malicious user take advantage of this new feature?] - +There are no known security vulnerabilities introduced by lazy imports. How to Teach This ================= From 7207ec95ed5d5c1b106321e882f380e48f54efd7 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Germ=C3=A1n=20M=C3=A9ndez=20Bravo?= Date: Tue, 23 Sep 2025 11:43:23 -0700 Subject: [PATCH 10/39] Backwards Compatibility --- peps/pep-0805.rst | 49 ++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 48 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 557be460abd..5b48d677602 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -261,7 +261,54 @@ resolved explicitly by calling the ``get`` method. Backwards Compatibility ======================= -[Describe potential impact and severity on pre-existing code.] +Lazy imports are **opt-in**. Existing programs continue to run unchanged unless +a project explicitly enables laziness (via ``lazy`` syntax, ``__lazy_modules__``, +or an interpreter-wide switch). + +Unchanged semantics +------------------- + +* Regular ``import`` and ``from ... import ...`` remain eager unless explicitly + made *potentially lazy* by the mechanisms below. +* Dynamic import APIs remain eager and unchanged: ``__import__()`` and + ``importlib.import_module()``. +* Import hooks and loaders continue to run under the standard import protocol + when a lazy binding is first used. + +Observable behavioral shifts (opt-in only) +------------------------------------------ + +These changes are limited to bindings explicitly made lazy: + +* **Error timing.** Exceptions that would have occurred during an eager import + (for example ``ImportError`` or ``AttributeError`` for a missing member) now + occur at the first *use* of the lazy name. +* **Side-effect timing.** Import-time side effects in lazily imported modules + occur at first use of the binding, not at module import time. +* **Presence in ``sys.modules``.** A lazily imported module may not appear in + ``sys.modules`` until first use. After reification it must appear in + ``sys.modules``. If some other code eagerly imports the same module before + first use, the lazy binding resolves to that existing module object. +* **Proxy visibility.** Before first use, the bound name refers to a lazy proxy. + Indirect introspection that touches the value may observe a proxy lazy object + representation. After first use, the name is rebound to the real object and + becomes indistinguishable from an eager import. + +Thread-safety and reification +----------------------------- + +First use of a lazy binding follows the existing import-lock discipline. Exactly +one thread performs the import and **atomically rebinds** the importing module's +global to the resolved object. Concurrent readers thereafter observe the real +object. + +Typing and tools +---------------- + +Type checkers and static analyzers may treat ``lazy`` imports as ordinary +imports for name resolution. At runtime, annotation-only imports can be marked +``lazy`` to avoid startup overhead. IDEs and debuggers should be prepared to +display lazy proxies before first use and the real objects thereafter. Security Implications From 0064549bee39c30d45e75e42dcb956d45e067531 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Tue, 23 Sep 2025 17:09:35 -0400 Subject: [PATCH 11/39] Initial pass over "How to Teach This" --- peps/pep-0805.rst | 31 ++++++++++++++++++++++++++++++- 1 file changed, 30 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 5b48d677602..406876728f4 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -319,7 +319,36 @@ There are no known security vulnerabilities introduced by lazy imports. How to Teach This ================= -[How to teach users, new and experienced, how to apply the PEP to their work.] +The new ``lazy`` keyword will be documented as part of the language standard. + +As this feature is opt-in, new Python users should be able to continue using the +language as they are used to. For experienced developers, we expect them to leverage +Lazy Imports for the variety of benefits listed above (decreased duration, decreased +memory usage, etc) on a case by case basis. Developers interested in the performance +of their Python binary will likely leverage profiling to understand the import time +overhead in their codebase and mark the necessary imports as ``lazy``. In addition, +developers can mark imports that will only be used for type annotations as ``lazy`` as +well. + +Below is guidance on how to best take advange of Lazy Imports and how to avoid +incompatibilities: + +* When adopting lazy imports, users should be aware that eliding an import until it is + used will result in side effects not being executed. In turn, users should be wary of + modules that rely on import time side effects. Perhaps the most common reliance on + import side effects is the registry pattern, where population of some external + registry happens implicitly during the importing of modules, often via + decorators but sometimes implemented via metaclasses. Instead, registries of objects + should be constructed via explicit discovery processes. + +* Always import needed submodules explicitly. It is not enough to rely on a different import + to ensure a module has its submodules as attributes. Plainly, unless there is an + explicit ``from . import bar`` in ``foo/__init__.py``, always use ``import + foo.bar; foo.bar.Baz``, not ``import foo; foo.bar.Baz``. The latter only works + (unreliably) because the attribute ``foo.bar`` is added as a side effect of + ``foo.bar`` being imported somewhere else. + +* Avoid using star imports, as those are always eager. FAQ === From 73c7c14f7b3b8e4113ac5d03df26c70ca3688e12 Mon Sep 17 00:00:00 2001 From: Tim Stumbaugh Date: Thu, 25 Sep 2025 15:27:47 -0600 Subject: [PATCH 12/39] Add miscellaneous improvements --- peps/pep-0805.rst | 184 ++++++++++++++++++++++++++-------------------- 1 file changed, 105 insertions(+), 79 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 406876728f4..84e0063f0b5 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -33,39 +33,49 @@ remain unchanged, and lazy imports are enabled only where explicitly requested. Motivation ========== -By convention, Python places imports at module level to avoid repetition and -extra overhead. The drawback is that importing a main module often triggers a +A common convention in Python code is to place all imports at the module +level (often at the top!). This avoids repetition, makes dependencies clear +and minimizes runtime overhead by only evaluating an import statement once +per module. A major drawback with this approach is that importing the first +module for an execution of Python (the "main" module) often triggers an immediate cascade of imports, loading many dependencies that may never be used. The effect is especially costly for command-line tools with multiple subcommands, where -even a simple ``--help`` can load dozens of unnecessary modules. This is -particularly bad for big CLIs that grew organically. - -Some projects try to delay imports by moving them into functions, but this practice -is fragile, hard to maintain, and obfuscates the full set of dependencies of a module. -Existing tools such as ``importlib.util.LazyLoader`` and existing third-party packages provide -partial solutions but either fail to cover all cases or add runtime overhead. Scientific -Python libraries have adopted similar patterns, formalized in `SPEC 1 -`_ and the ``lazy_loader`` -package. In addition, typing imports are another common case where unneeded imports are -wastefully eagerly loaded. - -This proposal introduces lazy imports with a design that is local, explicit, -controlled, and granular. Each of these qualities is essential to making the -feature predictable and safe to use in practice. +even running the command with ``--help`` can load dozens of unnecessary modules and +take several seconds, just to get feedback to the user of how to run the program at +all. Then the user incurs all that overhead again when they figure out the command +they want and invoke the program "for real." + +A somewhat common response is to delay imports by moving them into functions +(inline imports), but this practice is very manual to implement and maintain. +It obfuscates the full set of dependencies of a module. Additionally, there's a +slight penalty to each invocation of the function, as the import and binding must +be re-resolved. +The standard library provides ``importlib.util.LazyLoader`` to solve some of these +problems. It permits imports at the module level to work *mostly* like inline +imports do. Scientific Python libraries have adopted a similar pattern, formalized +in `SPEC 1 `_. There's also the +third-party `lazy_loader `_ package. +Imports used solely for static type checking are another source of potentially unneeded +imports, and there are similarly disparate approaches to minimizing the overhead. + +These approaches don't cover all cases, add runtime overhead in unexpected places and +are non-obvious and non-standardized. This proposal introduces lazy imports with a +design that is local, explicit, controlled, and granular. Each of these qualities +is essential to making the feature predictable and safe to use in practice. The behavior is **local**: laziness applies only to the specific import marked with the ``lazy`` keyword, and it does not cascade recursively into other imports. This ensures that developers can reason about the effect of laziness by looking only at the line of code in front of them, without worrying about whether imported modules will themselves behave differently. A ``lazy import`` -is an isolated decision, not a global shift in semantics. +is an isolated decision in a single module, not a global shift in semantics. -The semantics are also **explicit**. When a name is imported lazily, the binding +The semantics are **explicit**. When a name is imported lazily, the binding is created in the importing module immediately, but the target module is not loaded until the first time the name is accessed. After this point, the binding -is indistinguishable from a normal import. This clarity reduces surprises and -makes the feature accessible to developers who may not be deeply familiar with -Python’s import machinery. +is indistinguishable from one created by a normal import. This clarity reduces +surprises and makes the feature accessible to developers who may not be +deeply familiar with Python’s import machinery. Lazy imports are **controlled**, in the sense that deferred loading is only triggered by the importing code itself. In the general case, a library will only @@ -84,24 +94,26 @@ needs of each project. The design of lazy imports provides several concrete advantages: -* Startup latency is critical for command-line tools, many of which are - short-lived processes. A CLI with multiple subcommands typically imports - every dependency up front, even if the user only requests ``tool --help``. - With lazy imports, only the code paths actually reached trigger module - loading. This can reduce startup time by 50–70% in practice, restoring - Python's competitiveness in domains where fast startup matters most. +* Command-line tools are often invoked directly by a user, so latency — in particular + startup latency — is quite noticeable. These programs are also typically + short-lived processes (contrasted with, e.g., a web server). Most conventions + would have a CLI with multiple subcommands import every dependency up front, + even if the user only requests ``tool --help`` (or ``tool subcommand --help```). + With lazy imports, only the code paths actually reached will import a module. + This can reduce startup time by 50–70% in practice, providing a visceral improvement + to a common user experience and improving Python's competitiveness in domains + where fast startup matters most. * Type annotations frequently require imports that are never used at runtime. - The common workaround is to wrap them in ``if TYPE_CHECKING:`` blocks, which - adds boilerplate and complexity. With lazy imports, annotation-only imports - impose no runtime penalty, eliminating the need for guards and making - annotated codebases cleaner. + The common workaround is to wrap them in ``if TYPE_CHECKING:`` blocks [#f1]_. + With lazy imports, annotation-only imports impose no runtime penalty, eliminating + the need for such guards and making annotated codebases cleaner. -* Large applications often import thousands of modules, each adding objects and - state to memory. In long-lived processes this raises baseline memory usage - and reduces stability under load. Lazy imports defer this cost until a module - is needed, keeping unused subsystems unloaded. Memory savings of 30–40% have - been observed in real workloads. +* Large applications often import thousands of modules, and each module creates + function and type objects, incurring memory costs. In long-lived processes, + this noticeably raises baseline memory usage. Lazy imports defer these costs + until a module is needed, keeping unused subsystems unloaded. Memory savings of + 30–40% have been observed in real workloads. Rationale ========= @@ -148,10 +160,14 @@ Finally, the feature is designed to be adopted incrementally. Nothing changes unless a developer explicitly opts in, and adoption can begin with just a few imports in performance-sensitive areas. This mirrors the experience of gradual typing in Python: a mechanism that can be introduced progressively, without -forcing projects to commit globally from day one. By combining explicit syntax, -a simple runtime model, a compatibility layer, and gradual adoption, this -proposal balances performance improvements with the clarity and stability that -Python users expect. +forcing projects to commit globally from day one. Notably, the adoption can also +be done from the "outside in," permitting CLI authors to introduce lazy imports +and speed up user-facing tools, without requiring changes to every library the +tool might use. + +By combining explicit syntax, a simple runtime model, a compatibility layer, +and gradual adoption, this proposal balances performance improvements with the +clarity and stability that Python users expect. Other design decisions @@ -184,39 +200,40 @@ can appear in front of both forms of import: * ``lazy from spam import ham`` -The soft keyword is only allowed at the global level, so not inside +The soft keyword is only allowed at the global (module) level, **not** inside functions. It's also not allowed in try blocks. Import statements that use -the soft keyword are potentially lazy. +the soft keyword are *potentially lazy*. -If the importing module has a ``__lazy_modules__`` attribute, it contains a -list of names the imports of which are potentially lazy (as if the lazy -keyword was used). +A module may contain a ``__lazy_modules__`` attribute, which is a list of +names of imports to make *potentially lazy* (as if the ``lazy``` keyword was +used). -The normal (non-lazy) import statement also checks the global lazy imports -flag. If it is ``"enabled"``, all imports at the global level that are not in -a try block are potentially lazy. +The normal (non-lazy) import statement will check the global lazy imports +flag. If it is ``"enabled"``, all imports at the global level of all modules — +except statements in a ``try`` block — are *potentially lazy*. -If the global lazy imports flag is set to ``"disabled"``, the import is _not_ -lazy (as if the lazy keyword was not used). +If the global lazy imports flag is set to ``"disabled"``, no *potentially lazy* +import is ever imported lazily, and the behavior is equivalent to a regular +``import`` statement: the import is *eager* (as if the lazy keyword was not used). -For potentially lazy imports, the lazy imports filter (if set) is called, -passing the name of the module doing the import, the module being imported, -and (if applicable) the fromlist. If the lazy import filter returns True the -lazy import continues. Otherwise, the import is _not_ lazy, and the normal -import continues. +For *potentially lazy* imports, the lazy imports filter (if set) is called, +passing the name of the module doing the import, the name of the module being +imported, and (if applicable) the fromlist. If the lazy import filter returns +``True``, the *potentially lazy* import becomes a lazy import. Otherwise, the +import is *not* lazy, and the normal (eager) import continues. When an import is lazy, ``__lazy_import__`` is called instead of ``__import__``. ``__lazy_import__`` will: -* Check if the module already exists in ``sys.modules``, and return that if it is. +* Check if the module already exists in ``sys.modules``, and return it if so. * Add the module to ``sys.lazy_modules``, a set of modules which have been lazily imported. -* Return a "lazy module object". +* Return a "lazy module object." The implementation of ``from ... import`` (the ``IMPORT_FROM`` bytecode implementation) checks if the module it's fetching from is a lazy module -object, and if so, returns lazy objects for each name instead. +object, and if so, returns a lazy object for each name instead. The end result of this process is that lazy imports (regardless of how they are enabled) result in lazy objects being assigned to global variables. @@ -235,26 +252,26 @@ Reification When a lazy object is first used, it needs to be reified. This means resolving the import at that point in the program, and replacing the lazy -object with the concrete one. Reification imports the module the same as it -would have been if it had been imported eagerly, barring changes to the -import system (like changes to ``sys.path``, ``sys.meta_path``, ``sys.path_hooks`` -or ``__import__``). +object with the concrete one. Reification imports the module in the same way +as it would have been if it had been imported eagerly, barring intervening +changes to the import system (e.g. to ``sys.path``, ``sys.meta_path``, +``sys.path_hooks`` or ``__import__``). -Reification calls ``__import__`` to resolve the import. Once the module is -reified it's removed from ``sys.lazy_modules``. +Reification still calls ``__import__`` to resolve the import. Once the module is +reified, it's removed from ``sys.lazy_modules``. -Reification does _not_ automatically occur when a module that was lazily -imported before is eagerly imported. Reification does _not_ resolve all lazy -objects referencing the module. It only resolves the lazy object being -accessed. +Reification does **not** automatically occur when a module that was previously lazily +imported is subsequently eagerly imported. Reification also does **not** immediately +resolve all lazy objects (e.g. ``lazy from`` statements) that referenced the module. +It **only** resolves the lazy object being accessed. Accessing a lazy object (from a global variable or a module attribute) reifies the object. Accessing a module's ``__dict__`` or calling ``globals()`` (and ``locals()`` at the global level, where it means the same as ``globals()``) -reifies _all_ lazy objects in that module. +reifies **all** lazy objects in that module. More indirect ways of accessing arbitrary globals (e.g. inspecting -``frame.f_globals``) does _not_ reify all the objects. A lazy object can be +``frame.f_globals``) do **not** reify all the objects. A lazy object can be resolved explicitly by calling the ``get`` method. @@ -286,7 +303,7 @@ These changes are limited to bindings explicitly made lazy: * **Side-effect timing.** Import-time side effects in lazily imported modules occur at first use of the binding, not at module import time. * **Presence in ``sys.modules``.** A lazily imported module may not appear in - ``sys.modules`` until first use. After reification it must appear in + ``sys.modules`` until first use. After reification, it must appear in ``sys.modules``. If some other code eagerly imports the same module before first use, the lazy binding resolves to that existing module object. * **Proxy visibility.** Before first use, the bound name refers to a lazy proxy. @@ -323,14 +340,13 @@ The new ``lazy`` keyword will be documented as part of the language standard. As this feature is opt-in, new Python users should be able to continue using the language as they are used to. For experienced developers, we expect them to leverage -Lazy Imports for the variety of benefits listed above (decreased duration, decreased +lazy imports for the variety of benefits listed above (decreased latency, decreased memory usage, etc) on a case by case basis. Developers interested in the performance of their Python binary will likely leverage profiling to understand the import time overhead in their codebase and mark the necessary imports as ``lazy``. In addition, -developers can mark imports that will only be used for type annotations as ``lazy`` as -well. +developers can mark imports that will only be used for type annotations as ``lazy``. -Below is guidance on how to best take advange of Lazy Imports and how to avoid +Below is guidance on how to best take advantage of lazy imports and how to avoid incompatibilities: * When adopting lazy imports, users should be aware that eliding an import until it is @@ -338,8 +354,9 @@ incompatibilities: modules that rely on import time side effects. Perhaps the most common reliance on import side effects is the registry pattern, where population of some external registry happens implicitly during the importing of modules, often via - decorators but sometimes implemented via metaclasses. Instead, registries of objects - should be constructed via explicit discovery processes. + decorators but sometimes implemented via metaclasses or ``__init_subclass__``. + Instead, registries of objects should be constructed via explicit discovery + processes (e.g. a well-known function to call). * Always import needed submodules explicitly. It is not enough to rely on a different import to ensure a module has its submodules as attributes. Plainly, unless there is an @@ -348,6 +365,11 @@ incompatibilities: (unreliably) because the attribute ``foo.bar`` is added as a side effect of ``foo.bar`` being imported somewhere else. +* Users who are moving imports into functions to improve startup time, should instead + consider keeping them where they are but adding the ``lazy`` keyword. This allows + them to keep dependencies clear and avoid the overhead of repeatedly re-resolving + the import but will still speed up the program. + * Avoid using star imports, as those are always eager. FAQ @@ -422,7 +444,11 @@ Acknowledgements Footnotes ========= -[A collection of footnotes cited in the PEP, and a place to list non-inline hyperlink targets.] +.. [#f1] Furthermore, there's also external tooling, in the form of +`flake8-type-checking `_, because it is +common for developers to mislocate imports and accidentally introduce a runtime +dependency on an import only imported in such a block. Ironically, the static type +checker is of no help in these circumstances. Copyright From bed9dfa1a951b49b9d092e606fcda42f5df3581f Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Fri, 26 Sep 2025 13:16:43 +0100 Subject: [PATCH 13/39] Small fixes --- peps/pep-0805.rst | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 84e0063f0b5..2e760ceff60 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -34,7 +34,7 @@ Motivation ========== A common convention in Python code is to place all imports at the module -level (often at the top!). This avoids repetition, makes dependencies clear +level, typically at the beginning of the file. This avoids repetition, makes dependencies clear and minimizes runtime overhead by only evaluating an import statement once per module. A major drawback with this approach is that importing the first module for an execution of Python (the "main" module) often triggers an immediate @@ -50,6 +50,7 @@ A somewhat common response is to delay imports by moving them into functions It obfuscates the full set of dependencies of a module. Additionally, there's a slight penalty to each invocation of the function, as the import and binding must be re-resolved. + The standard library provides ``importlib.util.LazyLoader`` to solve some of these problems. It permits imports at the module level to work *mostly* like inline imports do. Scientific Python libraries have adopted a similar pattern, formalized @@ -98,7 +99,7 @@ The design of lazy imports provides several concrete advantages: startup latency — is quite noticeable. These programs are also typically short-lived processes (contrasted with, e.g., a web server). Most conventions would have a CLI with multiple subcommands import every dependency up front, - even if the user only requests ``tool --help`` (or ``tool subcommand --help```). + even if the user only requests ``tool --help`` (or ``tool subcommand --help``). With lazy imports, only the code paths actually reached will import a module. This can reduce startup time by 50–70% in practice, providing a visceral improvement to a common user experience and improving Python's competitiveness in domains @@ -205,7 +206,7 @@ functions. It's also not allowed in try blocks. Import statements that use the soft keyword are *potentially lazy*. A module may contain a ``__lazy_modules__`` attribute, which is a list of -names of imports to make *potentially lazy* (as if the ``lazy``` keyword was +names of imports to make *potentially lazy* (as if the ``lazy`` keyword was used). The normal (non-lazy) import statement will check the global lazy imports @@ -341,7 +342,7 @@ The new ``lazy`` keyword will be documented as part of the language standard. As this feature is opt-in, new Python users should be able to continue using the language as they are used to. For experienced developers, we expect them to leverage lazy imports for the variety of benefits listed above (decreased latency, decreased -memory usage, etc) on a case by case basis. Developers interested in the performance +memory usage, etc) on a case-by-case basis. Developers interested in the performance of their Python binary will likely leverage profiling to understand the import time overhead in their codebase and mark the necessary imports as ``lazy``. In addition, developers can mark imports that will only be used for type annotations as ``lazy``. @@ -445,10 +446,10 @@ Footnotes ========= .. [#f1] Furthermore, there's also external tooling, in the form of -`flake8-type-checking `_, because it is -common for developers to mislocate imports and accidentally introduce a runtime -dependency on an import only imported in such a block. Ironically, the static type -checker is of no help in these circumstances. + `flake8-type-checking `_, because it is + common for developers to mislocate imports and accidentally introduce a runtime + dependency on an import only imported in such a block. Ironically, the static type + checker is of no help in these circumstances. Copyright From 573b2fd5dbd12950816ed628fcbf502de1ca383b Mon Sep 17 00:00:00 2001 From: Pablo Galindo Date: Fri, 26 Sep 2025 14:13:36 +0100 Subject: [PATCH 14/39] Add FAQ --- peps/pep-0805.rst | 136 +++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 135 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 2e760ceff60..0417299ed82 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -376,7 +376,141 @@ incompatibilities: FAQ === -[A list of frequently asked questions with answers] +**Q: How does this differ from the rejected PEP 690?** + +A: PEP 805 takes an explicit, opt-in approach instead of PEP 690's implicit global approach. The key differences are: + +- **Explicit syntax**: ``lazy import foo`` clearly marks which imports are lazy +- **Local scope**: Laziness only affects the specific import statement, not cascading to dependencies +- **Simpler implementation**: Uses proxy objects instead of modifying core dictionary behavior + +**Q: What happens when lazy imports encounter errors?** + +A: Import errors (``ImportError``, ``ModuleNotFoundError``, syntax errors) are +deferred until first use of the lazy name. This is similar to moving an import +into a function. The error will occur with a clear traceback pointing to the +first access of the lazy object. + +**Q: How do lazy imports affect modules with import-time side effects?** + +A: Side effects are deferred until first use. This is generally desirable for performance, but may require code changes for modules that rely on import-time registration patterns. We recommend: + +- Use explicit initialization functions instead of import-time side effects +- Call initialization functions explicitly when needed +- Avoid relying on import order for side effects + +**Q: Can I use lazy imports with** ``from ... import ...`` **statements?** + +A: Yes as long as you don't use ``from ... import *``. Both ``lazy import foo`` +and ``lazy from foo import bar`` are supported. The ``bar`` name will be bound +to a lazy object that resolves to ``foo.bar`` on first use. + +**Q: What about type annotations and** ``TYPE_CHECKING`` **imports?** + +A: Lazy imports eliminate the need for ``TYPE_CHECKING`` guards. You can write: + +.. code-block:: python + + lazy from typing import List, Dict # No runtime cost + +Instead of: + +.. code-block:: python + + from typing import TYPE_CHECKING + if TYPE_CHECKING: + from typing import List, Dict + +**Q: What's the performance overhead of lazy imports?** + +A: The overhead is minimal: + +- Zero overhead after first use thanks to the adaptive interpreter optimizing the slow path away. +- Small one-time cost to create the proxy object. +- Reification (first use) has the same cost as a regular import. +- No ongoing performance penalty unlike ``importlib.util.LazyLoader``. + +**Q: Can I mix lazy and eager imports of the same module?** + +A: Yes. If module ``foo`` is imported both lazily and eagerly in the same +program, the eager import takes precedence and both bindings resolve to the same +module object. + +**Q: How do I migrate existing code to use lazy imports?** + +A: Migration is incremental: + +1. Identify slow-loading modules using profiling tools +2. Add ``lazy`` keyword to imports that aren't needed immediately +3. Test that side-effect timing changes don't break functionality +4. Use ``__lazy_modules__`` for compatibility with older Python versions + +**Q: What about star imports** (``from module import *``)? + +A: Star imports cannot be lazy - they remain eager. This is because the set of +names being imported cannot be determined without loading the module. Using the +``lazy`` keyword with star imports will be a syntax error. + +**Q: How do lazy imports interact with import hooks and custom loaders?** + +A: Import hooks and loaders work normally. When a lazy object is first used, the +standard import protocol runs, including any custom hooks or loaders that were +in place at reification time. + +**Q: What happens in multi-threaded environments?** + +A: Lazy import reification is thread-safe. Only one thread will perform the +actual import, and the binding is atomically updated. Other threads will see +either the lazy proxy or the final resolved object. + +**Q: Can I force reification of a lazy import without using it?** + +A: Yes, calling ``globals()`` or accessing a module's ``__dict__`` will reify +all lazy objects in that module. Individual lazy objects can be resolved by +calling their ``get()`` method. + +**Q: Why not use** ``importlib.util.LazyLoader`` **instead?** + +A: ``LazyLoader`` has significant limitations: + +- Requires verbose setup code for each lazy import +- Has ongoing performance overhead on every attribute access +- Doesn't work well with ``from ... import`` statements +- Less clear and standard than dedicated syntax + +**Q: Will this break tools like** ``isort`` **or** ``black``? + +A: Tools will need updates to recognize the ``lazy`` keyword, but the changes +should be minimal since the import structure remains the same. The keyword +appears at the beginning, making it easy to parse. + +**Q: How do I know if a library is compatible with lazy imports?** + +A: Most libraries should work fine with lazy imports. Libraries that might have issues: + +- Those with essential import-time side effects (registration, monkey-patching) +- Those that expect specific import ordering +- Those that modify global state during import + +When in doubt, test lazy imports with your specific use cases. + +**Q: Can I use lazy imports inside functions?** + +A: No, the ``lazy`` keyword is only allowed at module level. For function-level +lazy loading, use traditional inline imports or move the import to module level +with ``lazy``. + +**Q: What about forwards compatibility with older Python versions?** + +A: Use the ``__lazy_modules__`` global for compatibility: + +..code-block: python + + # Works on Python 3.15+ as lazy, eager on older versions + __lazy_modules__ = ['expensive_module'] + import expensive_module + +This provides a migration path until you can rely on the ``lazy`` keyword. Reference Implementation ======================== From e018473f7fe92bb39e333adcded0c6d6ffe154fb Mon Sep 17 00:00:00 2001 From: Tim Stumbaugh Date: Fri, 26 Sep 2025 10:37:42 -0600 Subject: [PATCH 15/39] Clarify how __lazy_modules__ works from discord discusison --- peps/pep-0805.rst | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 0417299ed82..735ac7a0052 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -207,7 +207,7 @@ the soft keyword are *potentially lazy*. A module may contain a ``__lazy_modules__`` attribute, which is a list of names of imports to make *potentially lazy* (as if the ``lazy`` keyword was -used). +used). This name is checked on each ``import`` statement. The normal (non-lazy) import statement will check the global lazy imports flag. If it is ``"enabled"``, all imports at the global level of all modules — @@ -504,13 +504,16 @@ with ``lazy``. A: Use the ``__lazy_modules__`` global for compatibility: -..code-block: python +.. code-block:: python # Works on Python 3.15+ as lazy, eager on older versions __lazy_modules__ = ['expensive_module'] import expensive_module -This provides a migration path until you can rely on the ``lazy`` keyword. +This provides a migration path until you can rely on the ``lazy`` keyword. For +maximum predictability, it's recommended to define ``__lazy_modules__`` once, +before any imports. But as it is checked on each import, it can be modified between +``import`` statements Reference Implementation ======================== From d66187f797e9c6298750076c3fe778799f074a9c Mon Sep 17 00:00:00 2001 From: Thomas Wouters Date: Mon, 29 Sep 2025 16:25:43 +0200 Subject: [PATCH 16/39] Make it more obvious the local/explicit/controlled/granular checklist applies to the *new syntax*, not the global switch/configuration API. Call out the global switch and configuration API separately. Drop the use of existing modules in sys.modules from the specification, and mention the signature of `__lazy_import__`. Mention import ordering explicitly in visible changes, since it can have knock-on effects. Minor language tweaks to emphasize ordering of effects. --- peps/pep-0805.rst | 34 ++++++++++++++++++++-------------- 1 file changed, 20 insertions(+), 14 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 735ac7a0052..b594722c799 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -59,10 +59,11 @@ third-party `lazy_loader `_ package. Imports used solely for static type checking are another source of potentially unneeded imports, and there are similarly disparate approaches to minimizing the overhead. -These approaches don't cover all cases, add runtime overhead in unexpected places and -are non-obvious and non-standardized. This proposal introduces lazy imports with a -design that is local, explicit, controlled, and granular. Each of these qualities -is essential to making the feature predictable and safe to use in practice. +These approaches don't cover all cases, add runtime overhead in unexpected +places and are non-obvious and non-standardized. This proposal introduces +lazy imports syntax with a design that is local, explicit, controlled, and +granular. Each of these qualities is essential to making the feature +predictable and safe to use in practice. The behavior is **local**: laziness applies only to the specific import marked with the ``lazy`` keyword, and it does not cascade recursively into other @@ -93,6 +94,12 @@ performance-sensitive areas of a codebase. The experience is similar to the way typing was introduced into Python: optional, progressive, and adaptable to the needs of each project. +In addition to the new lazy import syntax, we *also* propose a way to +control lazy imports at the application level: globally disabling or +enabling, and selectively disabling. These are provided for debugging, +testing and experimentation, and are not expected to be the common way to +control lazy imports. + The design of lazy imports provides several concrete advantages: * Command-line tools are often invoked directly by a user, so latency — in particular @@ -223,14 +230,10 @@ imported, and (if applicable) the fromlist. If the lazy import filter returns ``True``, the *potentially lazy* import becomes a lazy import. Otherwise, the import is *not* lazy, and the normal (eager) import continues. -When an import is lazy, ``__lazy_import__`` is called instead of ``__import__``. -``__lazy_import__`` will: - -* Check if the module already exists in ``sys.modules``, and return it if so. - -* Add the module to ``sys.lazy_modules``, a set of modules which have been lazily imported. - -* Return a "lazy module object." +When an import is lazy, ``__lazy_import__`` is called instead of +``__import__``. ``__lazy_import__`` has the same function signature as +``__import__``. It adds the module to ``sys.lazy_modules``, a set of modules +which have been lazily imported, and returns a "lazy module object". The implementation of ``from ... import`` (the ``IMPORT_FROM`` bytecode implementation) checks if the module it's fetching from is a lazy module @@ -287,7 +290,7 @@ Unchanged semantics ------------------- * Regular ``import`` and ``from ... import ...`` remain eager unless explicitly - made *potentially lazy* by the mechanisms below. + made *potentially lazy* by the local or global mechanisms provided. * Dynamic import APIs remain eager and unchanged: ``__import__()`` and ``importlib.import_module()``. * Import hooks and loaders continue to run under the standard import protocol @@ -303,10 +306,13 @@ These changes are limited to bindings explicitly made lazy: occur at the first *use* of the lazy name. * **Side-effect timing.** Import-time side effects in lazily imported modules occur at first use of the binding, not at module import time. +* **Import order.** Because modules are imported on first use, the order in + which modules are imported may change. * **Presence in ``sys.modules``.** A lazily imported module may not appear in ``sys.modules`` until first use. After reification, it must appear in ``sys.modules``. If some other code eagerly imports the same module before - first use, the lazy binding resolves to that existing module object. + first use, the lazy binding resolves to that existing module object when + it is first used. * **Proxy visibility.** Before first use, the bound name refers to a lazy proxy. Indirect introspection that touches the value may observe a proxy lazy object representation. After first use, the name is rebound to the real object and From 3b40056f2db71bb5ba936514a25ea16bd5d03f11 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Mon, 29 Sep 2025 17:47:29 -0400 Subject: [PATCH 17/39] Tiny nits & fixes --- peps/pep-0805.rst | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index b594722c799..8cae3a8c95e 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -22,7 +22,7 @@ This PEP introduces lazy imports as an explicit language feature. Lazy imports defer the loading and execution of a module until the first time the imported name is used, rather than at the point of the import statement. -By allowing developers to mark imports as lazy with new syntax, Python programs +By allowing developers to mark individual imports as lazy with explicit syntax, Python programs can reduce startup time, memory usage, and unnecessary work. This is particularly beneficial for command-line tools, test suites, and applications with large dependency graphs. @@ -60,7 +60,7 @@ Imports used solely for static type checking are another source of potentially u imports, and there are similarly disparate approaches to minimizing the overhead. These approaches don't cover all cases, add runtime overhead in unexpected -places and are non-obvious and non-standardized. This proposal introduces +places, are non-obvious, and non-standardized. This proposal introduces lazy imports syntax with a design that is local, explicit, controlled, and granular. Each of these qualities is essential to making the feature predictable and safe to use in practice. From d3989dba515dfdf3f20e5299ce6d81170bf56ef5 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Tue, 30 Sep 2025 14:22:39 -0400 Subject: [PATCH 18/39] 1/N Addressing feedback 1, 2, & 8 also update my email --- peps/pep-0805.rst | 16 +++++++--------- 1 file changed, 7 insertions(+), 9 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 8cae3a8c95e..cecd509e584 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -4,7 +4,7 @@ Author: Pablo Galindo , Germán Méndez Bravo , Thomas Wouters , Dino Viehland , - Brittany Reynoso , + Brittany Reynoso , Noah Kim , Tim Stumbaugh Discussions-To: Pending @@ -47,9 +47,7 @@ they want and invoke the program "for real." A somewhat common response is to delay imports by moving them into functions (inline imports), but this practice is very manual to implement and maintain. -It obfuscates the full set of dependencies of a module. Additionally, there's a -slight penalty to each invocation of the function, as the import and binding must -be re-resolved. +Additionally, it obfuscates the full set of dependencies for a module. The standard library provides ``importlib.util.LazyLoader`` to solve some of these problems. It permits imports at the module level to work *mostly* like inline @@ -90,9 +88,9 @@ applied. The mechanism is also **granular**. It is introduced through explicit syntax on individual imports, rather than a global flag or implicit setting. This allows developers to adopt it incrementally, starting with the most -performance-sensitive areas of a codebase. The experience is similar to the way -typing was introduced into Python: optional, progressive, and adaptable to the -needs of each project. +performance-sensitive areas of a codebase. As this feature is introduced to the +community, we want to make the experience of onboarding optional, progressive, and +adaptable to the needs of each project. In addition to the new lazy import syntax, we *also* propose a way to control lazy imports at the application level: globally disabling or @@ -538,8 +536,8 @@ we could create a subclass of the dict object to be used specifically for Lazy I Alternate Keyword Names ----------------------- -We also considered a variety of other keywords to support explicit lazy imports. The most compelling -alternate options were ``defer`` and ``delay``. +For this PEP, we decided to propose ``lazy`` for the explicit keyword. We also considered a variety of other +options to support explicit lazy imports. The most compelling alternates were ``defer`` and ``delay``. Rejected Ideas From 7d1e0e4ec4be07ad1a67e8868aa8d9c017e20813 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Tue, 30 Sep 2025 15:20:39 -0400 Subject: [PATCH 19/39] Address #6 & 'G' follow up item --- peps/pep-0805.rst | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index cecd509e584..d2139f49598 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -208,7 +208,9 @@ can appear in front of both forms of import: The soft keyword is only allowed at the global (module) level, **not** inside functions. It's also not allowed in try blocks. Import statements that use -the soft keyword are *potentially lazy*. +the soft keyword are *potentially lazy*. In addition, star imports will not +be supported for lazy import semantics by the soft keyword and will instead +raise a syntax error. A module may contain a ``__lazy_modules__`` attribute, which is a list of names of imports to make *potentially lazy* (as if the ``lazy`` keyword was @@ -563,6 +565,12 @@ cycles runs the risk of creating a dependency on lazy imports. As the intention give users the power to opt in or opt out of lazy imports based on the specific needs within their codebases, we’ve decided not to implement this feature and instead prioritize backwards compatibility. +Placing the ``lazy`` Keyword in the Middle of From Imports +---------------------------------------------------------- + +While we found, ``from foo lazy import bar`` to be a really intuitive placement for the new explicit syntax, +we quickly learned that placing the ``lazy`` keyword here is already syntactically allowed in Python. + Placing the ``lazy`` Keyword at the End of Import Statements ------------------------------------------------------------ From c06c43811c35688cc84ca9d43bf56459d3af1a77 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Tue, 30 Sep 2025 15:52:56 -0400 Subject: [PATCH 20/39] [3/N] Address feedback no. 3 --- peps/pep-0805.rst | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index d2139f49598..3ad2cc054be 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -521,6 +521,21 @@ maximum predictability, it's recommended to define ``__lazy_modules__`` once, before any imports. But as it is checked on each import, it can be modified between ``import`` statements +**Q: How do explicit lazy imports interact with PEP-649/PEP-749** + +A: If an annotation is not stringified, then it is a reference to the type, but it +will only be resolved if the annotation is accessed. In the example below, the ``fake_typing`` module +is only loaded when the user inspects the ``__annotations__`` dictionary. The ``fake_typing`` +module would also be loaded if the user had leveraged ``inspect.get_annotations()`` or `getattr` to access +the annotations. + +.. code-block:: python + + lazy from fake_typing import MyFakeType + def foo(x: MyFakeType): + pass + print(foo.__annotations__) # Triggers loading the fake_typing module + Reference Implementation ======================== From 4a72eb81a9873a71307162feaea574a5a7a3f202 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Tue, 30 Sep 2025 16:01:15 -0400 Subject: [PATCH 21/39] Fix missing back ticks --- peps/pep-0805.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 3ad2cc054be..169a9232f56 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -526,7 +526,7 @@ before any imports. But as it is checked on each import, it can be modified betw A: If an annotation is not stringified, then it is a reference to the type, but it will only be resolved if the annotation is accessed. In the example below, the ``fake_typing`` module is only loaded when the user inspects the ``__annotations__`` dictionary. The ``fake_typing`` -module would also be loaded if the user had leveraged ``inspect.get_annotations()`` or `getattr` to access +module would also be loaded if the user had leveraged ``inspect.get_annotations()`` or ``getattr`` to access the annotations. .. code-block:: python From 504c2fc16e481d596ce0fd4704870164a185ae52 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Tue, 30 Sep 2025 16:04:57 -0400 Subject: [PATCH 22/39] Make answer for no 8 clearer --- peps/pep-0805.rst | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 169a9232f56..af228e91e00 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -553,7 +553,8 @@ we could create a subclass of the dict object to be used specifically for Lazy I Alternate Keyword Names ----------------------- -For this PEP, we decided to propose ``lazy`` for the explicit keyword. We also considered a variety of other +For this PEP, we decided to propose ``lazy`` for the explicit keyword as it felt the most familar to those +already focused on optimizing import overhead. We also considered a variety of other options to support explicit lazy imports. The most compelling alternates were ``defer`` and ``delay``. From 32608e45f71ec070b58e1c343a075801fa4dcbad Mon Sep 17 00:00:00 2001 From: Dino Viehland Date: Wed, 1 Oct 2025 08:41:11 -0700 Subject: [PATCH 23/39] globals() doesn't reify --- peps/pep-0805.rst | 26 ++++++++++++++++++-------- 1 file changed, 18 insertions(+), 8 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 8cae3a8c95e..f547b58d1bd 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -245,8 +245,8 @@ are enabled) result in lazy objects being assigned to global variables. Lazy module objects do not appear in ``sys.modules``, just the ``sys.lazy_modules`` set. Lazy objects should only end up stored in global variables, and the common ways to access those variables (regular variable -access, module attributes, ``globals()``) will resolve lazy imports ("reify") -and replace them when they're accessed. +access, module attributes) will resolve lazy imports ("reify") and replace +them when they're accessed. It is still possible to expose lazy objects through other means, like debuggers. This is not considered a problem. @@ -270,9 +270,8 @@ resolve all lazy objects (e.g. ``lazy from`` statements) that referenced the mod It **only** resolves the lazy object being accessed. Accessing a lazy object (from a global variable or a module attribute) -reifies the object. Accessing a module's ``__dict__`` or calling ``globals()`` -(and ``locals()`` at the global level, where it means the same as ``globals()``) -reifies **all** lazy objects in that module. +reifies the object. Accessing a module's ``__dict__`` reifies **all** lazy objects +in that module. More indirect ways of accessing arbitrary globals (e.g. inspecting ``frame.f_globals``) do **not** reify all the objects. A lazy object can be @@ -471,9 +470,8 @@ either the lazy proxy or the final resolved object. **Q: Can I force reification of a lazy import without using it?** -A: Yes, calling ``globals()`` or accessing a module's ``__dict__`` will reify -all lazy objects in that module. Individual lazy objects can be resolved by -calling their ``get()`` method. +A: Yes, accessing a module's ``__dict__`` will reify all lazy objects in that +module. Individual lazy objects can be resolved by calling their ``get()`` method. **Q: Why not use** ``importlib.util.LazyLoader`` **instead?** @@ -573,6 +571,18 @@ We discussed appending lazy to the end of import statements like such ``import f For example, if multiple modules are imported in a single statement, it is unclear if the lazy binding applies to all of the imported objects or just a subset of the items. +Reifying lazy imports with ``globals()`` is called. +--------------------------------------------------- +While ``globals()`` is also a common way to access variables which may contain lazy imports we will +not reify the lazy imports when it is called. There are a couple of problems with doing this. One +reason is that it is impossible to safely do this on-demand: we cannot return a proxy object as this +will break other usages of globals (e.g. passing them back to ``exec``). + +Therefore it would require reifying all of the objects on the access and this behavior may be +surprising. There may also be a performance cost as it is impractical to cache whether this scan has +been done with just the globals dictionary in hand (versus when accessing from the module where we +can cache the results in the module). + Open Issues =========== From a2da77a71ce0900e10bb6b3260b4e6508e6a7b95 Mon Sep 17 00:00:00 2001 From: Brittany Reynoso Date: Wed, 1 Oct 2025 14:53:58 -0400 Subject: [PATCH 24/39] Add PG feedback --- peps/pep-0805.rst | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index af228e91e00..5257413d71a 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -133,9 +133,8 @@ It is also worth noting that while this PEP outlines one specific approach, we list alternate implementation strategies for some of the core aspects and semantics of the proposal. If the community expresses a strong preference for a different technical path that still preserves the same core semantics or there -is fundamental disagreement over the specific option, the Steering Council can -easily choose to accept the proposal with what they think is the best set of -options for the language. +is fundamental disagreement over the specific option, we have included the +brainstorming we have already completed in preparation for this proposal as reference. The choice to introduce a new ``lazy`` keyword reflects the need for explicit syntax. Import behavior is too fundamental to be left implicit or hidden behind @@ -417,7 +416,7 @@ A: Lazy imports eliminate the need for ``TYPE_CHECKING`` guards. You can write: .. code-block:: python - lazy from typing import List, Dict # No runtime cost + lazy from collections.abc import Sequence, Mapping # No runtime cost Instead of: @@ -425,7 +424,7 @@ Instead of: from typing import TYPE_CHECKING if TYPE_CHECKING: - from typing import List, Dict + from collections.abc import Sequence, Mapping **Q: What's the performance overhead of lazy imports?** @@ -513,8 +512,9 @@ A: Use the ``__lazy_modules__`` global for compatibility: .. code-block:: python # Works on Python 3.15+ as lazy, eager on older versions - __lazy_modules__ = ['expensive_module'] + __lazy_modules__ = ['expensive_module', 'expensive_module_2'] import expensive_module + from expensive_module_2 import MyClass This provides a migration path until you can rely on the ``lazy`` keyword. For maximum predictability, it's recommended to define ``__lazy_modules__`` once, From 928c511406526eb11f90d264b6e543ebf7544350 Mon Sep 17 00:00:00 2001 From: Dino Viehland Date: Wed, 1 Oct 2025 13:00:51 -0700 Subject: [PATCH 25/39] Expand on lazy position and subclass of dict --- peps/pep-0805.rst | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 6da585313c4..662e64d7f71 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -546,7 +546,9 @@ Leveraging a Subclass of Dict ----------------------------- Instead of updating the internal dict object to directly add the fields needed to support lazy imports, -we could create a subclass of the dict object to be used specifically for Lazy Import enablement. +we could create a subclass of the dict object to be used specifically for Lazy Import enablement. This +would still be a leaky abstraction though - methods can be called directly such as ``dict.__getitem__`` +and it would impact the performance of globals lookup in the interpreter. Alternate Keyword Names ----------------------- @@ -583,7 +585,8 @@ Placing the ``lazy`` Keyword in the Middle of From Imports ---------------------------------------------------------- While we found, ``from foo lazy import bar`` to be a really intuitive placement for the new explicit syntax, -we quickly learned that placing the ``lazy`` keyword here is already syntactically allowed in Python. +we quickly learned that placing the ``lazy`` keyword here is already syntactically allowed in Python. This +is because ``from foo. lazy import bar`` is legal syntax. Placing the ``lazy`` Keyword at the End of Import Statements ------------------------------------------------------------ From 42bd36452710bdb33b179f841fc6d0c04f808540 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Thu, 2 Oct 2025 01:56:59 +0100 Subject: [PATCH 26/39] Mega overhaul --- peps/pep-0805.rst | 778 ++++++++++++++++++++++++++++++++++++++++++---- 1 file changed, 720 insertions(+), 58 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 662e64d7f71..0f49aeb1a17 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -147,19 +147,22 @@ Another important decision is to represent lazy imports with proxy objects in the module's namespace, rather than by modifying dictionary lookup. Earlier approaches experimented with embedding laziness into dictionaries, but this blurred abstractions and risked affecting unrelated parts of the runtime. The -proxy approach is simpler: it behaves like a placeholder until first use, at -which point it resolves the import and rebinds the name. From then on, the -binding is indistinguishable from a normal import. This makes the mechanism -easy to explain and keeps the rest of the interpreter unchanged. +dictionary is a fundamental data structure in Python—literally every object is +built on top of dicts—and adding hooks to dictionaries would prevent critical +optimizations and complicate the entire runtime. The proxy approach is simpler: +it behaves like a placeholder until first use, at which point it resolves the +import and rebinds the name. From then on, the binding is indistinguishable +from a normal import. This makes the mechanism easy to explain and keeps the +rest of the interpreter unchanged. Compatibility for library authors was also a key concern. Many maintainers need a migration path that allows them to support both new and old versions of Python at once. For this reason, the proposal includes the ``__lazy_modules__`` global as a transitional mechanism. A module can declare which imports should -be treated as lazy, and on Python 3.15 or later those imports will become lazy -automatically. On earlier versions the declaration is ignored, leaving imports -eager. This gives authors a practical bridge until they can rely on the keyword -as the canonical syntax. +be treated as lazy (by listing the module names as strings), and on Python 3.15 +or later those imports will become lazy automatically. On earlier versions the +declaration is ignored, leaving imports eager. This gives authors a practical +bridge until they can rely on the keyword as the canonical syntax. Finally, the feature is designed to be adopted incrementally. Nothing changes unless a developer explicitly opts in, and adoption can begin with just a few @@ -198,41 +201,107 @@ Other design decisions Specification ============= -A new soft keyword is added, ``lazy`` (other names have been suggested), which -can appear in front of both forms of import: +Grammar +------- -* ``lazy import spam`` +A new soft keyword ``lazy`` is added. A soft keyword is a context-sensitive keyword +that only has special meaning in specific grammatical contexts; elsewhere it can be +used as a regular identifier (e.g., as a variable name). The ``lazy`` keyword only +has special meaning when it appears before import statements: -* ``lazy from spam import ham`` +.. code-block:: text + + import_name: + | 'import' dotted_as_names + | 'lazy' 'import' dotted_as_names + + import_from: + | 'lazy'? 'from' ('.' | '...')* dotted_name 'import' import_from_targets + | 'lazy'? 'from' ('.' | '...')+ 'import' import_from_targets + +Syntax restrictions +~~~~~~~~~~~~~~~~~~~ The soft keyword is only allowed at the global (module) level, **not** inside -functions. It's also not allowed in try blocks. Import statements that use -the soft keyword are *potentially lazy*. In addition, star imports will not -be supported for lazy import semantics by the soft keyword and will instead -raise a syntax error. +functions, class bodies, or try blocks. Import statements that use the soft keyword +are *potentially lazy*. In addition, star imports will not be supported for lazy +import semantics by the soft keyword and will instead raise a syntax error. -A module may contain a ``__lazy_modules__`` attribute, which is a list of -names of imports to make *potentially lazy* (as if the ``lazy`` keyword was -used). This name is checked on each ``import`` statement. +Specifically: -The normal (non-lazy) import statement will check the global lazy imports -flag. If it is ``"enabled"``, all imports at the global level of all modules — -except statements in a ``try`` block — are *potentially lazy*. +1. **Module scope only**: Lazy imports are only allowed at module/global scope. + Using ``lazy`` inside a function, class body, or any nested scope raises a + ``SyntaxError``. -If the global lazy imports flag is set to ``"disabled"``, no *potentially lazy* -import is ever imported lazily, and the behavior is equivalent to a regular -``import`` statement: the import is *eager* (as if the lazy keyword was not used). +2. **Not in try blocks**: Lazy imports are not allowed inside ``try``/``except`` + blocks. This raises a ``SyntaxError``. -For *potentially lazy* imports, the lazy imports filter (if set) is called, -passing the name of the module doing the import, the name of the module being -imported, and (if applicable) the fromlist. If the lazy import filter returns -``True``, the *potentially lazy* import becomes a lazy import. Otherwise, the -import is *not* lazy, and the normal (eager) import continues. +3. **No star imports**: ``lazy from module import *`` raises a ``SyntaxError``. + Star imports are always eager, even when the global lazy imports flag is enabled. + +Examples of syntax errors: + +.. code-block:: python + + # SyntaxError: lazy import not allowed inside functions + def foo(): + lazy import json + + # SyntaxError: lazy import not allowed inside classes + class Bar: + lazy import json + + # SyntaxError: lazy import not allowed inside try/except blocks + try: + lazy import json + except ImportError: + pass + + # SyntaxError: lazy from ... import * is not allowed + lazy from json import * + +Semantics +--------- + +When the ``lazy`` keyword is used, the import becomes *potentially lazy*. The module +is not loaded immediately at the import statement; instead, a lazy proxy object is +created and bound to the name. The actual module is loaded on first use of that name. + +Example: + +.. code-block:: python + + import sys + + lazy import json + + print('json' in sys.modules) # False - module not loaded yet + + # First use triggers loading + result = json.dumps({"hello": "world"}) + + print('json' in sys.modules) # True - now loaded + +A module may contain a ``__lazy_modules__`` attribute, which is a list of +module names (strings) to make *potentially lazy* (as if the ``lazy`` keyword was +used). This attribute is checked on each ``import`` statement to determine whether +the import should be treated as lazy. + +The ``__lazy_modules__`` attribute provides a compatibility mechanism for libraries +that need to support both Python 3.15+ (with native lazy import support) and older +versions. When ``__lazy_modules__`` is defined, the specified imports will be lazy +on Python 3.15+, but will fall back to eager imports on older Python versions that +don't recognize the mechanism. The attribute should be defined at module level before +the import statements it affects, though it is technically evaluated on each import. + +Lazy import mechanism +--------------------- When an import is lazy, ``__lazy_import__`` is called instead of ``__import__``. ``__lazy_import__`` has the same function signature as -``__import__``. It adds the module to ``sys.lazy_modules``, a set of modules -which have been lazily imported, and returns a "lazy module object". +``__import__``. It adds the module name to ``sys.lazy_modules``, a set of +module names which have been lazily imported at some point (primarily for +diagnostics and introspection), and returns a "lazy module object". The implementation of ``from ... import`` (the ``IMPORT_FROM`` bytecode implementation) checks if the module it's fetching from is a lazy module @@ -263,6 +332,44 @@ changes to the import system (e.g. to ``sys.path``, ``sys.meta_path``, Reification still calls ``__import__`` to resolve the import. Once the module is reified, it's removed from ``sys.lazy_modules``. +If reification fails (e.g., due to an ``ImportError``), the exception is enhanced +with chaining to show both where the lazy import was defined and where it was first +accessed. This provides clear debugging information: + +.. code-block:: python + + # app.py - has a typo in the import + lazy from json import dumsp # Typo: should be 'dumps' + + print("App started successfully") + print("Processing data...") + + # Error occurs here on first use + result = dumsp({"key": "value"}) + +The traceback shows both locations: + +.. code-block:: text + + App started successfully + Processing data... + Traceback (most recent call last): + File "app.py", line 2, in + lazy from json import dumsp + ImportError: deferred import of 'json.dumsp' raised an exception during resolution + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "app.py", line 8, in + result = dumsp({"key": "value"}) + ^^^^^ + ImportError: cannot import name 'dumsp' from 'json'. Did you mean: 'dump'? + +This exception chaining clearly shows: (1) where the lazy import was defined, +(2) that it was deferred, and (3) where the actual access happened that triggered +the error. + Reification does **not** automatically occur when a module that was previously lazily imported is subsequently eagerly imported. Reification also does **not** immediately resolve all lazy objects (e.g. ``lazy from`` statements) that referenced the module. @@ -270,12 +377,94 @@ It **only** resolves the lazy object being accessed. Accessing a lazy object (from a global variable or a module attribute) reifies the object. Accessing a module's ``__dict__`` reifies **all** lazy objects -in that module. +in that module. Operations that indirectly access ``__dict__`` (such as ``dir()``) +also trigger this behavior. However, calling ``globals()`` does **not** trigger +reification — it returns the module's dictionary, and accessing lazy objects through +that dictionary still returns lazy proxy objects that need to be reified upon use. More indirect ways of accessing arbitrary globals (e.g. inspecting ``frame.f_globals``) do **not** reify all the objects. A lazy object can be resolved explicitly by calling the ``get`` method. +Bytecode and adaptive specialization +------------------------------------- + +Lazy imports are implemented through modifications to four bytecode instructions: +``IMPORT_NAME``, ``IMPORT_FROM``, ``LOAD_GLOBAL``, and ``LOAD_NAME``. + +The ``lazy`` syntax sets a flag in the ``IMPORT_NAME`` instruction's oparg +(``oparg & 0x01``). The interpreter checks this flag and calls +``_PyEval_LazyImportName()`` instead of ``_PyEval_ImportName()``, creating a lazy +import object rather than executing the import immediately. The ``IMPORT_FROM`` +instruction checks whether its source is a lazy import (``PyLazyImport_CheckExact()``) +and creates a lazy import for the attribute rather than accessing it immediately. + +When a lazy import is accessed, it must be reified. The ``LOAD_GLOBAL`` instruction +(used in function scopes) and ``LOAD_NAME`` instruction (used at module level) both +check whether the object being loaded is a lazy import. If so, they call +``_PyImport_LoadLazyImportTstate()`` to perform the actual import and store the +module in ``sys.modules``. + +This check incurs a small cost on each access. However, Python's adaptive interpreter +can specialize ``LOAD_GLOBAL`` after observing that a lazy import has been reified. +After several executions, ``LOAD_GLOBAL`` becomes ``LOAD_GLOBAL_MODULE``, which +accesses the module dictionary directly without checking for lazy imports. + +Examples of the bytecode generated: + +.. code-block:: python + + lazy import json # IMPORT_NAME with flag set + +Generates: + +.. code-block:: text + + IMPORT_NAME 1 (json + lazy) + +.. code-block:: python + + lazy from json import dumps # IMPORT_NAME + IMPORT_FROM + +Generates: + +.. code-block:: text + + IMPORT_NAME 1 (json + lazy) + IMPORT_FROM 1 (dumps) + +.. code-block:: python + + lazy import json + x = json # Module-level access + +Generates: + +.. code-block:: text + + LOAD_NAME 0 (json) + +.. code-block:: python + + lazy import json + + def use_json(): + return json.dumps({}) # Function scope + +Before any calls: + +.. code-block:: text + + LOAD_GLOBAL 0 (json) + LOAD_ATTR 2 (dumps) + +After several calls, ``LOAD_GLOBAL`` specializes to ``LOAD_GLOBAL_MODULE``: + +.. code-block:: text + + LOAD_GLOBAL_MODULE 0 (json) + LOAD_ATTR_MODULE 2 (dumps) + Backwards Compatibility ======================= @@ -302,6 +491,17 @@ These changes are limited to bindings explicitly made lazy: * **Error timing.** Exceptions that would have occurred during an eager import (for example ``ImportError`` or ``AttributeError`` for a missing member) now occur at the first *use* of the lazy name. + + .. code-block:: python + + # With eager import - error at import statement + import broken_module # ImportError raised here + + # With lazy import - error deferred + lazy import broken_module + print("Import succeeded") + broken_module.foo() # ImportError raised here on first use + * **Side-effect timing.** Import-time side effects in lazily imported modules occur at first use of the binding, not at module import time. * **Import order.** Because modules are imported on first use, the order in @@ -324,6 +524,16 @@ one thread performs the import and **atomically rebinds** the importing module's global to the resolved object. Concurrent readers thereafter observe the real object. +Lazy imports are thread-safe and have no special considerations for free-threading. +A module that would normally be imported in the main thread may be imported in a +different thread if that thread triggers the first access to the lazy import. This +is not a problem—the import lock ensures thread safety regardless of which thread +performs the import. + +Subinterpreters are supported. Each subinterpreter maintains its own +``sys.lazy_modules`` and import state, so lazy imports in one subinterpreter do +not affect others. + Typing and tools ---------------- @@ -332,6 +542,98 @@ imports for name resolution. At runtime, annotation-only imports can be marked ``lazy`` to avoid startup overhead. IDEs and debuggers should be prepared to display lazy proxies before first use and the real objects thereafter. +Lazy imports filter +------------------- + +For *potentially lazy* imports, the lazy imports filter (if set) is called, +passing the name of the module doing the import, the name of the module being +imported, and (if applicable) the fromlist. If the lazy import filter returns +``True``, the *potentially lazy* import becomes a lazy import. Otherwise, the +import is *not* lazy, and the normal (eager) import continues. + +This PEP adds two new functions to the ``sys`` module to manage the filter: + +* ``sys.set_lazy_imports_filter(func)`` - Sets the filter function. The ``func`` + parameter must have the signature: ``func(importer: str, name: str, fromlist: tuple[str, ...] | None) -> bool`` +* ``sys.get_lazy_imports_filter()`` - Returns the currently installed filter function, + or ``None`` if no filter is set. + +This allows for fine-grained control over which imports should be lazy, useful +for excluding modules with known side-effect dependencies or registration patterns. + +The filter mechanism serves as a foundation that tools, debuggers, linters, and +other ecosystem utilities can leverage to provide better lazy import experiences. +For example, static analysis tools could detect modules with side effects and +automatically configure appropriate filters. **In the future** (out of scope for +this PEP), this foundation may enable better ways to declaratively specify which +modules are safe for lazy importing, such as package metadata, type stubs with +lazy-safety annotations, or configuration files. The current filter API is designed +to be flexible enough to accommodate such future enhancements without requiring +changes to the core language specification. + +Example: + +.. code-block:: python + + import sys + + def exclude_side_effect_modules(importer, name, fromlist): + """ + Filter function to exclude modules with import-time side effects. + + Args: + importer: Name of the module doing the import + name: Name of the module being imported + fromlist: Tuple of names being imported (for 'from' imports), or None + + Returns: + True to allow lazy import, False to force eager import + """ + # Modules known to have important import-time side effects + side_effect_modules = {'legacy_plugin_system', 'metrics_collector'} + + if name in side_effect_modules: + return False # Force eager import + + return True # Allow lazy import + + # Install the filter + sys.set_lazy_imports_filter(exclude_side_effect_modules) + + # These imports are checked by the filter + lazy import data_processor # Filter returns True -> stays lazy + lazy import legacy_plugin_system # Filter returns False -> imported eagerly + + print('data_processor' in sys.modules) # False - still lazy + print('legacy_plugin_system' in sys.modules) # True - loaded eagerly + + # First use of data_processor triggers loading + result = data_processor.transform(data) + print('data_processor' in sys.modules) # True - now loaded + +Global lazy imports control +---------------------------- + +The global lazy imports flag can be controlled through: + +* The ``-X lazy_imports=`` command-line option +* The ``PYTHON_LAZY_IMPORTS=`` environment variable +* The ``sys.set_lazy_imports(mode)`` function (primarily for testing) + +Where ```` can be: + +* ``"default"`` (or unset): Only explicitly marked lazy imports are lazy +* ``"enabled"``: All module-level imports (except in ``try`` blocks and ``import *``) become *potentially lazy* +* ``"disabled"``: No imports are lazy, even those explicitly marked with ``lazy`` keyword + +When the global flag is set to ``"enabled"``, all imports at the global level of +all modules—except statements in a ``try`` block and ``from ... import *``—are +*potentially lazy*. + +If the global lazy imports flag is set to ``"disabled"``, no *potentially lazy* +import is ever imported lazily, and the behavior is equivalent to a regular +``import`` statement: the import is *eager* (as if the lazy keyword was not used). + Security Implications ===================== @@ -354,7 +656,7 @@ developers can mark imports that will only be used for type annotations as ``laz Below is guidance on how to best take advantage of lazy imports and how to avoid incompatibilities: -* When adopting lazy imports, users should be aware that eliding an import until it is +* When adopting lazy imports, users should be aware that eliding an import until it is used will result in side effects not being executed. In turn, users should be wary of modules that rely on import time side effects. Perhaps the most common reliance on import side effects is the registry pattern, where population of some external @@ -363,6 +665,29 @@ incompatibilities: Instead, registries of objects should be constructed via explicit discovery processes (e.g. a well-known function to call). + .. code-block:: python + + # Problematic: Plugin registers itself on import + # my_plugin.py + from plugin_registry import register_plugin + + @register_plugin("MyPlugin") + class MyPlugin: + pass + + # In main code: + lazy import my_plugin + # Plugin NOT registered yet - module not loaded! + + # Better: Explicit discovery + # plugin_registry.py + def discover_plugins(): + from my_plugin import MyPlugin + register_plugin(MyPlugin) + + # In main code: + plugin_registry.discover_plugins() # Explicit loading + * Always import needed submodules explicitly. It is not enough to rely on a different import to ensure a module has its submodules as attributes. Plainly, unless there is an explicit ``from . import bar`` in ``foo/__init__.py``, always use ``import @@ -375,6 +700,19 @@ incompatibilities: them to keep dependencies clear and avoid the overhead of repeatedly re-resolving the import but will still speed up the program. + .. code-block:: python + + # Before: Inline import (repeated overhead) + def process_data(data): + import json # Re-resolved on every call + return json.dumps(data) + + # After: Lazy import at module level + lazy import json + + def process_data(data): + return json.dumps(data) # Loaded once on first call + * Avoid using star imports, as those are always eager. FAQ @@ -395,6 +733,28 @@ deferred until first use of the lazy name. This is similar to moving an import into a function. The error will occur with a clear traceback pointing to the first access of the lazy object. +The implementation provides enhanced error reporting through exception chaining. +When a lazy import fails during reification, the original exception is preserved +and chained, showing both where the import was defined and where it was first +used: + +.. code-block:: python + + Traceback (most recent call last): + File "test.py", line 1, in + lazy import broken_module + ImportError: deferred import of 'broken_module' raised an exception during resolution + + The above exception was the direct cause of the following exception: + + Traceback (most recent call last): + File "test.py", line 3, in + broken_module.foo() + ^^^^^^^^^^^^^ + File "broken_module.py", line 2, in + 1/0 + ZeroDivisionError: division by zero + **Q: How do lazy imports affect modules with import-time side effects?** A: Side effects are deferred until first use. This is generally desirable for performance, but may require code changes for modules that rely on import-time registration patterns. We recommend: @@ -405,10 +765,46 @@ A: Side effects are deferred until first use. This is generally desirable for pe **Q: Can I use lazy imports with** ``from ... import ...`` **statements?** -A: Yes as long as you don't use ``from ... import *``. Both ``lazy import foo`` +A: Yes, as long as you don't use ``from ... import *``. Both ``lazy import foo`` and ``lazy from foo import bar`` are supported. The ``bar`` name will be bound to a lazy object that resolves to ``foo.bar`` on first use. +**Q: Does** ``lazy from module import Class`` **load the entire module or just the class?** + +A: It loads the **entire module**, not just the class. This is because Python's +import system always executes the complete module file—there's no mechanism to +execute only part of a ``.py`` file. When you first access ``Class``, Python: + +1. Loads and executes the entire ``module.py`` file +2. Extracts the ``Class`` attribute from the resulting module object +3. Binds ``Class`` to the name in your namespace + +This is identical to eager ``from module import Class`` behavior. The only difference +with lazy imports is that steps 1-3 happen on first use instead of at the import +statement. + +.. code-block:: python + + # heavy_module.py + print("Loading heavy_module") # This ALWAYS runs when module loads + + class MyClass: + pass + + class UnusedClass: + pass # Also gets defined, even though we don't import it + + # app.py + lazy from heavy_module import MyClass + + print("Import statement done") # heavy_module not loaded yet + obj = MyClass() # NOW "Loading heavy_module" prints + # (and UnusedClass gets defined too) + +**Key point**: Lazy imports defer *when* a module loads, not *what* gets loaded. +You cannot selectively load only parts of a module—Python's import system doesn't +support partial module execution. + **Q: What about type annotations and** ``TYPE_CHECKING`` **imports?** A: Lazy imports eliminate the need for ``TYPE_CHECKING`` guards. You can write: @@ -417,6 +813,9 @@ A: Lazy imports eliminate the need for ``TYPE_CHECKING`` guards. You can write: lazy from collections.abc import Sequence, Mapping # No runtime cost + def process(items: Sequence[str]) -> Mapping[str, int]: + ... + Instead of: .. code-block:: python @@ -425,6 +824,9 @@ Instead of: if TYPE_CHECKING: from collections.abc import Sequence, Mapping + def process(items: Sequence[str]) -> Mapping[str, int]: + ... + **Q: What's the performance overhead of lazy imports?** A: The overhead is minimal: @@ -434,6 +836,11 @@ A: The overhead is minimal: - Reification (first use) has the same cost as a regular import. - No ongoing performance penalty unlike ``importlib.util.LazyLoader``. +Benchmarking with the `pyperformance suite`_ shows the implementation is performance +neutral when lazy imports are not used. + +.. _pyperformance suite: https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20250922-3.15.0a0-27836e5/bm-20250922-vultr-x86_64-DinoV-lazy_imports-3.15.0a0-27836e5-vs-base.svg + **Q: Can I mix lazy and eager imports of the same module?** A: Yes. If module ``foo`` is imported both lazily and eagerly in the same @@ -472,6 +879,31 @@ either the lazy proxy or the final resolved object. A: Yes, accessing a module's ``__dict__`` will reify all lazy objects in that module. Individual lazy objects can be resolved by calling their ``get()`` method. +**Q: What's the difference between** ``globals()`` **and** ``mod.__dict__`` **for lazy imports?** + +A: Calling ``globals()`` returns the module's dictionary without reifying lazy +imports — you'll see lazy proxy objects when accessing them through the returned +dictionary. However, accessing ``mod.__dict__`` from external code reifies all lazy +imports in that module first. This design ensures: + +.. code-block:: python + + # In your module: + lazy import json + + g = globals() + print(type(g['json'])) # - your problem + + # From external code: + import sys + mod = sys.modules['your_module'] + d = mod.__dict__ + print(type(d['json'])) # - reified for external access + +This distinction means adding lazy imports and calling ``globals()`` is your +responsibility to manage, while external code accessing ``mod.__dict__`` always +sees fully loaded modules. + **Q: Why not use** ``importlib.util.LazyLoader`` **instead?** A: ``LazyLoader`` has significant limitations: @@ -514,18 +946,25 @@ A: Use the ``__lazy_modules__`` global for compatibility: import expensive_module from expensive_module_2 import MyClass +The ``__lazy_modules__`` attribute is a list of module name strings. When an import +statement is executed, Python checks if the module name being imported appears in +``__lazy_modules__``. If it does, the import is treated as if it had the ``lazy`` +keyword (becoming *potentially lazy*). On Python versions before 3.15 that don't +support lazy imports, the ``__lazy_modules__`` attribute is simply ignored and +imports proceed eagerly as normal. + This provides a migration path until you can rely on the ``lazy`` keyword. For maximum predictability, it's recommended to define ``__lazy_modules__`` once, before any imports. But as it is checked on each import, it can be modified between -``import`` statements +``import`` statements. **Q: How do explicit lazy imports interact with PEP-649/PEP-749** A: If an annotation is not stringified, then it is a reference to the type, but it -will only be resolved if the annotation is accessed. In the example below, the ``fake_typing`` module +will only be resolved if the annotation is accessed. In the example below, the ``fake_typing`` module is only loaded when the user inspects the ``__annotations__`` dictionary. The ``fake_typing`` module would also be loaded if the user had leveraged ``inspect.get_annotations()`` or ``getattr`` to access -the annotations. +the annotations. .. code-block:: python @@ -533,15 +972,172 @@ the annotations. def foo(x: MyFakeType): pass print(foo.__annotations__) # Triggers loading the fake_typing module + +**Q: How do lazy imports interact with** ``dir()``, ``getattr()``, **and module introspection?** + +A: Accessing lazy imports through normal attribute access or ``getattr()`` will trigger +reification. Calling ``dir()`` on a module will reify all lazy imports in that module +to ensure the directory listing is complete. This is similar to accessing ``mod.__dict__``. + +.. code-block:: python + + lazy import json + + # Before any access + # json not in sys.modules + + # This triggers reification: + dumps_func = getattr(json, 'dumps') + # Now json is in sys.modules + +**Q: What happens if I enable global lazy imports mode and a library breaks?** + +A: *Note: This is an advanced feature.* You can use the lazy imports filter to exclude +specific modules that are known to have problematic side effects: + +.. code-block:: python + + import sys + + def my_filter(importer, name, fromlist): + # Don't lazily import modules known to have side effects + if name in ('problematic_module', 'another_module'): + return False # Import eagerly + return True # Allow lazy import + + sys.set_lazy_imports_filter(my_filter) + +The filter function receives the importer module name, the module being imported, and +the fromlist (if using ``from ... import``). Returning ``False`` forces an eager import. + +Alternatively, set the global mode to ``"disabled"`` via ``-X lazy_imports=disabled`` +to turn off all lazy imports for debugging. + +**Q: Do lazy imports work with circular imports?** + +A: Lazy imports don't automatically solve circular import problems. If two modules +have a circular dependency, making the imports lazy might help **only if** the circular +reference isn't accessed during module initialization. However, if either module +accesses the other during import time, you'll still get an error. + +**Example that works** (deferred access in functions): + +.. code-block:: python + + # user_model.py + lazy import post_model + + class User: + def get_posts(self): + # OK - post_model accessed inside function, not during import + return post_model.Post.get_by_user(self.name) + + # post_model.py + lazy import user_model + + class Post: + @staticmethod + def get_by_user(username): + return f"Posts by {username}" + +This works because neither module accesses the other at module level—the access +happens later when ``get_posts()`` is called. + +**Example that fails** (access during import): + +.. code-block:: python + + # module_a.py + lazy import module_b + + result = module_b.get_value() # Error! Accessing during import + + def func(): + return "A" + + # module_b.py + lazy import module_a + + result = module_a.func() # Circular dependency error here + + def get_value(): + return "B" + +This fails because ``module_a`` tries to access ``module_b`` at import time, which +then tries to access ``module_a`` before it's fully initialized. + +The best practice is still to avoid circular imports in your code design. + +Note that PEP 690 (the implicit lazy imports PEP) did help with some circular imports, +but PEP 805 explicitly does not support that use case to maintain backward compatibility +— code should not become dependent on lazy imports to function correctly. + +**Q: Will lazy imports affect the performance of my hot paths?** + +A: After first use, lazy imports have **zero overhead** thanks to the adaptive interpreter. +The interpreter specializes the bytecode (e.g., ``LOAD_GLOBAL`` becomes ``LOAD_GLOBAL_MODULE``) +which eliminates the lazy check on subsequent accesses. This means once a lazy import +is reified, accessing it is just as fast as a normal import. + +.. code-block:: python + + lazy import json + + def use_json(): + return json.dumps({"test": 1}) + + # First call triggers reification + use_json() + + # After 2-3 calls, bytecode is specialized + use_json() + use_json() + +You can observe the specialization using ``dis.dis(use_json, adaptive=True)``: + +.. code-block:: text + + === Before specialization === + LOAD_GLOBAL 0 (json) + LOAD_ATTR 2 (dumps) + + === After 3 calls (specialized) === + LOAD_GLOBAL_MODULE 0 (json) + LOAD_ATTR_MODULE 2 (dumps) + +The specialized ``LOAD_GLOBAL_MODULE`` and ``LOAD_ATTR_MODULE`` instructions are +optimized fast paths with no overhead for checking lazy imports. + +**Q: What about** ``sys.modules``? **When does a lazy import appear there?** + +A: A lazily imported module does **not** appear in ``sys.modules`` until it's reified +(first used). Once reified, it appears in ``sys.modules`` just like any eager import. + +.. code-block:: python + + import sys + lazy import json + + print('json' in sys.modules) # False + + result = json.dumps({"key": "value"}) # First use + + print('json' in sys.modules) # True Reference Implementation ======================== -[Link to any existing implementation and details about its state, e.g. proof-of-concept.] +A reference implementation is available at: +https://github.com/LazyImportsCabal/cpython/tree/lazy Alternate Implementation ======================== +Here are some alternative design decisions that were considered during the development +of this PEP. While the current proposal represents what we believe to be the best balance +of simplicity, performance, and maintainability, these alternatives offer different +trade-offs that may be valuable for implementers to consider or for future refinements. + Leveraging a Subclass of Dict ----------------------------- @@ -564,11 +1160,24 @@ Rejected Ideas Modification of the Dict Object ------------------------------- -The initial PEP for lazy imports (PEP 690) relied heavily on the modification of the internal dict -object to support lazy imports. We recognize that this data structure is highly tuned, heavily used -across the codebase, and very performance sensitive. Because of the importance of this data structure -and the desire to keep the implementation of lazy imports encapsulated from users who may have no -interest in the feature, we’ve decided to invest in an alternate approach. +The initial PEP for lazy imports (PEP 690) relied heavily on the modification of the internal dict +object to support lazy imports. We recognize that this data structure is highly tuned, heavily used +across the codebase, and very performance sensitive. Because of the importance of this data structure +and the desire to keep the implementation of lazy imports encapsulated from users who may have no +interest in the feature, we've decided to invest in an alternate approach. + +The dictionary is the foundational data structure in Python. Every object's attributes are stored +in a dict, and dicts are used throughout the runtime for namespaces, keyword arguments, and more. +Adding any kind of hook or special behavior to dicts to support lazy imports would: + +1. Prevent critical interpreter optimizations including future JIT compilation +2. Add complexity to a data structure that must remain simple and fast +3. Affect every part of Python, not just import behavior +4. Violate separation of concerns—the hash table shouldn't know about the import system + +Past decisions that violated this principle of keeping core abstractions clean have caused +significant pain in the CPython ecosystem, making optimization difficult and introducing +subtle bugs. Support of Import Cycles ------------------------ @@ -596,28 +1205,81 @@ We discussed appending lazy to the end of import statements like such ``import f For example, if multiple modules are imported in a single statement, it is unclear if the lazy binding applies to all of the imported objects or just a subset of the items. -Reifying lazy imports with ``globals()`` is called. ---------------------------------------------------- -While ``globals()`` is also a common way to access variables which may contain lazy imports we will -not reify the lazy imports when it is called. There are a couple of problems with doing this. One -reason is that it is impossible to safely do this on-demand: we cannot return a proxy object as this -will break other usages of globals (e.g. passing them back to ``exec``). +Returning a Proxy Dict from ``globals()`` +------------------------------------------ -Therefore it would require reifying all of the objects on the access and this behavior may be -surprising. There may also be a performance cost as it is impractical to cache whether this scan has -been done with just the globals dictionary in hand (versus when accessing from the module where we -can cache the results in the module). +An alternative to reifying on ``globals()`` or exposing lazy objects would be to +return a proxy dictionary that automatically reifies lazy objects when they're +accessed through the proxy. This would seemingly give the best of both worlds: +``globals()`` returns immediately without reification cost, but accessing items +through the result would automatically resolve lazy imports. -Open Issues -=========== +However, this approach is fundamentally incompatible with how ``globals()`` is used +in practice. Many standard library functions and built-ins expect ``globals()`` to +return a real ``dict`` object, not a proxy: -[Any points that are still being decided/discussed.] +- ``exec(code, globals())`` requires a real dict +- ``eval(expr, globals())`` requires a real dict +- Functions that check ``type(globals()) is dict`` would break +- Dictionary methods like ``.update()`` would need special handling +- Performance would suffer from the indirection on every access + +The proxy would need to be so transparent that it would be indistinguishable from +a real dict in almost all cases, which is extremely difficult to achieve correctly. +Any deviation from true dict behavior would be a source of subtle bugs. + +Reifying lazy imports when ``globals()`` is called +--------------------------------------------------- +Calling ``globals()`` returns the module's namespace dictionary without triggering +reification of lazy imports. Accessing lazy objects through the returned dictionary +yields the lazy proxy objects themselves. This is an intentional design decision +for several reasons: + +**The key distinction**: Adding a lazy import and calling ``globals()`` is the +module author's concern and under their control. However, accessing ``mod.__dict__`` +from external code is a different scenario — it crosses module boundaries and affects +someone else's code. Therefore, ``mod.__dict__`` access reifies all lazy imports to +ensure external code sees fully realized modules, while ``globals()`` preserves lazy +objects for the module's own introspection needs. + +**Technical challenges**: It is impossible to safely reify on-demand when ``globals()`` +is called because we cannot return a proxy dictionary — this would break common usages +like passing the result to ``exec()`` or other built-ins that expect a real dictionary. +The only alternative would be to eagerly reify all lazy imports whenever ``globals()`` +is called, but this behavior would be surprising and potentially expensive. + +**Performance concerns**: It is impractical to cache whether a reification scan has +been performed with just the globals dictionary reference, whereas module attribute +access (the primary use case) can efficiently cache reification state in the module +object itself. + +**Use case rationale**: The chosen design makes sense precisely because of this distinction: +adding a lazy import and calling ``globals()`` is your problem to manage, while having lazy +imports visible in ``mod.__dict__`` becomes someone else's problem. By reifying on +``__dict__`` access but not on ``globals()``, we ensure external code always sees +fully loaded modules while giving module authors control over their own introspection. + +Note that three options were considered: + +1. Calling ``globals()`` or ``mod.__dict__`` traverses and resolves all lazy objects before returning +2. Calling ``globals()`` or ``mod.__dict__`` returns the dictionary with lazy objects present +3. Calling ``globals()`` returns the dictionary with lazy objects, but ``mod.__dict__`` reifies everything + +We chose the third option because it properly delineates responsibility: if you add lazy imports +to your module and call ``globals()``, you're responsible for handling the lazy objects. +But external code accessing your module's ``__dict__`` shouldn't need to know about your +lazy imports—it gets fully resolved modules. Acknowledgements ================ -[Thank anyone who has helped with the PEP.] +We would like to thank Paul Ganssle, Yury Selivanov, Łukasz Langa, Lysandros +Nikolaou, Pradyun Gedam, Mark Shannon, Hana Joo and the Python Google team, the +Python team(s) @ Meta, the Python @ HRT team, the Bloomberg Python team, the +Scientific Python community, everyone who participated in the initial discussion +of :pep:`690`, and many others who provided valuable feedback and insights that +helped shape this PEP. Footnotes From bc72140a13987ea0518dd1b60575294c4b22632a Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Thu, 2 Oct 2025 02:11:54 +0100 Subject: [PATCH 27/39] Small stdlib analysis --- peps/pep-0805.rst | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 0f49aeb1a17..46d73c2067b 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -47,7 +47,14 @@ they want and invoke the program "for real." A somewhat common response is to delay imports by moving them into functions (inline imports), but this practice is very manual to implement and maintain. -Additionally, it obfuscates the full set of dependencies for a module. +Additionally, it obfuscates the full set of dependencies for a module. Analysis +of the Python standard library shows that approximately 17% of all imports outside +teests (nearly 3500 total imports across 730 files) are already placed inside +functions, classes, or methods specifically to defer their execution. This +demonstrates that developers are already manually implementing lazy imports in +performance-sensitive code, but doing so requires scattering imports throughout +the codebase and makes the full dependency graph harder to understand at a +glance. The standard library provides ``importlib.util.LazyLoader`` to solve some of these problems. It permits imports at the module level to work *mostly* like inline From e3c6ffefc2dc8f009be36de2599fcdae51d927f6 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Thu, 2 Oct 2025 02:26:35 +0100 Subject: [PATCH 28/39] Simplify grammar --- peps/pep-0805.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 46d73c2067b..f2353aa8264 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -219,8 +219,7 @@ has special meaning when it appears before import statements: .. code-block:: text import_name: - | 'import' dotted_as_names - | 'lazy' 'import' dotted_as_names + | 'lazy'? 'import' dotted_as_names import_from: | 'lazy'? 'from' ('.' | '...')* dotted_name 'import' import_from_targets From e525ef0ffae3b634765b310dc449da822fdfacca Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Thu, 2 Oct 2025 02:38:06 +0100 Subject: [PATCH 29/39] More examples --- peps/pep-0805.rst | 43 ++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index f2353aa8264..418f57c79a4 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -384,7 +384,29 @@ It **only** resolves the lazy object being accessed. Accessing a lazy object (from a global variable or a module attribute) reifies the object. Accessing a module's ``__dict__`` reifies **all** lazy objects in that module. Operations that indirectly access ``__dict__`` (such as ``dir()``) -also trigger this behavior. However, calling ``globals()`` does **not** trigger +also trigger this behavior. + +Example using ``__dict__`` from external code: + +.. code-block:: python + + # my_module.py + import sys + lazy import json + + print('json' in sys.modules) # False - still lazy + + # main.py + import sys + import my_module + + # Accessing __dict__ from external code DOES reify all lazy imports + d = my_module.__dict__ + + print('json' in sys.modules) # True - reified by __dict__ access + print(type(d['json'])) # + +However, calling ``globals()`` does **not** trigger reification — it returns the module's dictionary, and accessing lazy objects through that dictionary still returns lazy proxy objects that need to be reified upon use. @@ -392,6 +414,25 @@ More indirect ways of accessing arbitrary globals (e.g. inspecting ``frame.f_globals``) do **not** reify all the objects. A lazy object can be resolved explicitly by calling the ``get`` method. +Example using ``globals()``: + +.. code-block:: python + + import sys + lazy import json + + # Calling globals() does NOT trigger reification + g = globals() + + print('json' in sys.modules) # False - still lazy + print(type(g['json'])) # + + # Explicitly reify using the get() method + resolved = g['json'].get() + + print(type(resolved)) # + print('json' in sys.modules) # True - now loaded + Bytecode and adaptive specialization ------------------------------------- From f33685e91348398d21c2d8c8f2287df8bc1d8b78 Mon Sep 17 00:00:00 2001 From: Pablo Galindo Salgado Date: Thu, 2 Oct 2025 02:41:04 +0100 Subject: [PATCH 30/39] remove false sentence --- peps/pep-0805.rst | 4 ---- 1 file changed, 4 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 418f57c79a4..552dd9fe63a 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -1115,10 +1115,6 @@ then tries to access ``module_a`` before it's fully initialized. The best practice is still to avoid circular imports in your code design. -Note that PEP 690 (the implicit lazy imports PEP) did help with some circular imports, -but PEP 805 explicitly does not support that use case to maintain backward compatibility -— code should not become dependent on lazy imports to function correctly. - **Q: Will lazy imports affect the performance of my hot paths?** A: After first use, lazy imports have **zero overhead** thanks to the adaptive interpreter. From a5b4df73b9800af54b9eb6a11bbc46af84d9990c Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Thu, 2 Oct 2025 17:57:29 +0200 Subject: [PATCH 31/39] Fix small typo. --- peps/pep-0805.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 552dd9fe63a..423a672a792 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -49,7 +49,7 @@ A somewhat common response is to delay imports by moving them into functions (inline imports), but this practice is very manual to implement and maintain. Additionally, it obfuscates the full set of dependencies for a module. Analysis of the Python standard library shows that approximately 17% of all imports outside -teests (nearly 3500 total imports across 730 files) are already placed inside +tests (nearly 3500 total imports across 730 files) are already placed inside functions, classes, or methods specifically to defer their execution. This demonstrates that developers are already manually implementing lazy imports in performance-sensitive code, but doing so requires scattering imports throughout From 918b6ff12668e7c990f00a2244bd1235cee52a38 Mon Sep 17 00:00:00 2001 From: Tim Stumbaugh Date: Thu, 2 Oct 2025 10:22:00 -0600 Subject: [PATCH 32/39] Make my final pass of language --- peps/pep-0805.rst | 81 +++++++++++++++++++++++++---------------------- 1 file changed, 43 insertions(+), 38 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 552dd9fe63a..613f2d1c700 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -49,7 +49,7 @@ A somewhat common response is to delay imports by moving them into functions (inline imports), but this practice is very manual to implement and maintain. Additionally, it obfuscates the full set of dependencies for a module. Analysis of the Python standard library shows that approximately 17% of all imports outside -teests (nearly 3500 total imports across 730 files) are already placed inside +tests (nearly 3500 total imports across 730 files) are already placed inside functions, classes, or methods specifically to defer their execution. This demonstrates that developers are already manually implementing lazy imports in performance-sensitive code, but doing so requires scattering imports throughout @@ -230,7 +230,7 @@ Syntax restrictions The soft keyword is only allowed at the global (module) level, **not** inside functions, class bodies, or try blocks. Import statements that use the soft keyword -are *potentially lazy*. In addition, star imports will not be supported for lazy +are *potentially lazy*. In addition, wild card imports will not be supported for lazy import semantics by the soft keyword and will instead raise a syntax error. Specifically: @@ -242,8 +242,9 @@ Specifically: 2. **Not in try blocks**: Lazy imports are not allowed inside ``try``/``except`` blocks. This raises a ``SyntaxError``. -3. **No star imports**: ``lazy from module import *`` raises a ``SyntaxError``. - Star imports are always eager, even when the global lazy imports flag is enabled. +3. **No wild card imports**: ``lazy from module import *`` raises a ``SyntaxError``. + Wild card ("star") imports are always eager, even when the global lazy + imports flag is enabled. Examples of syntax errors: @@ -291,14 +292,21 @@ Example: A module may contain a ``__lazy_modules__`` attribute, which is a list of module names (strings) to make *potentially lazy* (as if the ``lazy`` keyword was used). This attribute is checked on each ``import`` statement to determine whether -the import should be treated as lazy. +the import should be made *potentially lazy*. The ``__lazy_modules__`` attribute provides a compatibility mechanism for libraries that need to support both Python 3.15+ (with native lazy import support) and older -versions. When ``__lazy_modules__`` is defined, the specified imports will be lazy -on Python 3.15+, but will fall back to eager imports on older Python versions that -don't recognize the mechanism. The attribute should be defined at module level before -the import statements it affects, though it is technically evaluated on each import. +versions. When ``__lazy_modules__`` is defined, the specified imports will be +*potentially lazy* on Python 3.15+, but will fall back to eager imports on older +Python versions that don't recognize this mechanism. The attribute should be +defined at module level before the import statements it affects, though it is +technically evaluated on each import. + +For a *potentially lazy* import, the lazy imports filter (if set) is called with +the name of the module doing the import, the name of the module being +imported, and (if applicable) the fromlist. If the lazy import filter returns +``True``, the *potentially lazy* import becomes a lazy import. Otherwise, the +import is *not* lazy, and the normal (eager) import continues. Lazy import mechanism --------------------- @@ -307,7 +315,7 @@ When an import is lazy, ``__lazy_import__`` is called instead of ``__import__``. ``__lazy_import__`` has the same function signature as ``__import__``. It adds the module name to ``sys.lazy_modules``, a set of module names which have been lazily imported at some point (primarily for -diagnostics and introspection), and returns a "lazy module object". +diagnostics and introspection), and returns a "lazy module object." The implementation of ``from ... import`` (the ``IMPORT_FROM`` bytecode implementation) checks if the module it's fetching from is a lazy module @@ -329,7 +337,7 @@ Reification ----------- When a lazy object is first used, it needs to be reified. This means -resolving the import at that point in the program, and replacing the lazy +resolving the import at that point in the program and replacing the lazy object with the concrete one. Reification imports the module in the same way as it would have been if it had been imported eagerly, barring intervening changes to the import system (e.g. to ``sys.path``, ``sys.meta_path``, @@ -340,7 +348,8 @@ reified, it's removed from ``sys.lazy_modules``. If reification fails (e.g., due to an ``ImportError``), the exception is enhanced with chaining to show both where the lazy import was defined and where it was first -accessed. This provides clear debugging information: +accessed (even though it propagates from the code that triggered reification). +This provides clear debugging information: .. code-block:: python @@ -444,9 +453,9 @@ The ``lazy`` syntax sets a flag in the ``IMPORT_NAME`` instruction's oparg ``_PyEval_LazyImportName()`` instead of ``_PyEval_ImportName()``, creating a lazy import object rather than executing the import immediately. The ``IMPORT_FROM`` instruction checks whether its source is a lazy import (``PyLazyImport_CheckExact()``) -and creates a lazy import for the attribute rather than accessing it immediately. +and creates a lazy object for the attribute rather than accessing it immediately. -When a lazy import is accessed, it must be reified. The ``LOAD_GLOBAL`` instruction +When a lazy object is accessed, it must be reified. The ``LOAD_GLOBAL`` instruction (used in function scopes) and ``LOAD_NAME`` instruction (used at module level) both check whether the object being loaded is a lazy import. If so, they call ``_PyImport_LoadLazyImportTstate()`` to perform the actual import and store the @@ -523,12 +532,12 @@ or an interpreter-wide switch). Unchanged semantics ------------------- -* Regular ``import`` and ``from ... import ...`` remain eager unless explicitly - made *potentially lazy* by the local or global mechanisms provided. +* Regular ``import`` and ``from ... import ...`` statements remain eager unless + explicitly made *potentially lazy* by the local or global mechanisms provided. * Dynamic import APIs remain eager and unchanged: ``__import__()`` and ``importlib.import_module()``. * Import hooks and loaders continue to run under the standard import protocol - when a lazy binding is first used. + when a lazy object is reified. Observable behavioral shifts (opt-in only) ------------------------------------------ @@ -552,11 +561,11 @@ These changes are limited to bindings explicitly made lazy: * **Side-effect timing.** Import-time side effects in lazily imported modules occur at first use of the binding, not at module import time. * **Import order.** Because modules are imported on first use, the order in - which modules are imported may change. -* **Presence in ``sys.modules``.** A lazily imported module may not appear in + which modules are imported may differ from how they appear in code. +* **Presence in ``sys.modules``.** A lazily imported module does not appear in ``sys.modules`` until first use. After reification, it must appear in ``sys.modules``. If some other code eagerly imports the same module before - first use, the lazy binding resolves to that existing module object when + first use, the lazy binding resolves to that existing (lazy) module object when it is first used. * **Proxy visibility.** Before first use, the bound name refers to a lazy proxy. Indirect introspection that touches the value may observe a proxy lazy object @@ -574,7 +583,7 @@ object. Lazy imports are thread-safe and have no special considerations for free-threading. A module that would normally be imported in the main thread may be imported in a different thread if that thread triggers the first access to the lazy import. This -is not a problem—the import lock ensures thread safety regardless of which thread +is not a problem: the import lock ensures thread safety regardless of which thread performs the import. Subinterpreters are supported. Each subinterpreter maintains its own @@ -592,13 +601,7 @@ display lazy proxies before first use and the real objects thereafter. Lazy imports filter ------------------- -For *potentially lazy* imports, the lazy imports filter (if set) is called, -passing the name of the module doing the import, the name of the module being -imported, and (if applicable) the fromlist. If the lazy import filter returns -``True``, the *potentially lazy* import becomes a lazy import. Otherwise, the -import is *not* lazy, and the normal (eager) import continues. - -This PEP adds two new functions to the ``sys`` module to manage the filter: +This PEP adds two new functions to the ``sys`` module to manage the lazy imports filter: * ``sys.set_lazy_imports_filter(func)`` - Sets the filter function. The ``func`` parameter must have the signature: ``func(importer: str, name: str, fromlist: tuple[str, ...] | None) -> bool`` @@ -674,12 +677,13 @@ Where ```` can be: * ``"disabled"``: No imports are lazy, even those explicitly marked with ``lazy`` keyword When the global flag is set to ``"enabled"``, all imports at the global level of -all modules—except statements in a ``try`` block and ``from ... import *``—are -*potentially lazy*. +all modules are *potentially lazy* **except** for those inside a ``try`` or +``with`` block or any wild card (``from ... import *``) import. If the global lazy imports flag is set to ``"disabled"``, no *potentially lazy* -import is ever imported lazily, and the behavior is equivalent to a regular -``import`` statement: the import is *eager* (as if the lazy keyword was not used). +import is ever imported lazily, the import filter is never called, and the +behavior is equivalent to a regular ``import`` statement: +the import is *eager* (as if the lazy keyword was not used). Security Implications @@ -760,7 +764,7 @@ incompatibilities: def process_data(data): return json.dumps(data) # Loaded once on first call -* Avoid using star imports, as those are always eager. +* Avoid using wild card (star) imports, as those are always eager. FAQ === @@ -905,9 +909,10 @@ A: Migration is incremental: **Q: What about star imports** (``from module import *``)? -A: Star imports cannot be lazy - they remain eager. This is because the set of -names being imported cannot be determined without loading the module. Using the -``lazy`` keyword with star imports will be a syntax error. +A: Wild card (star) imports cannot be lazy - they remain eager. This is because the +set of names being imported cannot be determined without loading the module. Using the +``lazy`` keyword with star imports will be a syntax error. If lazy imports are globally +enabled, star imports will still be eager. **Q: How do lazy imports interact with import hooks and custom loaders?** @@ -1037,7 +1042,7 @@ to ensure the directory listing is complete. This is similar to accessing ``mod. dumps_func = getattr(json, 'dumps') # Now json is in sys.modules -**Q: What happens if I enable global lazy imports mode and a library breaks?** +**Q: What happens if I globally enable lazy imports mode and a library doesn't work correctly?** A: *Note: This is an advanced feature.* You can use the lazy imports filter to exclude specific modules that are known to have problematic side effects: @@ -1236,7 +1241,7 @@ codebases, we’ve decided not to implement this feature and instead prioritize Placing the ``lazy`` Keyword in the Middle of From Imports ---------------------------------------------------------- -While we found, ``from foo lazy import bar`` to be a really intuitive placement for the new explicit syntax, +While we found ``from foo lazy import bar`` to be a really intuitive placement for the new explicit syntax, we quickly learned that placing the ``lazy`` keyword here is already syntactically allowed in Python. This is because ``from foo. lazy import bar`` is legal syntax. From 0244d19d4d052cc2278bb82c8d2fb00a0a75ecd5 Mon Sep 17 00:00:00 2001 From: Tim Stumbaugh Date: Thu, 2 Oct 2025 10:37:35 -0600 Subject: [PATCH 33/39] more with blocks --- peps/pep-0805.rst | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 613f2d1c700..aeeca116f37 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -229,7 +229,7 @@ Syntax restrictions ~~~~~~~~~~~~~~~~~~~ The soft keyword is only allowed at the global (module) level, **not** inside -functions, class bodies, or try blocks. Import statements that use the soft keyword +functions, class bodies, or try/with blocks. Import statements that use the soft keyword are *potentially lazy*. In addition, wild card imports will not be supported for lazy import semantics by the soft keyword and will instead raise a syntax error. @@ -239,8 +239,8 @@ Specifically: Using ``lazy`` inside a function, class body, or any nested scope raises a ``SyntaxError``. -2. **Not in try blocks**: Lazy imports are not allowed inside ``try``/``except`` - blocks. This raises a ``SyntaxError``. +2. **Not in try/with blocks**: Lazy imports are not allowed inside ``try``/``except`` + or ``with`` block. This raises a ``SyntaxError``. 3. **No wild card imports**: ``lazy from module import *`` raises a ``SyntaxError``. Wild card ("star") imports are always eager, even when the global lazy @@ -264,6 +264,10 @@ Examples of syntax errors: except ImportError: pass + # SyntaxError: lazy import not allowed inside with blocks + with suppress(ImportError): + lazy import json + # SyntaxError: lazy from ... import * is not allowed lazy from json import * @@ -673,7 +677,7 @@ The global lazy imports flag can be controlled through: Where ```` can be: * ``"default"`` (or unset): Only explicitly marked lazy imports are lazy -* ``"enabled"``: All module-level imports (except in ``try`` blocks and ``import *``) become *potentially lazy* +* ``"enabled"``: All module-level imports (except in ``try`` or ``with`` blocks and ``import *``) become *potentially lazy* * ``"disabled"``: No imports are lazy, even those explicitly marked with ``lazy`` keyword When the global flag is set to ``"enabled"``, all imports at the global level of From 9dbcbbac5e4201122bb847529922151aaf311420 Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Thu, 2 Oct 2025 21:22:13 +0200 Subject: [PATCH 34/39] Move a sentence around Move a sentence around so the emphasis on the paragraph that starts talking about the new explicit keyword is actually on the explicit keyword, and not the old workarounds. --- peps/pep-0805.rst | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index aeeca116f37..ab3743bca5c 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -63,11 +63,11 @@ in `SPEC 1 `_. There's also the third-party `lazy_loader `_ package. Imports used solely for static type checking are another source of potentially unneeded imports, and there are similarly disparate approaches to minimizing the overhead. +These approaches don't cover all cases, add runtime overhead in unexpected places, +are non-obvious, and non-standardized. -These approaches don't cover all cases, add runtime overhead in unexpected -places, are non-obvious, and non-standardized. This proposal introduces -lazy imports syntax with a design that is local, explicit, controlled, and -granular. Each of these qualities is essential to making the feature +This proposal introduces lazy imports syntax with a design that is local, explicit, +controlled, and granular. Each of these qualities is essential to making the feature predictable and safe to use in practice. The behavior is **local**: laziness applies only to the specific import marked From 6b7e48aa75578e639cfe9cda8f1236d33728daab Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Thu, 2 Oct 2025 21:22:48 +0200 Subject: [PATCH 35/39] Emphasize "lazy import syntax" by bolding it. --- peps/pep-0805.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index ab3743bca5c..d75751ec50b 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -66,7 +66,7 @@ imports, and there are similarly disparate approaches to minimizing the overhead These approaches don't cover all cases, add runtime overhead in unexpected places, are non-obvious, and non-standardized. -This proposal introduces lazy imports syntax with a design that is local, explicit, +This proposal introduces **lazy imports syntax** with a design that is local, explicit, controlled, and granular. Each of these qualities is essential to making the feature predictable and safe to use in practice. From 9d3f1b8c7fc16b85b6e21490ed38b08b11df5a5f Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Thu, 2 Oct 2025 21:24:02 +0200 Subject: [PATCH 36/39] Markup change. --- peps/pep-0805.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index d75751ec50b..f47ecc8b0b3 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -229,7 +229,7 @@ Syntax restrictions ~~~~~~~~~~~~~~~~~~~ The soft keyword is only allowed at the global (module) level, **not** inside -functions, class bodies, or try/with blocks. Import statements that use the soft keyword +functions, class bodies, or ``try``/``with`` blocks. Import statements that use the soft keyword are *potentially lazy*. In addition, wild card imports will not be supported for lazy import semantics by the soft keyword and will instead raise a syntax error. From 095aa17a3928eeba595a4c0b863c20f0e4df2ef5 Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Thu, 2 Oct 2025 21:24:56 +0200 Subject: [PATCH 37/39] Fix indentation in continuation --- peps/pep-0805.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index f47ecc8b0b3..a4fa8029cef 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -240,7 +240,7 @@ Specifically: ``SyntaxError``. 2. **Not in try/with blocks**: Lazy imports are not allowed inside ``try``/``except`` - or ``with`` block. This raises a ``SyntaxError``. + or ``with`` block. This raises a ``SyntaxError``. 3. **No wild card imports**: ``lazy from module import *`` raises a ``SyntaxError``. Wild card ("star") imports are always eager, even when the global lazy From 344067e54b8a0f8ab9d3994459db09ba16663cd9 Mon Sep 17 00:00:00 2001 From: "T. Wouters" Date: Thu, 2 Oct 2025 21:28:02 +0200 Subject: [PATCH 38/39] Clarify the effect of the global lazy flag --- peps/pep-0805.rst | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index a4fa8029cef..031bfcbf956 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -231,7 +231,8 @@ Syntax restrictions The soft keyword is only allowed at the global (module) level, **not** inside functions, class bodies, or ``try``/``with`` blocks. Import statements that use the soft keyword are *potentially lazy*. In addition, wild card imports will not be supported for lazy -import semantics by the soft keyword and will instead raise a syntax error. +import semantics by the soft keyword and will instead raise a syntax error. Imports that +can't be lazy are unaffected by the global lazy imports flag, and instead are always eager. Specifically: @@ -243,8 +244,6 @@ Specifically: or ``with`` block. This raises a ``SyntaxError``. 3. **No wild card imports**: ``lazy from module import *`` raises a ``SyntaxError``. - Wild card ("star") imports are always eager, even when the global lazy - imports flag is enabled. Examples of syntax errors: From c5e41d2d8dc603b1116a3fe84b1ca7bca58d251e Mon Sep 17 00:00:00 2001 From: Thomas Wouters Date: Thu, 2 Oct 2025 22:15:15 +0200 Subject: [PATCH 39/39] Re-specify the spec. --- peps/pep-0805.rst | 70 ++++++++++++++++++++--------------------------- 1 file changed, 30 insertions(+), 40 deletions(-) diff --git a/peps/pep-0805.rst b/peps/pep-0805.rst index 031bfcbf956..d6e1bb1878f 100644 --- a/peps/pep-0805.rst +++ b/peps/pep-0805.rst @@ -229,21 +229,10 @@ Syntax restrictions ~~~~~~~~~~~~~~~~~~~ The soft keyword is only allowed at the global (module) level, **not** inside -functions, class bodies, or ``try``/``with`` blocks. Import statements that use the soft keyword -are *potentially lazy*. In addition, wild card imports will not be supported for lazy -import semantics by the soft keyword and will instead raise a syntax error. Imports that -can't be lazy are unaffected by the global lazy imports flag, and instead are always eager. - -Specifically: - -1. **Module scope only**: Lazy imports are only allowed at module/global scope. - Using ``lazy`` inside a function, class body, or any nested scope raises a - ``SyntaxError``. - -2. **Not in try/with blocks**: Lazy imports are not allowed inside ``try``/``except`` - or ``with`` block. This raises a ``SyntaxError``. - -3. **No wild card imports**: ``lazy from module import *`` raises a ``SyntaxError``. +functions, class bodies, with ``try``/``with`` blocks, or ``import *``. Import +statements that use the soft keyword are *potentially lazy*. Imports that +can't be lazy are unaffected by the global lazy imports flag, and instead +are always eager. Examples of syntax errors: @@ -273,9 +262,11 @@ Examples of syntax errors: Semantics --------- -When the ``lazy`` keyword is used, the import becomes *potentially lazy*. The module -is not loaded immediately at the import statement; instead, a lazy proxy object is -created and bound to the name. The actual module is loaded on first use of that name. +When the ``lazy`` keyword is used, the import becomes *potentially lazy*. +Unless lazy imports are disabled or suppressed (see below), the module is +not loaded immediately at the import statement; instead, a lazy proxy object +is created and bound to the name. The actual module is loaded on first use +of that name. Example: @@ -297,13 +288,13 @@ module names (strings) to make *potentially lazy* (as if the ``lazy`` keyword wa used). This attribute is checked on each ``import`` statement to determine whether the import should be made *potentially lazy*. -The ``__lazy_modules__`` attribute provides a compatibility mechanism for libraries -that need to support both Python 3.15+ (with native lazy import support) and older -versions. When ``__lazy_modules__`` is defined, the specified imports will be -*potentially lazy* on Python 3.15+, but will fall back to eager imports on older -Python versions that don't recognize this mechanism. The attribute should be -defined at module level before the import statements it affects, though it is -technically evaluated on each import. +The normal (non-lazy) import statement will check the global lazy imports +flag. If it is "enabled", all imports are *potentially lazy* (except for +imports that can't be lazy, as mentioned above.) + +If the global lazy imports flag is set to "disabled", no *potentially lazy* +import is ever imported lazily, and the behavior is equivalent to a regular +import statement: the import is *eager* (as if the lazy keyword was not used). For a *potentially lazy* import, the lazy imports filter (if set) is called with the name of the module doing the import, the name of the module being @@ -327,11 +318,11 @@ object, and if so, returns a lazy object for each name instead. The end result of this process is that lazy imports (regardless of how they are enabled) result in lazy objects being assigned to global variables. -Lazy module objects do not appear in ``sys.modules``, just the -``sys.lazy_modules`` set. Lazy objects should only end up stored in global -variables, and the common ways to access those variables (regular variable -access, module attributes) will resolve lazy imports ("reify") and replace -them when they're accessed. +Lazy module objects do not appear in ``sys.modules``, they're just listed in +the ``sys.lazy_modules`` set. Under normal operation lazy objects should +only end up stored in global variables, and the common ways to access those +variables (regular variable access, module attributes) will resolve lazy +imports ("reify") and replace them when they're accessed. It is still possible to expose lazy objects through other means, like debuggers. This is not considered a problem. @@ -389,7 +380,7 @@ This exception chaining clearly shows: (1) where the lazy import was defined, the error. Reification does **not** automatically occur when a module that was previously lazily -imported is subsequently eagerly imported. Reification also does **not** immediately +imported is subsequently eagerly imported. Reification does **not** immediately resolve all lazy objects (e.g. ``lazy from`` statements) that referenced the module. It **only** resolves the lazy object being accessed. @@ -418,13 +409,12 @@ Example using ``__dict__`` from external code: print('json' in sys.modules) # True - reified by __dict__ access print(type(d['json'])) # -However, calling ``globals()`` does **not** trigger -reification — it returns the module's dictionary, and accessing lazy objects through -that dictionary still returns lazy proxy objects that need to be reified upon use. - -More indirect ways of accessing arbitrary globals (e.g. inspecting -``frame.f_globals``) do **not** reify all the objects. A lazy object can be -resolved explicitly by calling the ``get`` method. +However, calling ``globals()`` does **not** trigger reification — it returns +the module's dictionary, and accessing lazy objects through that dictionary +still returns lazy proxy objects that need to be manually reified upon use. +A lazy object can be resolved explicitly by calling the ``get`` method. +Other, more indirect ways of accessing arbitrary globals (e.g. inspecting +``frame.f_globals``) also do **not** reify all the objects. Example using ``globals()``: @@ -459,12 +449,12 @@ instruction checks whether its source is a lazy import (``PyLazyImport_CheckExac and creates a lazy object for the attribute rather than accessing it immediately. When a lazy object is accessed, it must be reified. The ``LOAD_GLOBAL`` instruction -(used in function scopes) and ``LOAD_NAME`` instruction (used at module level) both +(used in function scopes) and ``LOAD_NAME`` instruction (used at module and class level) both check whether the object being loaded is a lazy import. If so, they call ``_PyImport_LoadLazyImportTstate()`` to perform the actual import and store the module in ``sys.modules``. -This check incurs a small cost on each access. However, Python's adaptive interpreter +This check incurs a very small cost on each access. However, Python's adaptive interpreter can specialize ``LOAD_GLOBAL`` after observing that a lazy import has been reified. After several executions, ``LOAD_GLOBAL`` becomes ``LOAD_GLOBAL_MODULE``, which accesses the module dictionary directly without checking for lazy imports.