Skip to content

Commit 6d8102a

Browse files
committed
Address more of Adam's comments.
1 parent 9e5db03 commit 6d8102a

File tree

1 file changed

+50
-59
lines changed

1 file changed

+50
-59
lines changed

peps/pep-0810.rst

Lines changed: 50 additions & 59 deletions
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,13 @@ Post-History:
1919
Abstract
2020
========
2121

22-
This PEP introduces syntax for lazy imports as an explicit language feature.
22+
This PEP introduces syntax for lazy imports as an explicit language feature:
23+
24+
.. code-block:: python
25+
26+
lazy import json
27+
lazy from json import dumps
28+
2329
Lazy imports defer the loading and execution of a module until the first time
2430
the imported name is used, in contrast to 'normal' imports, which eagerly load
2531
and execute a module at the point of the import statement.
@@ -36,9 +42,9 @@ Motivation
3642
==========
3743

3844
The dominant convention in Python code is to place all imports at the module
39-
level, typically at the beginning of the file. This avoids repetition, makes dependencies clear
40-
and minimizes runtime overhead by only evaluating an import statement once
41-
per module.
45+
level, typically at the beginning of the file. This avoids repetition, makes
46+
import dependencies clear and minimizes runtime overhead by only evaluating
47+
an import statement once per module.
4248

4349
A major drawback with this approach is that importing the first
4450
module for an execution of Python (the "main" module) often triggers an immediate
@@ -65,7 +71,7 @@ The standard library provides the :class:`~importlib.util.LazyLoader` class to s
6571
problems. It permits imports at the module level to work *mostly* like inline
6672
imports do. Many scientific Python libraries have adopted a similar pattern, formalized
6773
in `SPEC 1 <https://scientific-python.org/specs/spec-0001/>`__. There's also the
68-
third-party `lazy_loader <https://pypi.org/project/lazy-loader/>`_ package.
74+
third-party :pypi:`lazy_loader` package, yet another implementation of lazy imports.
6975
Imports used solely for static type checking are another source of potentially unneeded
7076
imports, and there are similarly disparate approaches to minimizing the overhead.
7177
The various approaches used here to defer or remove eager imports do not cover
@@ -106,21 +112,13 @@ performance-sensitive areas of a codebase. As this feature is introduced to the
106112
community, we want to make the experience of onboarding optional, progressive, and
107113
adaptable to the needs of each project.
108114

109-
In addition to the new lazy import syntax, we *also* propose a way to
110-
control lazy imports at the application level: globally disabling or
111-
enabling lazy imports, and selectively disabling.
112-
This global lazy imports flag is provided for debugging, testing, and experimentation,
113-
and is not expected to be the common way to control lazy imports.
115+
Lazy imports provide several concrete advantages:
114116

115-
The design of lazy imports provides several concrete advantages:
116-
117-
* Command-line tools are often invoked directly by a user, so latency — in particular
118-
startup latency — is quite noticeable. These programs are also typically
119-
short-lived processes (contrasted with, e.g., a web server). Most conventions
120-
would have a CLI with multiple subcommands import every dependency up front,
121-
even if the user only requests ``tool --help`` (or ``tool subcommand --help``).
117+
* Command-line tools are often invoked directly by a user, so latency -- in particular
118+
startup latency -- is quite noticeable. These programs are also typically
119+
short-lived processes (contrasted with, e.g., a web server).
122120
With lazy imports, only the code paths actually reached will import a module.
123-
This can reduce startup time by 5070% in practice, providing a visceral improvement
121+
This can reduce startup time by 50-70% in practice, providing a significant improvement
124122
to a common user experience and improving Python's competitiveness in domains
125123
where fast startup matters most.
126124

@@ -133,7 +131,7 @@ The design of lazy imports provides several concrete advantages:
133131
function and type objects, incurring memory costs. In long-lived processes,
134132
this noticeably raises baseline memory usage. Lazy imports defer these costs
135133
until a module is needed, keeping unused subsystems unloaded. Memory savings of
136-
3040% have been observed in real workloads.
134+
30-40% have been observed in real workloads.
137135

138136
Rationale
139137
=========
@@ -161,8 +159,8 @@ Another important decision is to represent lazy imports with proxy objects in
161159
the module's namespace, rather than by modifying dictionary lookup. Earlier
162160
approaches experimented with embedding laziness into dictionaries, but this
163161
blurred abstractions and risked affecting unrelated parts of the runtime. The
164-
dictionary is a fundamental data structure in Pythonliterally every object is
165-
built on top of dictsand adding hooks to dictionaries would prevent critical
162+
dictionary is a fundamental data structure in Python -- literally every object is
163+
built on top of dicts -- and adding hooks to dictionaries would prevent critical
166164
optimizations and complicate the entire runtime. The proxy approach is simpler:
167165
it behaves like a placeholder until first use, at which point it resolves the
168166
import and rebinds the name. From then on, the binding is indistinguishable
@@ -171,7 +169,7 @@ rest of the interpreter unchanged.
171169

172170
Compatibility for library authors was also a key concern. Many maintainers need
173171
a migration path that allows them to support both new and old versions of
174-
Python at once. For this reason, the proposal includes the ``__lazy_modules__``
172+
Python at once. For this reason, the proposal includes the :data:`!__lazy_modules__`
175173
global as a transitional mechanism. A module can declare which imports should
176174
be treated as lazy (by listing the module names as strings), and on Python 3.15
177175
or later those imports will become lazy automatically, as if they were imported
@@ -188,10 +186,6 @@ be done from the "outside in", permitting CLI authors to introduce lazy imports
188186
and speed up user-facing tools, without requiring changes to every library the
189187
tool might use.
190188

191-
By combining explicit syntax, a simple runtime model, a compatibility layer,
192-
and gradual adoption, this proposal balances performance improvements with the
193-
clarity and stability that Python users expect.
194-
195189

196190
Other design decisions
197191
----------------------
@@ -205,8 +199,8 @@ Other design decisions
205199

206200
* In addition, it is useful to provide a mechanism to activate or deactivate lazy
207201
imports at a global level. While the primary design centers on explicit syntax,
208-
there are scenariossuch as large applications, testing environments, or
209-
frameworkswhere enabling laziness consistently across many modules provides
202+
there are scenarios -- such as large applications, testing environments, or
203+
frameworks -- where enabling laziness consistently across many modules provides
210204
the most benefit. A global switch makes it easy to experiment with or enforce
211205
consistent behavior, while still working in combination with the filtering API
212206
to respect exclusions or tool-specific configuration. This ensures that global
@@ -291,7 +285,7 @@ Example:
291285
292286
print('json' in sys.modules) # True - now loaded
293287
294-
A module may contain a ``__lazy_modules__`` attribute, which is a sequence of
288+
A module may contain a :data:`!__lazy_modules__` attribute, which is a sequence of
295289
fully qualified module names (strings) to make *potentially lazy* (as if the
296290
``lazy`` keyword was used). This attribute is checked on each ``import``
297291
statement to determine whether the import should be made *potentially lazy*.
@@ -434,7 +428,7 @@ Example using ``__dict__`` from external code:
434428
print('json' in sys.modules) # True - reified by __dict__ access
435429
print(type(d['json'])) # <class 'module'>
436430
437-
However, calling ``globals()`` does **not** trigger reification it returns
431+
However, calling ``globals()`` does **not** trigger reification -- it returns
438432
the module's dictionary, and accessing lazy objects through that dictionary
439433
still returns lazy proxy objects that need to be manually reified upon use.
440434
A lazy object can be resolved explicitly by calling the ``get`` method.
@@ -461,8 +455,11 @@ Example using ``globals()``:
461455
print('json' in sys.modules) # True - now loaded
462456
463457
464-
Implementation
465-
==============
458+
Reference Implementation
459+
========================
460+
461+
A reference implementation is available at:
462+
https://github.com/LazyImportsCabal/cpython/tree/lazy
466463

467464
Bytecode and adaptive specialization
468465
-------------------------------------
@@ -638,7 +635,7 @@ Backwards Compatibility
638635
=======================
639636

640637
Lazy imports are **opt-in**. Existing programs continue to run unchanged unless
641-
a project explicitly enables laziness (via ``lazy`` syntax, ``__lazy_modules__``,
638+
a project explicitly enables laziness (via ``lazy`` syntax, :data:`!__lazy_modules__`,
642639
or an interpreter-wide switch).
643640

644641
Unchanged semantics
@@ -848,7 +845,7 @@ to a lazy object that resolves to ``foo.bar`` on first use.
848845
**Q: Does** ``lazy from module import Class`` **load the entire module or just the class?**
849846

850847
A: It loads the **entire module**, not just the class. This is because Python's
851-
import system always executes the complete module filethere's no mechanism to
848+
import system always executes the complete module file -- there's no mechanism to
852849
execute only part of a ``.py`` file. When you first access ``Class``, Python:
853850

854851
1. Loads and executes the entire ``module.py`` file
@@ -878,7 +875,7 @@ statement.
878875
# (and UnusedClass gets defined too)
879876
880877
**Key point**: Lazy imports defer *when* a module loads, not *what* gets loaded.
881-
You cannot selectively load only parts of a modulePython's import system doesn't
878+
You cannot selectively load only parts of a module -- Python's import system doesn't
882879
support partial module execution.
883880

884881
**Q: What about type annotations and** ``TYPE_CHECKING`` **imports?**
@@ -930,7 +927,7 @@ A: Migration is incremental:
930927
1. Identify slow-loading modules using profiling tools
931928
2. Add ``lazy`` keyword to imports that aren't needed immediately
932929
3. Test that side-effect timing changes don't break functionality
933-
4. Use ``__lazy_modules__`` for compatibility with older Python versions
930+
4. Use :data:`!__lazy_modules__` for compatibility with older Python versions
934931

935932
**Q: What about star imports** (``from module import *``)?
936933

@@ -959,7 +956,7 @@ module. Individual lazy objects can be resolved by calling their ``get()`` metho
959956
**Q: What's the difference between** ``globals()`` **and** ``mod.__dict__`` **for lazy imports?**
960957

961958
A: Calling ``globals()`` returns the module's dictionary without reifying lazy
962-
imports you'll see lazy proxy objects when accessing them through the returned
959+
imports -- you'll see lazy proxy objects when accessing them through the returned
963960
dictionary. However, accessing ``mod.__dict__`` from external code reifies all lazy
964961
imports in that module first. This design ensures:
965962

@@ -1037,7 +1034,7 @@ with ``lazy``.
10371034

10381035
**Q: What about forwards compatibility with older Python versions?**
10391036

1040-
A: Use the ``__lazy_modules__`` global for compatibility:
1037+
A: Use the :data:`!__lazy_modules__` global for compatibility:
10411038

10421039
.. code-block:: python
10431040
@@ -1046,15 +1043,15 @@ A: Use the ``__lazy_modules__`` global for compatibility:
10461043
import expensive_module
10471044
from expensive_module_2 import MyClass
10481045
1049-
The ``__lazy_modules__`` attribute is a list of module name strings. When an import
1046+
The :data:`!__lazy_modules__` attribute is a list of module name strings. When an import
10501047
statement is executed, Python checks if the module name being imported appears in
1051-
``__lazy_modules__``. If it does, the import is treated as if it had the ``lazy``
1048+
:data:`!__lazy_modules__`. If it does, the import is treated as if it had the ``lazy``
10521049
keyword (becoming *potentially lazy*). On Python versions before 3.15 that don't
1053-
support lazy imports, the ``__lazy_modules__`` attribute is simply ignored and
1050+
support lazy imports, the :data:`!__lazy_modules__` attribute is simply ignored and
10541051
imports proceed eagerly as normal.
10551052

10561053
This provides a migration path until you can rely on the ``lazy`` keyword. For
1057-
maximum predictability, it's recommended to define ``__lazy_modules__`` once,
1054+
maximum predictability, it's recommended to define :data:`!__lazy_modules__` once,
10581055
before any imports. But as it is checked on each import, it can be modified between
10591056
``import`` statements.
10601057

@@ -1120,7 +1117,7 @@ accesses the other during import time, you'll still get an error.
11201117
def get_by_user(username):
11211118
return f"Posts by {username}"
11221119
1123-
This works because neither module accesses the other at module levelthe access
1120+
This works because neither module accesses the other at module level -- the access
11241121
happens later when ``get_posts()`` is called.
11251122

11261123
**Example that fails** (access during import):
@@ -1204,12 +1201,6 @@ A: A lazily imported module does **not** appear in ``sys.modules`` until it's re
12041201
12051202
A: Not "why"... memorize! :)
12061203

1207-
Reference Implementation
1208-
========================
1209-
1210-
A reference implementation is available at:
1211-
https://github.com/LazyImportsCabal/cpython/tree/lazy
1212-
12131204
Alternate Implementation Ideas
12141205
==============================
12151206

@@ -1218,15 +1209,15 @@ of this PEP. While the current proposal represents what we believe to be the bes
12181209
of simplicity, performance, and maintainability, these alternatives offer different
12191210
trade-offs that may be valuable for implementers to consider or for future refinements.
12201211

1221-
Leveraging a Subclass of Dict
1212+
Leveraging a subclass of dict
12221213
-----------------------------
12231214

12241215
Instead of updating the internal dict object to directly add the fields needed to support lazy imports,
12251216
we could create a subclass of the dict object to be used specifically for Lazy Import enablement. This
12261217
would still be a leaky abstraction though - methods can be called directly such as ``dict.__getitem__``
12271218
and it would impact the performance of globals lookup in the interpreter.
12281219

1229-
Alternate Keyword Names
1220+
Alternate keyword names
12301221
-----------------------
12311222

12321223
For this PEP, we decided to propose ``lazy`` for the explicit keyword as it felt the most familar to those
@@ -1237,7 +1228,7 @@ options to support explicit lazy imports. The most compelling alternates were ``
12371228
Rejected Ideas
12381229
==============
12391230

1240-
Modification of the Dict Object
1231+
Modification of the dict object
12411232
-------------------------------
12421233

12431234
The initial PEP for lazy imports (PEP 690) relied heavily on the modification of the internal dict
@@ -1253,29 +1244,29 @@ Adding any kind of hook or special behavior to dicts to support lazy imports wou
12531244
1. Prevent critical interpreter optimizations including future JIT compilation
12541245
2. Add complexity to a data structure that must remain simple and fast
12551246
3. Affect every part of Python, not just import behavior
1256-
4. Violate separation of concernsthe hash table shouldn't know about the import system
1247+
4. Violate separation of concerns -- the hash table shouldn't know about the import system
12571248

12581249
Past decisions that violated this principle of keeping core abstractions clean have caused
12591250
significant pain in the CPython ecosystem, making optimization difficult and introducing
12601251
subtle bugs.
12611252

1262-
Placing the ``lazy`` Keyword in the Middle of From Imports
1253+
Placing the ``lazy`` keyword in the middle of from imports
12631254
----------------------------------------------------------
12641255

12651256
While we found ``from foo lazy import bar`` to be a really intuitive placement for the new explicit syntax,
12661257
we quickly learned that placing the ``lazy`` keyword here is already syntactically allowed in Python. This
12671258
is because ``from . lazy import bar`` is legal syntax (because whitespace
12681259
does not matter.)
12691260

1270-
Placing the ``lazy`` Keyword at the End of Import Statements
1261+
Placing the ``lazy`` keyword at the end of import statements
12711262
------------------------------------------------------------
12721263

12731264
We discussed appending lazy to the end of import statements like such ``import foo lazy`` or
12741265
``from foo import bar, baz lazy`` but ultimately decided that this approach provided less clarity.
12751266
For example, if multiple modules are imported in a single statement, it is unclear if the lazy binding
12761267
applies to all of the imported objects or just a subset of the items.
12771268

1278-
Returning a Proxy Dict from ``globals()``
1269+
Returning a proxy dict from ``globals()``
12791270
------------------------------------------
12801271

12811272
An alternative to reifying on ``globals()`` or exposing lazy objects would be to
@@ -1308,13 +1299,13 @@ for several reasons:
13081299

13091300
**The key distinction**: Adding a lazy import and calling ``globals()`` is the
13101301
module author's concern and under their control. However, accessing ``mod.__dict__``
1311-
from external code is a different scenario it crosses module boundaries and affects
1302+
from external code is a different scenario -- it crosses module boundaries and affects
13121303
someone else's code. Therefore, ``mod.__dict__`` access reifies all lazy imports to
13131304
ensure external code sees fully realized modules, while ``globals()`` preserves lazy
13141305
objects for the module's own introspection needs.
13151306

13161307
**Technical challenges**: It is impossible to safely reify on-demand when ``globals()``
1317-
is called because we cannot return a proxy dictionary this would break common usages
1308+
is called because we cannot return a proxy dictionary -- this would break common usages
13181309
like passing the result to ``exec()`` or other built-ins that expect a real dictionary.
13191310
The only alternative would be to eagerly reify all lazy imports whenever ``globals()``
13201311
is called, but this behavior would be surprising and potentially expensive.
@@ -1339,7 +1330,7 @@ Note that three options were considered:
13391330
We chose the third option because it properly delineates responsibility: if you add lazy imports
13401331
to your module and call ``globals()``, you're responsible for handling the lazy objects.
13411332
But external code accessing your module's ``__dict__`` shouldn't need to know about your
1342-
lazy importsit gets fully resolved modules.
1333+
lazy imports -- it gets fully resolved modules.
13431334

13441335
Acknowledgements
13451336
================

0 commit comments

Comments
 (0)