PEP 810: Update decisions on with blocks and __dict__ reification

This commit updates PEP 810 based on team discussion and feedback:

1. Allow lazy imports inside with blocks
   - Removed with blocks from syntax restrictions
   - Only try blocks, functions, classes, and import * are prohibited
   - Global flag treats with blocks consistently (makes imports lazy)
   - Added rejected ideas section explaining why restriction was considered

2. __dict__ does not automatically reify lazy imports
   - Both globals() and __dict__ return raw dictionary without reification
   - Provides clean symmetry and predictable behavior
   - Low-level introspection APIs should not trigger side effects
   - Based on real-world experience with stdlib implementation
   - Updated reification section, FAQ, and rejected ideas

The explicit lazy import syntax is sufficient for developers
to understand what they're doing (consenting adults model). Linters can
catch genuinely problematic cases. This creates simpler, more consistent
rules between explicit and implicit (global flag) laziness.

Author: pablogsal

peps/pep-0810.rst

@@ -246,7 +246,7 @@ Syntax restrictions
 ~~~~~~~~~~~~~~~~~~~
 
 The soft keyword is only allowed at the global (module) level, **not** inside
-functions, class bodies, with ``try``/``with`` blocks, or ``import *``. Import
+functions, class bodies, ``try`` blocks, or ``import *``. Import
 statements that use the soft keyword are *potentially lazy*. Imports that
 can't be lazy are unaffected by the global lazy imports flag, and instead are
 always eager. Additionally, ``from __future__ import`` statements cannot be
@@ -270,10 +270,6 @@ Examples of syntax errors:
   except ImportError:
       pass
 
-  # SyntaxError: lazy import not allowed inside with blocks
-  with suppress(ImportError):
-      lazy import json
-
   # SyntaxError: lazy from ... import * is not allowed
   lazy from json import *
 
@@ -465,54 +461,37 @@ immediately resolve all lazy objects (e.g. ``lazy from`` statements) that
 referenced the module. It **only** resolves the lazy object being accessed.
 
 Accessing a lazy object (from a global variable or a module attribute) reifies
-the object. Accessing a module's ``__dict__`` reifies **all** lazy objects in
-that module. Calling ``dir()`` at the global scope will not reify the globals
-and calling ``dir(mod)`` will be special cased in ``mod.__dir__`` avoid
+the object. Calling ``dir()`` at the global scope will not reify the globals
+and calling ``dir(mod)`` will be special cased in ``mod.__dir__`` to avoid
 reification as well.
 
-Example using ``__dict__`` from external code:
-
-.. code-block:: python
-
-  # my_module.py
-  import sys
-  lazy import json
-
-  print('json' in sys.modules)  # False - still lazy
-
-  # main.py
-  import sys
-  import my_module
-
-  # Accessing __dict__ from external code DOES reify all lazy imports
-  d = my_module.__dict__
-
-  print('json' in sys.modules)  # True - reified by __dict__ access
-  print(type(d['json']))  # <class 'module'>
+However, calling ``globals()`` or accessing a module's ``__dict__`` does **not**
+trigger reification -- they return the module's dictionary, and accessing lazy
+objects through that dictionary still returns lazy proxy objects that need to
+be manually reified upon use. A lazy object can be resolved explicitly by
+calling the ``resolve`` method. Other, more indirect ways of accessing
+arbitrary globals (e.g. inspecting ``frame.f_globals``) also do **not** reify
+all the objects.
 
-However, calling ``globals()`` does **not** trigger reification -- it returns
-the module's dictionary, and accessing lazy objects through that dictionary
-still returns lazy proxy objects that need to be manually reified upon use. A
-lazy object can be resolved explicitly by calling the ``resolve`` method.
-Other, more indirect ways of accessing arbitrary globals (e.g. inspecting
-``frame.f_globals``) also do **not** reify all the objects.
-
-Example using ``globals()``:
+Example using ``globals()`` and ``__dict__``:
 
 .. code-block:: python
 
+  # my_module.py
   import sys
   lazy import json
 
   # Calling globals() does NOT trigger reification
   g = globals()
-
   print('json' in sys.modules)  # False - still lazy
   print(type(g['json']))  # <class 'LazyImport'>
 
+  # Accessing __dict__ also does NOT trigger reification
+  d = __dict__
+  print(type(d['json']))  # <class 'LazyImport'>
+
   # Explicitly reify using the resolve() method
   resolved = g['json'].resolve()
-
   print(type(resolved))  # <class 'module'>
   print('json' in sys.modules)  # True - now loaded
 
@@ -707,15 +686,15 @@ Where ``<mode>`` can be:
 
 * ``"normal"`` (or unset): Only explicitly marked lazy imports are lazy
 
-* ``"all"``: All module-level imports (except in ``try``  or ``with``
+* ``"all"``: All module-level imports (except in ``try``
   blocks and ``import *``) become *potentially lazy*
 
 * ``"none"``: No imports are lazy, even those explicitly marked with
   ``lazy`` keyword
 
 When the global flag is set to ``"all"``, all imports at the global level
-of all modules are *potentially lazy* **except** for those inside a ``try`` or
-``with`` block or any wild card (``from ... import *``) import.
+of all modules are *potentially lazy* **except** for those inside a ``try``
+block or any wild card (``from ... import *``) import.
 
 If the global lazy imports flag is set to ``"none"``, no *potentially
 lazy* import is ever imported lazily, the import filter is never called, and
@@ -1251,35 +1230,40 @@ either the lazy proxy or the final resolved object.
 Can I force reification of a lazy import without using it?
 ----------------------------------------------------------
 
-Yes, accessing a module's ``__dict__`` will reify all lazy objects in that
-module. Individual lazy objects can be resolved by calling their ``resolve()``
-method.
+Yes, individual lazy objects can be resolved by calling their ``resolve()``
+method. Note that accessing a module's ``__dict__`` or calling ``globals()``
+does **not** automatically reify lazy imports -- you'll see the lazy proxy
+objects themselves, which you can then manually resolve if needed.
 
 What's the difference between ``globals()`` and ``mod.__dict__`` for lazy imports?
 ----------------------------------------------------------------------------------
 
-Calling ``globals()`` returns the module's dictionary without reifying lazy
-imports -- you'll see lazy proxy objects when accessing them through the
-returned dictionary. However, accessing ``mod.__dict__`` from external code
-reifies all lazy imports in that module first. This design ensures:
+Both ``globals()`` and ``mod.__dict__`` return the module's dictionary without
+reifying lazy imports. Accessing lazy objects through either will yield lazy
+proxy objects. This provides a consistent low-level API for introspection:
 
 .. code-block:: python
 
   # In your module:
   lazy import json
 
   g = globals()
-  print(type(g['json']))  # <class 'LazyImport'> - your problem
+  print(type(g['json']))  # <class 'LazyImport'>
+
+  d = __dict__
+  print(type(d['json']))  # <class 'LazyImport'>
 
   # From external code:
   import sys
   mod = sys.modules['your_module']
   d = mod.__dict__
-  print(type(d['json']))  # <class 'module'> - reified for external access
+  print(type(d['json']))  # <class 'LazyImport'>
 
-This distinction means adding lazy imports and calling ``globals()`` is your
-responsibility to manage, while external code accessing ``mod.__dict__``
-always sees fully loaded modules.
+Both ``globals()`` and ``__dict__`` expose the raw namespace view without
+implicit side effects. This symmetry makes the behavior predictable: accessing
+the namespace dictionary never triggers imports. If you need to ensure an
+import is resolved, call the ``resolve()`` method explicitly or access the
+attribute directly (e.g., ``json.dumps``).
 
 Why not use ``importlib.util.LazyLoader`` instead?
 --------------------------------------------------
@@ -1664,6 +1648,59 @@ From the discussion on :pep:`690` it is clear that this is a fairly
 contentious idea, although perhaps once we have wide-spread use of lazy
 imports this can be reconsidered.
 
+Disallowing lazy imports inside ``with`` blocks
+------------------------------------------------
+
+An earlier version of this PEP proposed disallowing ``lazy import`` statements
+inside ``with`` blocks, similar to the restriction on ``try`` blocks. The
+concern was that certain context managers (like ``contextlib.suppress(ImportError)``)
+could suppress import errors in confusing ways when combined with lazy imports.
+
+However, this restriction was rejected because ``with`` statements have much
+broader semantics than ``try/except`` blocks. While ``try/except`` is explicitly
+about catching exceptions, ``with`` blocks are commonly used for resource
+management, temporary state changes, or scoping -- contexts where lazy imports
+work perfectly fine. The ``lazy import`` syntax is explicit enough that
+developers who write it inside a ``with`` block are making an intentional choice,
+aligning with Python's "consenting adults" philosophy. For genuinely problematic
+cases like ``with suppress(ImportError): lazy import foo``, static analysis
+tools and linters are better suited to catch these patterns than hard language
+restrictions.
+
+Additionally, forbidding explicit ``lazy import`` in ``with`` blocks would
+create complex rules for how the global lazy imports flag should behave,
+leading to confusing inconsistencies between explicit and implicit laziness. By
+allowing ``lazy import`` in ``with`` blocks, the rule is simple: the global
+flag affects all module-level imports except those in ``try`` blocks and wild
+card imports, matching exactly what's allowed with explicit syntax.
+
+Forcing eager imports in ``with`` blocks under the global flag
+---------------------------------------------------------------
+
+Another rejected idea was to make imports inside ``with`` blocks remain eager
+even when the global lazy imports flag is set to ``"all"``. The rationale was
+to be conservative: since ``with`` statements can affect how imports behave
+(e.g., by modifying ``sys.path`` or suppressing exceptions), forcing imports to
+remain eager could prevent subtle bugs. However, this would create inconsistent
+behavior where ``lazy import`` is allowed explicitly in ``with`` blocks, but
+normal imports remain eager when the global flag is enabled. This inconsistency
+between explicit and implicit laziness is confusing and hard to explain.
+
+The simpler, more consistent rule is that the global flag affects imports
+everywhere that explicit ``lazy import`` syntax is allowed. This avoids having
+three different sets of rules (explicit syntax, global flag behavior, and filter
+mechanism) and instead provides two: explicit syntax rules match what the global
+flag affects, and the filter mechanism provides escape hatches for edge cases.
+For users who need fine-grained control, the filter mechanism
+(``sys.set_lazy_imports_filter()``) already provides a way to exclude specific
+imports or patterns. Additionally, there's no inverse operation: if the global
+flag forces imports eager in ``with`` blocks but a user wants them lazy, there's
+no way to override it, creating an asymmetry.
+
+In summary: imports in ``with`` blocks behave consistently whether marked
+explicitly with ``lazy import`` or implicitly via the global flag, creating a
+simple rule that's easy to explain and reason about.
+
 Modification of the dict object
 -------------------------------
 
@@ -1868,55 +1905,53 @@ from a real dict in almost all cases, which is extremely difficult to achieve
 correctly. Any deviation from true dict behavior would be a source of subtle
 bugs.
 
-Reifying lazy imports when ``globals()`` is called
----------------------------------------------------
+Automatically reifying on ``__dict__`` or ``globals()`` access
+--------------------------------------------------------------
 
-Calling ``globals()`` returns the module's namespace dictionary without
-triggering reification of lazy imports. Accessing lazy objects through the
-returned dictionary yields the lazy proxy objects themselves. This is an
-intentional design decision for several reasons:
-
-**The key distinction**: Adding a lazy import and calling ``globals()`` is the
-module author's concern and under their control. However, accessing
-``mod.__dict__`` from external code is a different scenario -- it crosses
-module boundaries and affects someone else's code. Therefore, ``mod.__dict__``
-access reifies all lazy imports to ensure external code sees fully realized
-modules, while ``globals()`` preserves lazy objects for the module's own
-introspection needs.
-
-**Technical challenges**: It is impossible to safely reify on-demand when
-``globals()`` is called because we cannot return a proxy dictionary -- this
-would break common usages like passing the result to ``exec()`` or other
-built-ins that expect a real dictionary. The only alternative would be to
-eagerly reify all lazy imports whenever ``globals()`` is called, but this
-behavior would be surprising and potentially expensive.
-
-**Performance concerns**: It is impractical to cache whether a reification
-scan has been performed with just the globals dictionary reference, whereas
-module attribute access (the primary use case) can efficiently cache
-reification state in the module object itself.
-
-**Use case rationale**: The chosen design makes sense precisely because of
-this distinction: adding a lazy import and calling ``globals()`` is your
-problem to manage, while having lazy imports visible in ``mod.__dict__``
-becomes someone else's problem. By reifying on ``__dict__`` access but not on
-``globals()``, we ensure external code always sees fully loaded modules while
-giving module authors control over their own introspection.
-
-Note that three options were considered:
+Three options were considered for how ``globals()`` and ``mod.__dict__`` should
+behave with lazy imports:
 
 1. Calling ``globals()`` or ``mod.__dict__`` traverses and resolves all lazy
    objects before returning.
 2. Calling ``globals()`` or ``mod.__dict__`` returns the dictionary with lazy
-   objects present.
+   objects present (chosen).
 3. Calling ``globals()`` returns the dictionary with lazy objects, but
    ``mod.__dict__`` reifies everything.
 
-We chose the third option because it properly delineates responsibility: if
-you add lazy imports to your module and call ``globals()``, you're responsible
-for handling the lazy objects. But external code accessing your module's
-``__dict__`` shouldn't need to know about your lazy imports -- it gets fully
-resolved modules.
+We chose option 2: both ``globals()`` and ``__dict__`` return the raw
+namespace dictionary without triggering reification. This provides a clean,
+predictable model where low-level introspection APIs don't trigger side
+effects.
+
+**Reasons for this choice:**
+
+**Consistency and symmetry**: Having ``globals()`` and ``__dict__`` behave
+identically creates a simple mental model: both expose the raw namespace view.
+This symmetry is easy to explain and predict. The alternative (option 3) would
+create a confusing distinction where accessing the same dictionary through
+different paths yields different behavior.
+
+**Low-level APIs shouldn't trigger side effects**: Both ``globals()`` and
+``__dict__`` are low-level introspection APIs. Making them automatically trigger
+imports would be surprising and potentially expensive. Code that inspects
+namespaces generally shouldn't cause module loading as a side effect.
+
+**Real-world experience**: When implementing lazy imports in parts of the
+standard library (such as the traceback module), it quickly became clear that
+automatic reification on ``__dict__`` access was cumbersome and not the right
+thing to do. Introspection code that needs to reason about module contents
+shouldn't be forced into loading modules it's only examining.
+
+**Explicit is better than implicit**: If code needs lazy imports resolved, it
+can call ``resolve()`` explicitly or access the attributes directly. This puts
+the control in the hands of the code that actually needs the imports loaded.
+
+Option 1 (always reifying) was rejected because it would make ``globals()``
+and ``__dict__`` access surprisingly expensive and would make it impossible to
+introspect the lazy state of a module. Option 3 was initially considered to
+"protect" external code from seeing lazy objects, but real-world usage showed
+this created more problems than it solved, particularly for stdlib code that
+needs to introspect modules without triggering side effects.
 
 Acknowledgements
 ================