change recommended semantics of generator expressions, add alternatives to rejected ideas

Author: adqm

peps/pep-0798.rst

@@ -209,29 +209,36 @@ respectively::
     for x in dicts:
         new_dict.update(expr)
 
+.. _pep798-genexpsemantics:
 
 Semantics: Generator Expressions
 --------------------------------
 
-A generator expression ``(*expr for x in it)`` forms a generator producing
-values from the concatenation of the iterables given by the expressions.
-Specifically, the behavior is defined to be equivalent to the following
-generator::
+Generator expressions using the unpacking syntax should form new generators
+producing values from the concatenation of the iterables given by the
+expressions.  Specifically, the behavior is defined to be equivalent to the
+following (though without defining for referencing the looping variable
+``i``)::
 
+    # equivalent to generator = (*expr for x in it)
     def generator():
         for x in it:
-            yield from expr
-
-Since ``yield from`` is not allowed inside of async generators (see the section
-of :pep:`525` on Asynchronous ``yield from``), the equivalent for ``(*expr
-async for x in ait())`` is more like the following (though of course this new
-form should not define or reference the looping variable ``i``)::
+            for i in expr:
+                yield i
 
+    # equivalent to generator = (*expr for x in ait())
     async def generator():
         async for x in ait():
             for i in expr:
                 yield i
 
+
+The specifics of these semantics should be revisited in the future,
+particularly if async generators receive support for ``yield from`` (in which
+case both forms may wish to be changed to make use of ``yield from`` instead of
+an explicit loop).  See :ref:`pep798-alternativegenexpsemantics` for more
+discussion.
+
 Interaction with Assignment Expressions
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -249,7 +256,8 @@ form, ``y`` will be bound in the containing scope instead of locally::
 
     def generator():
         for i in (0, 2, 4):
-            yield from (y := [i, i+1])
+            for j in (y := [i, i+1]):
+                yield j
 
 In this example, the subexpression ``(y := [i, i+1])`` is evaluated exactly
 three times before the generator is exhausted: just after assigning ``i`` in
@@ -577,7 +585,7 @@ expressions that involve unpacking::
             yield expr
     g = generator()
 
-    # equivalent to g = (*expr for x in it)
+    # roughly equivalent to g = (*expr for x in it)
     def generator():
         for x in it:
             yield from expr
@@ -705,6 +713,65 @@ PEP.  As such, these forms should continue to raise a ``SyntaxError``, but with
 a new error message as described above, though it should not be ruled out as a
 consideration for future proposals.
 
+.. _pep798-alternativegenexpsemantics:
+
+Alternative Generator Expression Semantics
+------------------------------------------
+
+Another point of discussion centered around the semantics of unpacking in
+generator expressions, particularly the relationship between the semantics of
+synchronous and asynchronous generator expressions given that the latter do not
+support ``yield from`` (see the section of :pep:`525` on Asynchronous ``yield
+from``).
+
+Several reasonable options were considered, none of which was a clear winner in
+a `poll in the Discourse thread
+<https://discuss.python.org/t/pep-798-unpacking-in-comprehensions/99435/33>`__:
+
+1. Using ``yield from`` for unpacking in synchronous generator expressions but
+   not in asynchronous generator expressions (as proposed in the original draft
+   of this PEP).
+
+   This strategy would have allowed unpacking in generator expressions to
+   closely mimic a popular way of writing generators that perform this
+   operation (using ``yield from``), but it would also have created an
+   asymmetry between synchronous and asynchronous versions, and also between
+   this new syntax and ``itertools.chain`` and the double-loop version.
+
+2. Using ``yield from`` for unpacking in synchronous generator expressions and
+   mimicking the behavior of ``yield from`` for unpacking in async generator
+   expressions.
+
+   This strategy would also make unpacking in synchronous and asynchronous
+   generators behave similarly, but it would also be more complex, enough so
+   that the cost may not be worth the benefit, particularly in the absence of a
+   compelling practical use case for delegating to subgenerators during
+   unpacking.
+
+3. Disallowing unpacking in asynchronous generator expressions until they
+   support ``yield from``.
+
+   This strategy could possibly reduce friction if asynchronous generator
+   expressions do gain support for ``yield from`` in the future, but in the
+   meantime, it would result in an even bigger discrepancy between synchronous
+   and asynchronous generator expressions than option 1.
+
+4. Disallowing unpacking in all generator expressions.
+
+   This would retain symmetry between the two cases, but with the downside of
+   losing a very expressive form.
+
+Each of these options (including the one presented in this PEP) has its
+benefits and drawbacks, with no option being clearly superior on all fronts;
+but the semantics proposed in :ref:`pep798-genexpsemantics` represent a
+reasonable compromise.
+
+As suggested above, this decision should be revisited in the event that
+asynchronous generators receive support for ``yield from`` in the future,
+in which case the ability to delegate to subgenerators during unpacking
+could be added without significant cost.
+
+
 Concerns and Disadvantages
 ==========================