Skip to content

PEP 728: Improve specification #4166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
JelleZijlstra merged 4 commits into from
Dec 20, 2024
Merged

PEP 728: Improve specification #4166

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
0c40686
PEP 728: Improve specification
JelleZijlstra Dec 13, 2024
4803404
Remove unnecessary quote
JelleZijlstra Dec 14, 2024
e9d870a
Merge remote-tracking branch 'upstream/main' into pep728-runtime
JelleZijlstra Dec 20, 2024
1b4d267
Feedback from Rebecca
JelleZijlstra Dec 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
129 changes: 78 additions & 51 deletions peps/pep-0728.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ Another possible use case for this is a sound way to
class Movie(TypedDict):
name: str
director: str

class Book(TypedDict):
name: str
author: str
Expand Down Expand Up @@ -194,12 +194,12 @@ to the ``extra_items`` argument. For example::

class Movie(TypedDict, extra_items=bool):
name: str

a: Movie = {"name": "Blade Runner", "novel_adaptation": True} # OK
b: Movie = {
"name": "Blade Runner",
"year": 1982, # Not OK. 'int' is not assignable to 'bool'
}
}

Here, ``extra_items=bool`` specifies that items other than ``'name'``
have a value type of ``bool`` and are non-required.
Expand All @@ -214,12 +214,12 @@ the ``extra_items`` argument::
def f(movie: Movie) -> None:
reveal_type(movie["name"]) # Revealed type is 'str'
reveal_type(movie["novel_adaptation"]) # Revealed type is 'bool'

``extra_items`` is inherited through subclassing::

class MovieBase(TypedDict, extra_items=int | None):
name: str

class Movie(MovieBase):
year: int

Expand All @@ -234,38 +234,46 @@ Here, ``'year'`` in ``a`` is an extra key defined on ``Movie`` whose value type
is ``int``. ``'other_extra_key'`` in ``b`` is another extra key whose value type
must be assignable to the value of ``extra_items`` defined on ``MovieBase``.

``extra_items`` is also supported with the functional syntax::

Movie = TypedDict("Movie", {"name": str}, extra_items=int | None)

The ``closed`` Class Parameter
------------------------------

When ``closed=True`` is set, no extra items are allowed. This is a shorthand for
When ``closed=True`` is set, no extra items are allowed. This is equivalent to
``extra_items=Never``, because there can't be a value type that is assignable to
:class:`~typing.Never`.
:class:`~typing.Never`. It is a runtime error to use the ``closed`` and
``extra_items`` parameters in the same TypedDict definition.

Similar to ``total``, only a literal ``True`` or ``False`` is supported as the
value of the ``closed`` argument; ``closed`` is ``False`` by default, which
preserves the previous TypedDict behavior.
value of the ``closed`` argument. Type checkers should reject any non-literal value.

Passing ``closed=False`` explicitly requests the default TypedDict behavior,
where arbitrary other keys may be present and subclasses may add arbitrary items.
It is a type checker error to pass ``closed=False`` if a superclass has
``closed=True`` or sets ``extra_items``.

The value of ``closed`` is not inherited through subclassing, but the
implicitly set ``extra_items=Never`` is. It should be an error to use the
default ``closed=False`` when subclassing a closed TypedDict type::
If ``closed`` is not provided, the behavior is inherited from the superclass.
If the superclass is TypedDict itself or the superclass does not have ``closed=True``
or the ``extra_items`` parameter, the previous TypedDict behavior is preserved:
arbitrary extra items are allowed. If the superclass has ``closed=True``, the
child class is also closed.

class BaseMovie(TypedDict, closed=True):
name: str

class MovieA(BaseMovie): # Not OK. An explicit 'closed=True' is required
class MovieA(BaseMovie): # OK, still closed
pass

class MovieB(BaseMovie, closed=True): # OK
class MovieB(BaseMovie, closed=True): # OK, but redundant
pass

Setting both ``closed`` and ``extra_items`` when defining a TypedDict type
should always be a runtime error::

class Person(TypedDict, closed=False, extra_items=bool): # Not OK. 'closed' and 'extra_items' are incompatible
name: str
class MovieC(BaseMovie, closed=False): # Type checker error
pass

As a consequence of ``closed=True`` being equivalent to ``extra_items=Never``.
The same rules that apply to ``extra_items=Never`` should also apply to
As a consequence of ``closed=True`` being equivalent to ``extra_items=Never``,
the same rules that apply to ``extra_items=Never`` also apply to
``closed=True``. It is possible to use ``closed=True`` when subclassing if the
``extra_items`` argument is a read-only type::

Expand All @@ -275,7 +283,7 @@ The same rules that apply to ``extra_items=Never`` should also apply to
class MovieClosed(Movie, closed=True): # OK
pass

class MovieNever(Movie, extra_items=Never): # Not OK. 'closed=True' is preferred
class MovieNever(Movie, extra_items=Never): # OK, but 'closed=True' is preferred
pass

This will be further discussed in
Expand All @@ -286,6 +294,10 @@ is assumed to allow non-required extra items of value type ``ReadOnly[object]``
during inheritance or assignability checks. This preserves the existing behavior
of TypedDict.

``closed`` is also supported with the functional syntax::

Movie = TypedDict("Movie", {"name": str}, closed=True)

Interaction with Totality
-------------------------

Expand Down Expand Up @@ -315,7 +327,7 @@ function parameters still apply::

class Movie(TypedDict, extra_items=int):
name: str

def f(**kwargs: Unpack[Movie]) -> None: ...

# Should be equivalent to:
Expand Down Expand Up @@ -356,7 +368,7 @@ unless it is declared to be ``ReadOnly`` in the superclass::

class Parent(TypedDict, extra_items=int | None):
pass

class Child(Parent, extra_items=int): # Not OK. Like any other TypedDict item, extra_items's type cannot be changed

Second, ``extra_items=T`` effectively defines the value type of any unnamed
Expand All @@ -378,20 +390,17 @@ added in a subclass, all of the following conditions should apply:

- The item's value type is :term:`typing:consistent` with ``T``

- If ``extra_items`` is not overriden, the subclass inherits it as-is.
- If ``extra_items`` is not overridden, the subclass inherits it as-is.

For example::

class MovieBase(TypedDict, extra_items=int | None):
name: str

class AdaptedMovie(MovieBase): # Not OK. 'bool' is not assignable to 'int | None'
adapted_from_novel: bool

class MovieRequiredYear(MovieBase): # Not OK. Required key 'year' is not known to 'Parent'

class MovieRequiredYear(MovieBase): # Not OK. Required key 'year' is not known to 'MovieBase'
year: int | None

class MovieNotRequiredYear(MovieBase): # Not OK. 'int | None' is not assignable to 'int'
class MovieNotRequiredYear(MovieBase): # Not OK. 'int | None' is not consistent with 'int'
year: NotRequired[int]

class MovieWithYear(MovieBase): # OK
Expand Down Expand Up @@ -478,7 +487,7 @@ checks::
class MovieDetails(TypedDict, extra_items=int | None):
name: str
year: NotRequired[int]

details: MovieDetails = {"name": "Kill Bill Vol. 1", "year": 2003}
movie: Movie = details # Not OK. While 'int' is assignable to 'int | None',
# 'int | None' is not assignable to 'int'
Expand All @@ -502,7 +511,7 @@ possible for an item to have a :term:`narrower <typing:narrow>` type than the

class Movie(TypedDict, extra_items=ReadOnly[str | int]):
name: str

class MovieDetails(TypedDict, extra_items=int):
name: str
year: NotRequired[int]
Expand All @@ -522,19 +531,19 @@ enforced::

class MovieExtraStr(TypedDict, extra_items=str):
name: str

extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007}
extra_str: MovieExtraStr = {"name": "No Country for Old Men", "description": ""}
extra_int = extra_str # Not OK. 'str' is not assignable to extra items type 'int'
extra_str = extra_int # Not OK. 'int' is not assignable to extra items type 'str'

A non-closed TypedDict type implicitly allows non-required extra keys of value
type ``ReadOnly[object]``. Applying the assignability rules between this type
and a closed TypedDict type is allowed::

class MovieNotClosed(TypedDict):
name: str

extra_int: MovieExtraInt = {"name": "No Country for Old Men", "year": 2007}
not_closed: MovieNotClosed = {"name": "No Country for Old Men"}
extra_int = not_closed # Not OK.
Expand Down Expand Up @@ -578,17 +587,13 @@ arguments of this type when constructed by calling the class object::
Interaction with Mapping[KT, VT]
--------------------------------

A TypedDict type can be assignable to ``Mapping[KT, VT]`` types other than
``Mapping[str, object]`` as long as all value types of the items on the
TypedDict type is :term:`typing:assignable` to ``VT``. This is an extension of this
A TypedDict type is :term:`typing:assignable` to a type of the form ``Mapping[str, VT]``
when all value types of the items in the TypedDict
are assignable to ``VT``. For the purpose of this rule, a
TypedDict that does not have ``extra_items=`` or ``closed=`` set is considered
to have an item with a value of type ``object``. This extends the current
assignability rule from the `typing spec
<https://typing.readthedocs.io/en/latest/spec/typeddict.html#assignability>`__:

* A TypedDict with all ``int`` values is not :term:`typing:assignable` to
``Mapping[str, int]``, since there may be additional non-``int`` values
not visible through the type, due to :term:`typing:structural`
assignability. These can be accessed using the ``values()`` and
``items()`` methods in ``Mapping``,
<https://typing.readthedocs.io/en/latest/spec/typeddict.html#assignability>`__.

For example::

Expand All @@ -598,6 +603,10 @@ For example::
extra_str: MovieExtraStr = {"name": "Blade Runner", "summary": ""}
str_mapping: Mapping[str, str] = extra_str # OK

class MovieExtraInt(TypedDict, extra_items=int):
name: str

extra_int: MovieExtraInt = {"name": "Blade Runner", "year": 1982}
int_mapping: Mapping[str, int] = extra_int # Not OK. 'int | str' is not assignable with 'int'
int_str_mapping: Mapping[str, int | str] = extra_int # OK

Expand All @@ -611,7 +620,7 @@ and ``items()`` on such TypedDict types::
Interaction with dict[KT, VT]
-----------------------------

Note that because the presence of ``extra_items`` on a closed TypedDict type
Because the presence of ``extra_items`` on a closed TypedDict type
prohibits additional required keys in its :term:`typing:structural`
:term:`typing:subtypes <subtype>`, we can determine if the TypedDict type and
its structural subtypes will ever have any required key during static analysis.
Expand All @@ -636,8 +645,8 @@ For example::
def f(x: IntDict) -> None:
v: dict[str, int] = x # OK
v.clear() # OK
not_required_num_dict: IntDictWithNum = {"num": 1, "bar": 2}

not_required_num_dict: IntDictWithNum = {"num": 1, "bar": 2}
regular_dict: dict[str, int] = not_required_num_dict # OK
f(not_required_num_dict) # OK

Expand All @@ -652,10 +661,28 @@ because such dict can be a subtype of dict::

class CustomDict(dict[str, int]):
pass

not_a_regular_dict: CustomDict = {"num": 1}
int_dict: IntDict = not_a_regular_dict # Not OK

Runtime behavior
----------------

At runtime, it is an error to pass both the ``closed`` and ``extra_items``
arguments in the same TypedDict definition, whether using the class syntax or
the functional syntax. For simplicity, the runtime does not check other invalid
combinations involving inheritance.

For introspection, the ``closed`` and ``extra_items`` arguments are mapped to
two new attributes on the resulting TypedDict object: ``__closed__`` and
``__extra_items__``. These attributes reflect exactly what was passed to the
TypedDict constructor, without considering superclasses.

If ``closed`` is not passed, the value of ``__closed__`` is None. If ``extra_items``
is not passed, the value of ``__extra_items__`` is the new sentinel object
``typing.NoExtraItems``. (It cannot be ``None``, because ``extra_items=None`` is a
valid definition that indicates all extra items must be ``None``.)

How to Teach This
=================

Expand All @@ -673,7 +700,7 @@ Because ``extra_items`` is an opt-in feature, no existing codebase will break
due to this change.

Note that ``closed`` and ``extra_items`` as keyword arguments do not collide
with othere keys when using something like
with other keys when using something like
``TD = TypedDict("TD", foo=str, bar=int)``, because this syntax has already
been removed in Python 3.13.

Expand Down
Loading