From 5e5e3886784c0884ae99ca9b3dc1799c3cc5f18f Mon Sep 17 00:00:00 2001 From: Viicos <65306057+Viicos@users.noreply.github.com> Date: Sat, 30 Aug 2025 23:01:32 +0200 Subject: [PATCH] Inline type expressions and inline typed dictionaries --- peps/pep-9999.rst | 456 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 456 insertions(+) create mode 100644 peps/pep-9999.rst diff --git a/peps/pep-9999.rst b/peps/pep-9999.rst new file mode 100644 index 00000000000..101d8be0e8c --- /dev/null +++ b/peps/pep-9999.rst @@ -0,0 +1,456 @@ +PEP: 9999 +Title: Inline type expressions and inline typed dictionaries +Author: Victorien Plot +Created: 30-Aug<-2025 +Status: Draft +Type: Standards Track + +Abstract +======== + +:pep:`589` defines a :ref:`class-based ` +and a :ref:`functional syntax ` to create +typed dictionaries. In both scenarios, it requires defining a class or +assigning to a value. In some situations, this can add unnecessary +boilerplate, especially if the typed dictionary is only used once. + +It is also currently not possible to dynamically create new typed dictionary classes +from existing ones, to change the value type and/or associated +:term:`type qualifiers ` of keys. + +This PEP proposes a new syntax in the language, to create inline typed dictionaries, +that can be constructed using a comprehension syntax:: + + type Movie = <{'name': str, 'location': <{'city': str}>}> + + type PartialMovie = <{K: NotRequired[ValueOf[Movie, K]] for K in KeyOf[Movie]}> + +While introducing a new syntax specifically for type expressions (and more precisely +typed dictionaries) might be considered not persuasive enough, the use of the ``<`` +and ``>`` characters partially mitigates this concern. This PEP introduces the concept of +*inline type expressions*, *inline typed dictionaries* being a single variant of this expression. +In the future, this syntax could be extended e.g. to define *inline tuple expressions*:: + + type TupleInt = tuple[int, int] + + type TupleListInt = <(list[T] for T in TupleInt)> + +New :term:`special forms ` (namely ``KeyOf`` and ``ValueOf``) are introduced, +and used by the new inline syntax (although can be used separately as well). + +Specification +============= + +.. note:: + This section describes the specification of this PEP for the + Python type system. See the :ref:`runtime implementation ` + section below for changes on the Python runtime. + +Key specification +----------------- + +A key specification is a type that represents keys for a typed dictionary. The typing system +already has types that are valid key specifications:: + + # Represents the keys 'a' and 'b': + Literal['a', 'b'] + + # Represents the keys of an "open" typed dictionary + # (i.e. a typed dictionary allowing extra items): + str + +A new ``KeyOf`` :term:`typing:special form` is introduced to the typing system, which is a +valid key specification. It takes a single type argument that must be a :class:`~typing.TypedDict` +class, or a type variable :ref:`bound ` to a :class:`~typing.TypedDict` +class. + +* If the type argument is a :class:`~!typing.TypedDict` class (say ``TD``), the + :term:`equivalence ` of ``KeyOf[TD]`` to other types + depends on the closed specification of the typed dictionary:: + + class TD(TypedDict): + a: str + b: NotRequired[str] + + assert_type(KeyOf[TD], Literal['a', 'b']) + + + class TDOpen(TypedDict, extra_items=int): + a: str + + assert_type(KeyOf[TDOpen], str) + +* If the type argument is a type variable ``T`` (bound to :class:`~!typing.TypedDict`), + ``KeyOf[T]`` can be seen as a type variable bound to :class:`str` and is called + a "bound ``KeyOf`` specification". Parameterizing ``KeyOf[T]`` with a typed dictionary + class ``TD`` is equivalent to ``KeyOf[TD]``:: + + def random_key_of[T: TypedDict](d: T) -> KeyOf[T]: + return list(d.keys())[random.randrange(len(d))] + + class TD(TypedDict): + a: str + b: NotRequired[str] + + d: TD = {'a': 'value'} + + assert_type(random_key_of(d), Literal['a', 'b']) + + +Operations on key specifications +'''''''''''''''''''''''''''''''' + +Two type-level operations can be applied on key specifications: + +* Removing keys from a key specification, using the :meth:`- ` operator:: + + assert_type(Literal['a', 'b'] - Literal['b'], Literal['a']) + assert_type(Literal['a', 'b'] - Literal['c'], Literal['a', 'b']) + + class TD(TypedDict): + a: str + b: NotRequired[str] + + assert_type(KeyOf[TD] - Literal['b'], Literal['a']) + + A key specification that results in an empty set of keys is equivalent to :data:`~typing.Never`:: + + assert_type(Literal['a'] - Literal['a'], Never) + +* Adding keys to a key specification, using the :meth:`+ ` operator:: + + assert_type(Literal['a'] + Literal['b'], Literal['a', 'b']) + +When applying such operations on bound KeyOf specifications, the evaluation of the type +is deferred until it is parameterized:: + + def keys_minus_a[T: TypedDict](d: T) -> KeyOf[T] - Literal['a']: + ... + + class TD(TypedDict): + a: str + b: NotRequired[str] + + d: TD = {'a': 'value'} + + assert_type(keys_minus_a(d), Literal['b']) + +Key specification views +''''''''''''''''''''''' + +A key specification view represents a single key in a key specification. +It can appear only in a specific context: +:ref:`inline typed dictionaries with the comprehension syntax `, +described below. + +The ``ValueOf`` special form +---------------------------- + +A new ``ValueOf`` :term:`typing:special form` is introduced to the typing system. It must +be parameterized with two parameters: a :class:`~typing.TypedDict` class or a type variable +:ref:`bound ` to a :class:`~typing.TypedDict` class, and a key specification +view. + +``ValueOf[..., ...]`` represents the value type of a typed dictionary item. Note that +:term:`type qualifiers ` *aren't* represented by this special form +(these are only carried by the key specification view, as described in the +:ref:`comprehension syntax section ` below). + +.. + It is expressed as a type variable :ref:`bound ` to a key specification. + TODO expand on why this isn't true. It could be intuitively, but a type variable T + bound to a key specification (i.e. T: Literal['a', 'b', 'c']) can be parameterized + with Literal['a', 'b'], so this isn't a single key. + Also if bound to a bound KeyOf specification, we arrive in HKT territory, or at least + in a limited version of HKT that allows generic bounds. + +Inline type expressions declarations +------------------------------------ + +Inline type expressions are enclosed by the 'Less-Than Sign' (``<``, U+003C) and +'Greater-Than Sign' (``>``, U+003E) characters, and must contain an inner +expression:: + + def func(arg: <...>): + pass + +Inline type expressions are a new form of :ref:`type expression `. + + +Inline typed dictionaries +------------------------- + +An inline typed dictionary is a variant of an inline type expression. It is defined using +a new display syntax, similar to the existing :ref:`dictionary display `. +The 'Left Curly Bracket' (``{``, U+007B) and 'Right Curly Bracket' (``}``, U+007D) +characters are used, respectively after the ``<`` and before the ``>`` characters:: + + def func(arg: <{...}>): + pass + +Two different syntaxes can be used, described in the following sections. + +.. _pep-9999-simple-display-syntax: + +Simple display syntax +''''''''''''''''''''' + +This syntax follows the same semantics as the +:ref:`functional syntax ` (the keys are +strings representing the field names, and values are valid +:ref:`annotation expressions `), and allows +defining typed dictionaries "statically":: + + type Movie = <{'name': str, 'location': <{'city': str}>}> + +Although it is not possible to specify class arguments such as ``total`` +(the :ref:`comprehension syntax ` aims to solve that), +any :term:`typing:type qualifier` can be used for indiviual fields:: + + type Movie = <{'name': NotRequired[str], 'year': ReadOnly[int]}> + + +.. _pep-9999-comprehension-syntax: + +Comprehension syntax +'''''''''''''''''''' + +The comprehension syntax allows creating typed dictionary types dynamically. It is inspired +from the existing :term:`dictionary comprehension` syntax, with some simplifications. The +general syntax is as follows:: + + type ComprTD = <{K: _type_expr_ for K in _key_spec_}> + +Conceptually, the comprehension syntax enables the possibility to express a new typed dictionary type +by mapping each key from a key specification to a specific value type, while preserving (and +potentially altering) the qualifiers of such key. + +By iterating [#iter-for-clause]_ over the key specification using the ``for`` clause, a key specification +view is created (in the given example, ``K`` is a key specification view). +It carries the following information about a key: + +* The key name. +* The associated value type. +* The associated :term:`type qualifiers `. + +These three informations are mapped to the newly created typed dictionary type, possibly with some +modifications: + +* The key name *cannot* be changed (e.g. it isn't possible to add a string suffix). In the + general syntax example, ``K`` must be specified as is in the first expression on the left side + of the colon. +* The associated value type *can* be changed. Any valid :term:`typing:annotation expression` can + be used (e.g. ``int``, ``Annotated[str, ...]``). If the key specification iterated over + is a ``KeyOf`` special form (and as such, brings a :class:`~typing.TypedDict` class or a + type variable ``T`` bound to :class:`~!typing.TypedDict` in scope), the original value + type can be accessed using the ``ValueOf`` special form. + +.. + TODO refer to newly added terms in https://github.com/python/typing/pull/2072: + +* The associated :term:`type qualifiers ` are carried over, and + *can* be overridden for :data:`~typing.Required` and :data:`~typing.NotRequired` + (:data:`~typing.ReadOnly` is always carried over), by wrapping the associated + value type inside the desired type qualifiers. + + +Here are some examples demonstrating these rules: + +* Standalone type: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + type Standalone = <{K: int for K in Literal['a', 'b']}> + * - Class equivalent + - .. code-block:: python + + class Standalone(TypedDict): + a: int + b: int + +* Invalid modifications on keys: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + type InvalidKeysAltering = <{K + '_suffix': int for K in Literal['a', 'b']}> + * - Class equivalent + - N/A + * - Notes + - Must raise a type checker error + +* Changing the value type to ``str``: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + class TD(TypedDict): + a: int + + type TDAsStr = <{K: str for K in KeyOf[TD]}> + * - Class equivalent + - .. code-block:: python + + class TDAsStr(TypedDict): + a: str + +* Changing the value type to ``str``, making the key not required: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + class TD(TypedDict): + a: int + + type TDAsNotRequiredStr = <{K: NotRequired[str] for K in KeyOf[TD]}> + * - Class equivalent + - .. code-block:: python + + class TDAsNotRequiredStr(TypedDict): + a: NotRequired[str] + * - Notes + - The :data:`~typing.NotRequired` type qualifier overrides the original ones + (in this case, we can assume :data:`~typing.Required` is an implicit qualifier on ``a``). + +* Making all keys not required, keeping the value type: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + class TD(TypedDict): + a: int + + type TDAsNotRequired = <{K: NotRequired[ValueOf[TD, K]] for K in KeyOf[TD]}> + * - Class equivalent + - .. code-block:: python + + class TDAsNotRequired(TypedDict): + a: NotRequired[int] + * - Notes + - The :data:`~typing.NotRequired` type qualifier overrides the original ones + (in this case, we can assume :data:`~typing.Required` is an implicit qualifier on ``a``). + +* Making all keys read only, wrapping the value type inside :class:`list`: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + class TD(TypedDict): + a: NotRequired[int] + + type TDAsReadOnlyList = <{K: ReadOnly[list[ValueOf[TD, K]]] for K in KeyOf[TD]}> + * - Class equivalent + - .. code-block:: python + + class TDAsReadOnlyList(TypedDict): + a: ReadOnly[NotRequired[list[int]]] + * - Notes + - Notice that :data:`~typing.NotRequired` is carried over, even if ``ValueOf[TD, K]`` is mapped in a + "nested" way. + +* Using a type alias to make every key not required: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + type Partial[T: TypedDict] = <{K: NotRequired[ValueOf[TD, K]] for K in KeyOf[T]}> + * - Class equivalent + - Not expressible + +* Using a type alias to select only certain keys: + + .. list-table:: + :widths: 25 75 + + * - Comprehension syntax + - .. code-block:: python + + type Pick[T: TypedDict, K: KeyOf[T]] = <{P: ValueOf[T, P] for P in KeyOf[T] - K}> + * - Class equivalent + - Not expressible + +While similar to the existing :term:`dictionary comprehension` syntax, this syntax is defined +separately, and the following differences can be found: + +* Only a single ``for`` clause can be used. It must iterate over a *key specification*, and + the key specification view variable used for the iteration must be used as the key (the first + expression on the left side of the colon). +* Unlike the dictionary comprehension syntax, it is not possible to use an ``if`` clause. + +.. _pep-9999-typing-spec-changes: + +Typing specification changes +---------------------------- + +The :external+typing:token:`~expression-grammar:type_expression` production will +be updated to include the ``KeyOf`` and ``ValueOf`` :term:`special forms `, +and the inline syntax: + +.. productionlist:: inline-expressions-grammar + new-type_expression: `~expression-grammar:type_expression` + : | '[' name ']' + : (where name refers to an in-scope TypedDict + : or type variable bound to a TypedDict) + : | '[' name ',' view ']' + : (where name refers to an in-scope TypedDict + : or a type variable bound to a TypedDict, + : and view refers to a key specification view) + : | `inline_type_expression` + inline_type_expression: '<' `inline_typed_dict` '>' + inline_typed_dict: '{' (string ':' `~expression-grammar:annotation_expression` ',')* '}' + : (where string is any string literal) + : | '{' K ':' `~expression-grammar:annotation_expression` 'for' 'K' 'in' `~expression-grammar:type_expression` '}' + : (the `~expression-grammar:type_expression` on which the + : 'for' clause is applied must be a key specification) + + +.. _pep-9999-runtime-impl: + +Runtime implementation +====================== + +Grammar changes +--------------- + +Inline type expressions are defined as an :ref:`atom ` expression, +more specifically as an :token:`~python-grammar:enclosure`: + +.. productionlist:: inline-type-expr-grammar + new-enclosure: `~python-grammar:enclosure` | `inline_type_expr_display` + inline_type_expr_display: "<" "..." ">" + +The grammar will define how the inner expression will be parsed (currently denoted +as ``"..."``) [#inner-expr]_, and can produce different AST nodes. + +.. rubric:: Footnotes + +.. [#iter-for-clause] Note that this *isn't* a real :keyword:`for` statement. + at runtime, the key specification isn't really iterated over, but the syntac + conceptually expresses the same logic. + +.. [#inner-expr] Technically, the inner expression isn't specified in the Python + grammar as being a standalone :term:`python:expression`. Only a specific set + of syntaxes are allowed inside the enclosing characters, and some of these syntaxes + are described in this PEP. In the future, new type expressions can be added.