diff --git a/peps/pep-0750.rst b/peps/pep-0750.rst index 7ed1f858946..986ad9dc831 100644 --- a/peps/pep-0750.rst +++ b/peps/pep-0750.rst @@ -109,13 +109,13 @@ Template String Literals This PEP introduces a new string prefix, ``t``, to define template string literals. These literals resolve to a new type, ``Template``, found in the standard library -module ``<>``. +module :mod:`!string.templatelib`. The following code creates a ``Template`` instance: .. code-block:: python - from TBD import Template + from string.templatelib import Template template = t"This is a template string." assert isinstance(template, Template) @@ -134,7 +134,8 @@ see the `Raw Template Strings`_ section below for more information. The ``Template`` Type --------------------- -Template strings evaluate to an instance of a new immutable type, ``<>.Template``: +Template strings evaluate to an instance of a new immutable type, +:class:`!string.templatelib.Template`: .. code-block:: python @@ -192,23 +193,23 @@ The ``Interpolation`` Type -------------------------- The ``Interpolation`` type represents an expression inside a template string. -Like ``Template``, it is a new class found in the ``<>`` module: +Like ``Template``, it is a new class found in the :mod:`!string.templatelib` module: .. code-block:: python class Interpolation: value: object - expr: str - conv: Literal["a", "r", "s"] | None + expression: str + conversion: Literal["a", "r", "s"] | None format_spec: str - __match_args__ = ("value", "expr", "conv", "format_spec") + __match_args__ = ("value", "expression", "conversion", "format_spec") def __new__( cls, value: object, - expr: str, - conv: Literal["a", "r", "s"] | None = None, + expression: str, + conversion: Literal["a", "r", "s"] | None = None, format_spec: str = "", ): ... @@ -224,20 +225,21 @@ The ``value`` attribute is the evaluated result of the interpolation: template = t"Hello {name}" assert template.interpolations[0].value == "World" -The ``expr`` attribute is the *original text* of the interpolation: +The ``expression`` attribute is the *original text* of the interpolation: .. code-block:: python name = "World" template = t"Hello {name}" - assert template.interpolations[0].expr == "name" + assert template.interpolations[0].expression == "name" -We expect that the ``expr`` attribute will not be used in most template processing -code. It is provided for completeness and for use in debugging and introspection. -See both the `Common Patterns Seen in Processing Templates`_ section and the -`Examples`_ section for more information on how to process template strings. +We expect that the ``expression`` attribute will not be used in most template +processing code. It is provided for completeness and for use in debugging and +introspection. See both the `Common Patterns Seen in Processing Templates`_ +section and the `Examples`_ section for more information on how to process +template strings. -The ``conv`` attribute is the :ref:`optional conversion ` +The ``conversion`` attribute is the :ref:`optional conversion ` to be used, one of ``r``, ``s``, and ``a``, corresponding to ``repr()``, ``str()``, and ``ascii()`` conversions. As with f-strings, no other conversions are supported: @@ -246,9 +248,9 @@ are supported: name = "World" template = t"Hello {name!r}" - assert template.interpolations[0].conv == "r" + assert template.interpolations[0].conversion == "r" -If no conversion is provided, ``conv`` is ``None``. +If no conversion is provided, ``conversion`` is ``None``. The ``format_spec`` attribute is the :ref:`format specification `. As with f-strings, this is an arbitrary string that defines how to present the value: @@ -275,17 +277,18 @@ string (``""``). This matches the ``format_spec`` parameter of Python's :func:`python:format` built-in. Unlike f-strings, it is up to code that processes the template to determine how to -interpret the ``conv`` and ``format_spec`` attributes. +interpret the ``conversion`` and ``format_spec`` attributes. Such code is not required to use these attributes, but when present they should be respected, and to the extent possible match the behavior of f-strings. It would be surprising if, for example, a template string that uses ``{value:.2f}`` did not round the value to two decimal places when processed. -Convenience Accessors in ``Template`` -------------------------------------- +The ``Template.values`` Property +-------------------------------- -The ``Template.values`` property is equivalent to: +The ``Template.values`` property is a shortcut for accessing the ``value`` +attribute of each ``Interpolation`` in the template and is equivalent to: .. code-block:: python @@ -294,7 +297,14 @@ The ``Template.values`` property is equivalent to: return tuple(i.value for i in self.interpolations) -The ``Template.__iter__()`` method is equivalent to: +Iterating ``Template`` Contents +------------------------------- + +The ``Template.__iter__()`` method provides a simple way to access the full +contents of a template. It yields the string parts and interpolations in +the order they appear, with empty strings omitted. + +The ``__iter__()`` method is equivalent to: .. code-block:: python @@ -306,6 +316,46 @@ The ``Template.__iter__()`` method is equivalent to: yield i +The following examples show the ``__iter__()`` method in action: + +.. code-block:: python + + assert list(t"") == [] + + assert list(t"Hello") == ["Hello"] + + name = "World" + template = t"Hello {name}!" + contents = list(template) + assert len(contents) == 3 + assert contents[0] == "Hello " + assert contents[1].value == "World" + assert contents[1].expression == "name" + assert contents[2] == "!" + +Empty strings, which may be present in ``Template.strings``, are not included +in the output of the ``__iter__()`` method: + +.. code-block:: python + + first = "Eat" + second = "Red Leicester" + template = t"{first}{second}" + contents = list(template) + assert len(contents) == 2 + assert contents[0].value == "Eat" + assert contents[0].expression == "first" + assert contents[1].value == "Red Leicester" + assert contents[1].expression == "second" + + # However, the strings attribute contains empty strings: + assert template.strings == ("", "", "") + +Template processing code can choose to work with any combination of +``strings``, ``interpolations``, ``values``, and ``__iter__()`` based on +requirements and convenience. + + Processing Template Strings --------------------------- @@ -315,7 +365,7 @@ interpolations in uppercase: .. code-block:: python - from TBD import Template, Interpolation + from string.templatelib import Template, Interpolation def lower_upper(template: Template) -> str: """Render static parts lowercased and interpolations uppercased.""" @@ -404,7 +454,7 @@ The debug specifier, ``=``, is supported in template strings and behaves similar to how it behaves in f-strings, though due to limitations of the implementation there is a slight difference. -In particular, ``t'{expr=}'`` is treated as ``t'expr={expr!r}'``: +In particular, ``t'{value=}'`` is treated as ``t'value={value!r}'``: .. code-block:: python @@ -412,13 +462,13 @@ In particular, ``t'{expr=}'`` is treated as ``t'expr={expr!r}'``: template = t"Hello {name=}" assert template.strings[0] == "Hello name=" assert template.interpolations[0].value == "World" - assert template.interpolations[0].conv == "r" + assert template.interpolations[0].conversion == "r" -If a separate format string is also provided, ``t'{expr=:fmt}`` is treated instead as -``t'expr={expr!s:fmt}'``. +If a separate format string is also provided, ``t'{value=:fmt}`` is treated +instead as ``t'value={value!s:fmt}'``. -Whitespace is preserved in the debug specifier, so ``t'{expr = }'`` is treated as -``t'expr = {expr!r}'``. +Whitespace is preserved in the debug specifier, so ``t'{value = }'`` is +treated as ``t'value = {value!r}'``. Raw Template Strings @@ -477,6 +527,16 @@ The ``Template`` and ``Interpolation`` types both provide useful ``__repr__()`` implementations. +The :mod:`!string.templatelib` Module +------------------------------------- + +The :mod:`string` module will be converted into a package, with a new +``templatelib`` submodule containing the ``Template`` and ``Interpolation`` +types. Following the implementation of this PEP, this new module may be used +for related functions, such as :func:`!convert`, or potential future template +processing code, such as shell script helpers. + + Examples ======== @@ -506,14 +566,14 @@ specifiers like ``:.2f``. The full code is fairly simple: .. code-block:: python - from TBD import Template, Interpolation + from string.templatelib import Template, Interpolation - def convert(value: object, conv: Literal["a", "r", "s"] | None) -> object: - if conv == "a": + def convert(value: object, conversion: Literal["a", "r", "s"] | None) -> object: + if conversion == "a": return ascii(value) - elif conv == "r": + elif conversion == "r": return repr(value) - elif conv == "s": + elif conversion == "s": return str(value) return value @@ -523,8 +583,8 @@ specifiers like ``:.2f``. The full code is fairly simple: match item: case str() as s: parts.append(s) - case Interpolation(value, _, conv, format_spec): - value = convert(value, conv) + case Interpolation(value, _, conversion, format_spec): + value = convert(value, conversion) value = format(value, format_spec) parts.append(value) return "".join(parts) @@ -579,7 +639,7 @@ We can implement an improved version of ``StructuredMessage`` using template str .. code-block:: python import json - from TBD import Interpolation, Template + from string.templatelib import Interpolation, Template from typing import Mapping class TemplateMessage: @@ -594,7 +654,7 @@ We can implement an improved version of ``StructuredMessage`` using template str @property def values(self) -> Mapping[str, object]: return { - item.expr: item.value + item.expression: item.value for item in self.template if isinstance(item, Interpolation) } @@ -614,7 +674,8 @@ class. With template strings it is no longer necessary for developers to make sure that their format string and values dictionary are kept in sync; a single template string literal is all that is needed. The ``TemplateMessage`` implementation can automatically extract structured keys and values from -the ``Interpolation.expr`` and ``Interpolation.value`` attributes, respectively. +the ``Interpolation.expression`` and ``Interpolation.value`` attributes, +respectively. Approach 2: Custom Formatters @@ -636,7 +697,7 @@ and a ``ValuesFormatter`` for JSON output: import json from logging import Formatter, LogRecord - from TBD import Interpolation, Template + from string.templatelib import Interpolation, Template from typing import Any, Mapping @@ -655,7 +716,7 @@ and a ``ValuesFormatter`` for JSON output: class ValuesFormatter(Formatter): def values(self, template: Template) -> Mapping[str, Any]: return { - item.expr: item.value + item.expression: item.value for item in template if isinstance(item, Interpolation) } @@ -782,9 +843,10 @@ in terms of formal grammars. Developers will need to learn how to work with the ``strings`` and ``interpolation`` attributes of a ``Template`` instance and how to process interpolations in a context-sensitive fashion. More sophisticated grammars will likely require parsing to intermediate representations like an -AST. Great template processing code will handle format specifiers and conversions -when appropriate. Writing production-grade template processing code -- for -instance, to support HTML templates -- can be a large undertaking. +abstract syntax tree (AST). Great template processing code will handle format +specifiers and conversions when appropriate. Writing production-grade template +processing code -- for instance, to support HTML templates -- can be a large +undertaking. We expect that template strings will provide framework authors with a powerful new tool in their toolbox. While the functionality of template strings overlaps @@ -820,7 +882,7 @@ best practice for many template function implementations: .. code-block:: python - from TBD import Template, Interpolation + from string.templatelib import Template, Interpolation def process(template: Template) -> Any: for item in template: @@ -1192,10 +1254,10 @@ Earlier versions of this PEP proposed that the ``Template`` and ``Interpolation` types should have their own implementations of ``__eq__`` and ``__hash__``. ``Templates`` were considered equal if their ``strings`` and ``interpolations`` -were equal; ``Interpolations`` were considered equal if their ``value``, ``expr``, -``conv``, and ``format_spec`` were equal. Interpolation hashing was similar to -tuple hashing: an ``Interpolation`` was hashable if and only if its ``value`` -was hashable. +were equal; ``Interpolations`` were considered equal if their ``value``, +``expression``, ``conversion``, and ``format_spec`` were equal. Interpolation +hashing was similar to tuple hashing: an ``Interpolation`` was hashable if and +only if its ``value`` was hashable. This was rejected because ``Template.__hash__`` so defined was not useful as a cache key in template processing code; we were concerned that it would be @@ -1228,14 +1290,8 @@ The Final Home for ``Template`` and ``Interpolation`` Previous versions of this PEP proposed placing the ``Template`` and ``Interpolation`` types in: ``types``, ``collections``, ``collections.abc``, -and even in a new top-level module, ``templatelib``. As of this writing, no core -team consensus has emerged on the final location for these types. The current -PEP leaves this open for a final decision. - -One argument in favor of a new top-level ``templatelib`` module is that it would -allow for future addition of related methods (like ``convert()``) and for -potential future template processing code to be added to submodules -(``templatelib.shell``, etc.). +and even in a new top-level module, ``templatelib``. The final decision was to +place them in ``string.templatelib``. Enable Full Reconstruction of Original Template Literal @@ -1243,26 +1299,49 @@ Enable Full Reconstruction of Original Template Literal Earlier versions of this PEP attempted to make it possible to fully reconstruct the text of the original template string from a ``Template`` instance. This was -rejected as being overly complex. +rejected as being overly complex. The mapping between template literal source +and the underlying AST is not one-to-one and there are several limitations with +respect to round-tripping to the original source text. + +First, ``Interpolation.format_spec`` defaults to ``""`` if not provided: + +.. code-block:: python -There are several limitations with respect to round-tripping to the original -source text: + value = 42 + template1 = t"{value}" + template2 = t"{value:}" + assert template1.interpolations[0].format_spec == "" + assert template2.interpolations[0].format_spec == "" + +Next, the debug specifier, ``=``, is treated as a special case and is processed +before the AST is created. It is therefore not possible to distinguish +``t"{value=}"`` from ``t"value={value!r}"``: -- ``Interpolation.format_spec`` defaults to ``""`` if not provided. It is therefore - impossible to distinguish ``t"{expr}"`` from ``t"{expr:}"``. -- The debug specifier, ``=``, is treated as a special case. It is therefore not - possible to distinguish ``t"{expr=}"`` from ``t"expr={expr}"``. -- Finally, format specifiers in f-strings allow arbitrary nesting. In this PEP - and in the reference implementation, the specifier is eagerly evaluated - to set the ``format_spec`` in the ``Interpolation``, thereby losing - the original expressions. For example: +.. code-block:: python + + value = 42 + template1 = t"{value=}" + template2 = t"value={value!r}" + assert template1.strings[0] == "value=" + assert template1.interpolations[0].expression == "value" + assert template1.interpolations[0].conversion == "r" + assert template2.strings[0] == "value=" + assert template2.interpolations[0].expression == "value" + assert template2.interpolations[0].conversion == "r" + +Finally, format specifiers in f-strings allow arbitrary nesting. In this PEP +and in the reference implementation, the specifier is eagerly evaluated to +set the ``format_spec`` in the ``Interpolation``, thereby losing the original +expressions. For example: .. code-block:: python value = 42 precision = 2 - template = t"Value: {value:.{precision}f}" - assert template.interpolations[0].format_spec == ".2f" + template1 = t"{value:.2f}" + template2 = t"{value:.{precision}f}" + assert template1.interpolations[0].format_spec == ".2f" + assert template2.interpolations[0].format_spec == ".2f" We do not anticipate that these limitations will be a significant issue in practice. Developers who need to obtain the original template string literal can always @@ -1307,23 +1386,24 @@ PEP adheres closely to :pep:`701`. Any changes to allowed values should be in a separate PEP. -Removing ``conv`` From ``Interpolation`` ----------------------------------------- +Removing ``conversion`` From ``Interpolation`` +---------------------------------------------- -During the authoring of this PEP, we considered removing the ``conv`` attribute -from ``Interpolation`` and specifying that the conversion should be performed -eagerly, before ``Interpolation.value`` is set. +While drafting this PEP, we considered removing the ``conversion`` +attribute from ``Interpolation`` and specifying that the conversion should be +performed eagerly, before ``Interpolation.value`` is set. This was done to simplify the work of writing template processing code. The -``conv`` attribute is of limited extensibility (it is typed as +``conversion`` attribute is of limited extensibility (it is typed as ``Literal["r", "s", "a"] | None``). It is not clear that it adds significant value or flexibility to template strings that couldn't better be achieved with custom format specifiers. Unlike with format specifiers, there is no -equivalent to Python's :func:`python:format` built-in. (Instead, we include an +equivalent to Python's :func:`python:format` built-in. (Instead, we include a sample implementation of ``convert()`` in the `Examples`_ section.) -Ultimately we decided to keep the ``conv`` attribute in the ``Interpolation`` type -to maintain compatibility with f-strings and to allow for future extensibility. +Ultimately we decided to keep the ``conversion`` attribute in the +``Interpolation`` type to maintain compatibility with f-strings and to allow +for future extensibility. Alternate Interpolation Symbols