Merge branch 'main' into gh-134786-managed-weakref

Author: sergey-miryanov

Doc/library/collections.rst

@@ -367,9 +367,11 @@ Several mathematical operations are provided for combining :class:`Counter`
 objects to produce multisets (counters that have counts greater than zero).
 Addition and subtraction combine counters by adding or subtracting the counts
 of corresponding elements.  Intersection and union return the minimum and
-maximum of corresponding counts.  Equality and inclusion compare
-corresponding counts.  Each operation can accept inputs with signed
-counts, but the output will exclude results with counts of zero or less.
+maximum of corresponding counts.  Symmetric difference returns the difference
+between the maximum and minimum of the corresponding counts. Equality and
+inclusion compare corresponding counts.  Each operation can accept inputs
+with signed counts, but the output will exclude results with counts of zero
+or below.
 
 .. doctest::
 
@@ -383,6 +385,8 @@ counts, but the output will exclude results with counts of zero or less.
     Counter({'a': 1, 'b': 1})
     >>> c | d                       # union:  max(c[x], d[x])
     Counter({'a': 3, 'b': 2})
+    >>> c ^ d                       # max(c[x], d[x]) - min(c[x], d[x])
+    Counter({'a': 2, 'b': 1})
     >>> c == d                      # equality:  c[x] == d[x]
     False
     >>> c <= d                      # inclusion:  c[x] <= d[x]
@@ -400,6 +404,9 @@ or subtracting from an empty counter.
 .. versionadded:: 3.3
     Added support for unary plus, unary minus, and in-place multiset operations.
 
+.. versionadded:: next
+    Added support for the symmetric difference multiset operation, ``c ^ d``.
+
 .. note::
 
     Counters were primarily designed to work with positive integers to represent

Doc/library/pyexpat.rst

@@ -72,6 +72,13 @@ The :mod:`xml.parsers.expat` module contains two functions:
    *encoding* [1]_ is given it will override the implicit or explicit encoding of the
    document.
 
+   .. _xmlparser-non-root:
+
+   Parsers created through :func:`!ParserCreate` are called "root" parsers,
+   in the sense that they do not have any parent parser attached. Non-root
+   parsers are created by :meth:`parser.ExternalEntityParserCreate
+   <xmlparser.ExternalEntityParserCreate>`.
+
    Expat can optionally do XML namespace processing for you, enabled by providing a
    value for *namespace_separator*.  The value must be a one-character string; a
    :exc:`ValueError` will be raised if the string has an illegal length (``None``
@@ -231,6 +238,55 @@ XMLParser Objects
    .. versionadded:: 3.13
 
 
+:class:`!xmlparser` objects have the following methods to mitigate some
+common XML vulnerabilities.
+
+.. method:: xmlparser.SetAllocTrackerActivationThreshold(threshold, /)
+
+   Sets the number of allocated bytes of dynamic memory needed to activate
+   protection against disproportionate use of RAM.
+
+   By default, parser objects have an allocation activation threshold of 64 MiB,
+   or equivalently 67,108,864 bytes.
+
+   An :exc:`ExpatError` is raised if this method is called on a
+   |xml-non-root-parser| parser.
+   The corresponding :attr:`~ExpatError.lineno` and :attr:`~ExpatError.offset`
+   should not be used as they may have no special meaning.
+
+   .. versionadded:: next
+
+.. method:: xmlparser.SetAllocTrackerMaximumAmplification(max_factor, /)
+
+   Sets the maximum amplification factor between direct input and bytes
+   of dynamic memory allocated.
+
+   The amplification factor is calculated as ``allocated / direct``
+   while parsing, where ``direct`` is the number of bytes read from
+   the primary document in parsing and ``allocated`` is the number
+   of bytes of dynamic memory allocated in the parser hierarchy.
+
+   The *max_factor* value must be a non-NaN :class:`float` value greater than
+   or equal to 1.0. Amplification factors greater than 100.0 can be observed
+   near the start of parsing even with benign files in practice. In particular,
+   the activation threshold should be carefully chosen to avoid false positives.
+
+   By default, parser objects have a maximum amplification factor of 100.0.
+
+   An :exc:`ExpatError` is raised if this method is called on a
+   |xml-non-root-parser| parser or if *max_factor* is outside the valid range.
+   The corresponding :attr:`~ExpatError.lineno` and :attr:`~ExpatError.offset`
+   should not be used as they may have no special meaning.
+
+   .. note::
+
+      The maximum amplification factor is only considered if the threshold
+      that can be adjusted by :meth:`.SetAllocTrackerActivationThreshold`
+      is exceeded.
+
+   .. versionadded:: next
+
+
 :class:`xmlparser` objects have the following attributes:
 
 
@@ -954,3 +1010,4 @@ The ``errors`` module has the following attributes:
    not. See https://www.w3.org/TR/2006/REC-xml11-20060816/#NT-EncodingDecl
    and https://www.iana.org/assignments/character-sets/character-sets.xhtml.
 
+.. |xml-non-root-parser| replace:: :ref:`non-root <xmlparser-non-root>`

Doc/library/xml.rst

@@ -55,7 +55,7 @@ An attacker can abuse XML features to carry out denial of service attacks,
 access local files, generate network connections to other machines, or
 circumvent firewalls.
 
-Expat versions lower that 2.6.0 may be vulnerable to "billion laughs",
+Expat versions lower than 2.6.0 may be vulnerable to "billion laughs",
 "quadratic blowup" and "large tokens". Python may be vulnerable if it uses such
 older versions of Expat as a system-provided library.
 Check :const:`!pyexpat.EXPAT_VERSION`.

Doc/using/configure.rst

@@ -869,9 +869,9 @@ Libraries options
    .. versionchanged:: 3.13
       Default to using the installed ``mpdecimal`` library.
 
-   .. deprecated-removed:: 3.13 3.15
+   .. deprecated-removed:: 3.13 3.16
       A copy of the ``mpdecimal`` library sources will no longer be distributed
-      with Python 3.15.
+      with Python 3.16.
 
    .. seealso:: :option:`LIBMPDEC_CFLAGS` and :option:`LIBMPDEC_LIBS`.
 

Doc/whatsnew/3.15.rst

@@ -294,6 +294,14 @@ New modules
 Improved modules
 ================
 
+collections
+-----------
+
+* Added :meth:`!collections.Counter.__xor__` and
+  :meth:`!collections.Counter.__ixor__` to compute the symmetric difference
+  between :class:`~collections.Counter` objects.
+  (Contributed by Raymond Hettinger in :gh:`138682`.)
+
 collections.abc
 ---------------
 
@@ -545,6 +553,16 @@ unittest
   (Contributed by Garry Cairns in :gh:`134567`.)
 
 
+xml.parsers.expat
+-----------------
+
+* Add :meth:`~xml.parsers.expat.xmlparser.SetAllocTrackerActivationThreshold`
+  and :meth:`~xml.parsers.expat.xmlparser.SetAllocTrackerMaximumAmplification`
+  to :ref:`xmlparser <xmlparser-objects>` objects to prevent use of
+  disproportional amounts of dynamic memory from within an Expat parser.
+  (Contributed by Bénédikt Tran in :gh:`90949`.)
+
+
 zlib
 ----