gh-144148: Update the urllib.parse documentation (GH-144497)

Document urlsplit() as the main parsing function and urlparse() as
an obsolete variant.

Author: serhiy-storchaka

Doc/library/urllib.parse.rst

@@ -50,11 +50,12 @@ URL Parsing
 The URL parsing functions focus on splitting a URL string into its components,
 or on combining URL components into a URL string.
 
-.. function:: urlparse(urlstring, scheme=None, allow_fragments=True, *, missing_as_none=False)
+.. function:: urlsplit(urlstring, scheme=None, allow_fragments=True, *, missing_as_none=False)
 
-   Parse a URL into six components, returning a 6-item :term:`named tuple`.  This
-   corresponds to the general structure of a URL:
-   ``scheme://netloc/path;parameters?query#fragment``.
+   Parse a URL into five components, returning a 5-item :term:`named tuple`
+   :class:`SplitResult` or :class:`SplitResultBytes`.
+   This corresponds to the general structure of a URL:
+   ``scheme://netloc/path?query#fragment``.
    Each tuple item is a string, possibly empty, or ``None`` if
    *missing_as_none* is true.
    Not defined component are represented an empty string (by default) or
@@ -68,15 +69,15 @@ or on combining URL components into a URL string.
    .. doctest::
       :options: +NORMALIZE_WHITESPACE
 
-      >>> from urllib.parse import urlparse
-      >>> urlparse("scheme://netloc/path;parameters?query#fragment")
-      ParseResult(scheme='scheme', netloc='netloc', path='/path;parameters', params='',
+      >>> from urllib.parse import urlsplit
+      >>> urlsplit("scheme://netloc/path?query#fragment")
+      SplitResult(scheme='scheme', netloc='netloc', path='/path',
                   query='query', fragment='fragment')
-      >>> o = urlparse("http://docs.python.org:80/3/library/urllib.parse.html?"
+      >>> o = urlsplit("http://docs.python.org:80/3/library/urllib.parse.html?"
       ...              "highlight=params#url-parsing")
       >>> o
-      ParseResult(scheme='http', netloc='docs.python.org:80',
-                  path='/3/library/urllib.parse.html', params='',
+      SplitResult(scheme='http', netloc='docs.python.org:80',
+                  path='/3/library/urllib.parse.html',
                   query='highlight=params', fragment='url-parsing')
       >>> o.scheme
       'http'
@@ -88,42 +89,42 @@ or on combining URL components into a URL string.
       80
       >>> o._replace(fragment="").geturl()
       'http://docs.python.org:80/3/library/urllib.parse.html?highlight=params'
-      >>> urlparse("http://docs.python.org?")
-      ParseResult(scheme='http', netloc='docs.python.org',
-                  path='', params='', query='', fragment='')
-      >>> urlparse("http://docs.python.org?", missing_as_none=True)
-      ParseResult(scheme='http', netloc='docs.python.org',
-                  path='', params=None, query='', fragment=None)
-
-   Following the syntax specifications in :rfc:`1808`, urlparse recognizes
+      >>> urlsplit("http://docs.python.org?")
+      SplitResult(scheme='http', netloc='docs.python.org', path='',
+                  query='', fragment='')
+      >>> urlsplit("http://docs.python.org?", missing_as_none=True)
+      SplitResult(scheme='http', netloc='docs.python.org', path='',
+                  query='', fragment=None)
+
+   Following the syntax specifications in :rfc:`1808`, :func:`!urlsplit` recognizes
    a netloc only if it is properly introduced by '//'.  Otherwise the
    input is presumed to be a relative URL and thus to start with
    a path component.
 
    .. doctest::
       :options: +NORMALIZE_WHITESPACE
 
-      >>> from urllib.parse import urlparse
-      >>> urlparse('//www.cwi.nl:80/%7Eguido/Python.html')
-      ParseResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
-                  params='', query='', fragment='')
-      >>> urlparse('www.cwi.nl/%7Eguido/Python.html')
-      ParseResult(scheme='', netloc='', path='www.cwi.nl/%7Eguido/Python.html',
-                  params='', query='', fragment='')
-      >>> urlparse('help/Python.html')
-      ParseResult(scheme='', netloc='', path='help/Python.html',
-                  params='', query='', fragment='')
-      >>> urlparse('help/Python.html', missing_as_none=True)
-      ParseResult(scheme=None, netloc=None, path='help/Python.html',
-                  params=None, query=None, fragment=None)
+      >>> from urllib.parse import urlsplit
+      >>> urlsplit('//www.cwi.nl:80/%7Eguido/Python.html')
+      SplitResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
+                  query='', fragment='')
+      >>> urlsplit('www.cwi.nl/%7Eguido/Python.html')
+      SplitResult(scheme='', netloc='', path='www.cwi.nl/%7Eguido/Python.html',
+                  query='', fragment='')
+      >>> urlsplit('help/Python.html')
+      SplitResult(scheme='', netloc='', path='help/Python.html',
+                  query='', fragment='')
+      >>> urlsplit('help/Python.html', missing_as_none=True)
+      SplitResult(scheme=None, netloc=None, path='help/Python.html',
+                  query=None, fragment=None)
 
    The *scheme* argument gives the default addressing scheme, to be
    used only if the URL does not specify one.  It should be the same type
    (text or bytes) as *urlstring* or ``None``, except that the ``''`` is
    always allowed, and is automatically converted to ``b''`` if appropriate.
 
    If the *allow_fragments* argument is false, fragment identifiers are not
-   recognized.  Instead, they are parsed as part of the path, parameters
+   recognized.  Instead, they are parsed as part of the path
    or query component, and :attr:`fragment` is set to ``None`` or the empty
    string (depending on the value of *missing_as_none*) in the return value.
 
@@ -140,12 +141,9 @@ or on combining URL components into a URL string.
    +------------------+-------+-------------------------+-------------------------------+
    | :attr:`path`     | 2     | Hierarchical path       | empty string                  |
    +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`params`   | 3     | Parameters for last     | ``None`` or empty string [1]_ |
-   |                  |       | path element            |                               |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`query`    | 4     | Query component         | ``None`` or empty string [1]_ |
+   | :attr:`query`    | 3     | Query component         | ``None`` or empty string [1]_ |
    +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`fragment` | 5     | Fragment identifier     | ``None`` or empty string [1]_ |
+   | :attr:`fragment` | 4     | Fragment identifier     | ``None`` or empty string [1]_ |
    +------------------+-------+-------------------------+-------------------------------+
    | :attr:`username` |       | User name               | ``None``                      |
    +------------------+-------+-------------------------+-------------------------------+
@@ -171,26 +169,30 @@ or on combining URL components into a URL string.
    ``#``, ``@``, or ``:`` will raise a :exc:`ValueError`. If the URL is
    decomposed before parsing, no error will be raised.
 
+   Following some of the `WHATWG spec`_ that updates :rfc:`3986`, leading C0
+   control and space characters are stripped from the URL. ``\n``,
+   ``\r`` and tab ``\t`` characters are removed from the URL at any position.
+
    As is the case with all named tuples, the subclass has a few additional methods
    and attributes that are particularly useful. One such method is :meth:`_replace`.
-   The :meth:`_replace` method will return a new ParseResult object replacing specified
-   fields with new values.
+   The :meth:`_replace` method will return a new :class:`SplitResult` object
+   replacing specified fields with new values.
 
    .. doctest::
       :options: +NORMALIZE_WHITESPACE
 
-      >>> from urllib.parse import urlparse
-      >>> u = urlparse('//www.cwi.nl:80/%7Eguido/Python.html')
+      >>> from urllib.parse import urlsplit
+      >>> u = urlsplit('//www.cwi.nl:80/%7Eguido/Python.html')
       >>> u
-      ParseResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
-                  params='', query='', fragment='')
+      SplitResult(scheme='', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
+                  query='', fragment='')
       >>> u._replace(scheme='http')
-      ParseResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
-                  params='', query='', fragment='')
+      SplitResult(scheme='http', netloc='www.cwi.nl:80', path='/%7Eguido/Python.html',
+                  query='', fragment='')
 
    .. warning::
 
-      :func:`urlparse` does not perform validation.  See :ref:`URL parsing
+      :func:`urlsplit` does not perform validation.  See :ref:`URL parsing
       security <url-parsing-security>` for details.
 
    .. versionchanged:: 3.2
@@ -209,9 +211,17 @@ or on combining URL components into a URL string.
       Characters that affect netloc parsing under NFKC normalization will
       now raise :exc:`ValueError`.
 
+   .. versionchanged:: 3.10
+      ASCII newline and tab characters are stripped from the URL.
+
+   .. versionchanged:: 3.12
+      Leading WHATWG C0 control and space characters are stripped from the URL.
+
    .. versionchanged:: next
       Added the *missing_as_none* parameter.
 
+.. _WHATWG spec: https://url.spec.whatwg.org/#concept-basic-url-parser
+
 
 .. function:: parse_qs(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace', max_num_fields=None, separator='&')
 
@@ -306,11 +316,11 @@ or on combining URL components into a URL string.
       separator key, with ``&`` as the default separator.
 
 
-.. function:: urlunparse(parts)
-              urlunparse(parts, *, keep_empty)
+.. function:: urlunsplit(parts)
+              urlunsplit(parts, *, keep_empty)
 
-   Construct a URL from a tuple as returned by ``urlparse()``. The *parts*
-   argument can be any six-item iterable.
+   Construct a URL from a tuple as returned by :func:`urlsplit`. The *parts*
+   argument can be any five-item iterable.
 
    This may result in a slightly different, but equivalent URL, if the
    URL that was parsed originally had unnecessary delimiters (for example,
@@ -321,97 +331,33 @@ or on combining URL components into a URL string.
    This allows rebuilding a URL that was parsed with option
    ``missing_as_none=True``.
    By default, *keep_empty* is true if *parts* is the result of the
-   :func:`urlparse` call with ``missing_as_none=True``.
+   :func:`urlsplit` call with ``missing_as_none=True``.
 
    .. versionchanged:: next
       Added the *keep_empty* parameter.
 
 
-.. function:: urlsplit(urlstring, scheme=None, allow_fragments=True, *, missing_as_none=False)
-
-   This is similar to :func:`urlparse`, but does not split the params from the URL.
-   This should generally be used instead of :func:`urlparse` if the more recent URL
-   syntax allowing parameters to be applied to each segment of the *path* portion
-   of the URL (see :rfc:`2396`) is wanted.  A separate function is needed to
-   separate the path segments and parameters.  This function returns a 5-item
-   :term:`named tuple`::
-
-      (addressing scheme, network location, path, query, fragment identifier).
-
-   The return value is a :term:`named tuple`, its items can be accessed by index
-   or as named attributes:
-
-   +------------------+-------+-------------------------+-------------------------------+
-   | Attribute        | Index | Value                   | Value if not present          |
-   +==================+=======+=========================+===============================+
-   | :attr:`scheme`   | 0     | URL scheme specifier    | *scheme* parameter or         |
-   |                  |       |                         | empty string [1]_             |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`netloc`   | 1     | Network location part   | ``None`` or empty string [2]_ |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`path`     | 2     | Hierarchical path       | empty string                  |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`query`    | 3     | Query component         | ``None`` or empty string [2]_ |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`fragment` | 4     | Fragment identifier     | ``None`` or empty string [2]_ |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`username` |       | User name               | ``None``                      |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`password` |       | Password                | ``None``                      |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`hostname` |       | Host name (lower case)  | ``None``                      |
-   +------------------+-------+-------------------------+-------------------------------+
-   | :attr:`port`     |       | Port number as integer, | ``None``                      |
-   |                  |       | if present              |                               |
-   +------------------+-------+-------------------------+-------------------------------+
-
-   .. [2] Depending on the value of the *missing_as_none* argument.
-
-   Reading the :attr:`port` attribute will raise a :exc:`ValueError` if
-   an invalid port is specified in the URL.  See section
-   :ref:`urlparse-result-object` for more information on the result object.
-
-   Unmatched square brackets in the :attr:`netloc` attribute will raise a
-   :exc:`ValueError`.
-
-   Characters in the :attr:`netloc` attribute that decompose under NFKC
-   normalization (as used by the IDNA encoding) into any of ``/``, ``?``,
-   ``#``, ``@``, or ``:`` will raise a :exc:`ValueError`. If the URL is
-   decomposed before parsing, no error will be raised.
-
-   Following some of the `WHATWG spec`_ that updates RFC 3986, leading C0
-   control and space characters are stripped from the URL. ``\n``,
-   ``\r`` and tab ``\t`` characters are removed from the URL at any position.
-
-   .. warning::
-
-      :func:`urlsplit` does not perform validation.  See :ref:`URL parsing
-      security <url-parsing-security>` for details.
-
-   .. versionchanged:: 3.6
-      Out-of-range port numbers now raise :exc:`ValueError`, instead of
-      returning ``None``.
-
-   .. versionchanged:: 3.8
-      Characters that affect netloc parsing under NFKC normalization will
-      now raise :exc:`ValueError`.
-
-   .. versionchanged:: 3.10
-      ASCII newline and tab characters are stripped from the URL.
-
-   .. versionchanged:: 3.12
-      Leading WHATWG C0 control and space characters are stripped from the URL.
+.. function:: urlparse(urlstring, scheme=None, allow_fragments=True, *, missing_as_none=False)
 
-   .. versionchanged:: next
-      Added the *missing_as_none* parameter.
+   This is similar to :func:`urlsplit`, but additionally splits the *path*
+   component on *path* and *params*.
+   This function returns a 6-item :term:`named tuple` :class:`ParseResult`
+   or :class:`ParseResultBytes`.
+   Its items are the same as for the :func:`!urlsplit` result, except that
+   *params* is inserted at index 3, between *path* and *query*.
 
-.. _WHATWG spec: https://url.spec.whatwg.org/#concept-basic-url-parser
+   This function is based on obsoleted :rfc:`1738` and :rfc:`1808`, which
+   listed *params* as the main URL component.
+   The more recent URL syntax allows parameters to be applied to each segment
+   of the *path* portion of the URL (see :rfc:`3986`).
+   :func:`urlsplit` should generally be used instead of :func:`urlparse`.
+   A separate function is needed to separate the path segments and parameters.
 
-.. function:: urlunsplit(parts)
-              urlunsplit(parts, *, keep_empty)
+.. function:: urlunparse(parts)
+              urlunparse(parts, *, keep_empty)
 
-   Combine the elements of a tuple as returned by :func:`urlsplit` into a
-   complete URL as a string. The *parts* argument can be any five-item
+   Combine the elements of a tuple as returned by :func:`urlparse` into a
+   complete URL as a string. The *parts* argument can be any six-item
    iterable.
 
    This may result in a slightly different, but equivalent URL, if the
@@ -423,7 +369,7 @@ or on combining URL components into a URL string.
    This allows rebuilding a URL that was parsed with option
    ``missing_as_none=True``.
    By default, *keep_empty* is true if *parts* is the result of the
-   :func:`urlsplit` call with ``missing_as_none=True``.
+   :func:`urlparse` call with ``missing_as_none=True``.
 
    .. versionchanged:: next
       Added the *keep_empty* parameter.
@@ -441,7 +387,7 @@ or on combining URL components into a URL string.
       'http://www.cwi.nl/%7Eguido/FAQ.html'
 
    The *allow_fragments* argument has the same meaning and default as for
-   :func:`urlparse`.
+   :func:`urlsplit`.
 
    .. note::
 
@@ -587,7 +533,7 @@ individual URL quoting functions.
 Structured Parse Results
 ------------------------
 
-The result objects from the :func:`urlparse`, :func:`urlsplit`  and
+The result objects from the :func:`urlsplit`, :func:`urlparse`  and
 :func:`urldefrag` functions are subclasses of the :class:`tuple` type.
 These subclasses add the attributes listed in the documentation for
 those functions, the encoding and decoding support described in the

Doc/library/venv.rst

@@ -550,7 +550,7 @@ subclass which installs setuptools and pip into a created virtual environment::
     from subprocess import Popen, PIPE
     import sys
     from threading import Thread
-    from urllib.parse import urlparse
+    from urllib.parse import urlsplit
     from urllib.request import urlretrieve
     import venv
 
@@ -621,7 +621,7 @@ subclass which installs setuptools and pip into a created virtual environment::
             stream.close()
 
         def install_script(self, context, name, url):
-            _, _, path, _, _, _ = urlparse(url)
+            _, _, path, _, _ = urlsplit(url)
             fn = os.path.split(path)[-1]
             binpath = context.bin_path
             distpath = os.path.join(binpath, fn)

Doc/whatsnew/3.15.rst

@@ -968,10 +968,10 @@ unittest
 urllib.parse
 ------------
 
-* Add the *missing_as_none* parameter to :func:`~urllib.parse.urlparse`,
-  :func:`~urllib.parse.urlsplit` and :func:`~urllib.parse.urldefrag` functions.
-  Add the *keep_empty* parameter to :func:`~urllib.parse.urlunparse` and
-  :func:`~urllib.parse.urlunsplit` functions.
+* Add the *missing_as_none* parameter to :func:`~urllib.parse.urlsplit`,
+  :func:`~urllib.parse.urlparse` and :func:`~urllib.parse.urldefrag` functions.
+  Add the *keep_empty* parameter to :func:`~urllib.parse.urlunsplit` and
+  :func:`~urllib.parse.urlunparse` functions.
   This allows to distinguish between empty and not defined URI components
   and preserve empty components.
   (Contributed by Serhiy Storchaka in :gh:`67041`.)