PEP 782: Address Steve's review (#4335)

Author: vstinner

peps/pep-0782.rst

@@ -80,8 +80,9 @@ Create, Finish, Discard
 
    Create a :c:type:`PyBytesWriter` to write *size* bytes.
 
-   If *size* is greater than zero, allocate *size* bytes for the
-   returned buffer.
+   If *size* is greater than zero, allocate *size* bytes, and set the
+   writer size to *size*. The caller is responsible to write *size*
+   bytes using :c:func:`PyBytesWriter_GetData`.
 
    On error, set an exception and return NULL.
 
@@ -107,14 +108,14 @@ Create, Finish, Discard
    Similar to :c:func:`PyBytesWriter_Finish`, but resize the writer
    using *buf* pointer before creating the :class:`bytes` object.
 
-   Pseudo-code::
+   Set an exception and return ``NULL`` if *buf* pointer is outside the
+   internal buffer bounds.
+
+   Function pseudo-code::
 
        Py_ssize_t size = (char*)buf - (char*)PyBytesWriter_GetData(writer);
        return PyBytesWriter_FinishWithSize(writer, size);
 
-   Set an exception and return ``NULL`` if *buf* pointer is outside the
-   internal buffer bounds.
-
 .. c:function:: void PyBytesWriter_Discard(PyBytesWriter *writer)
 
    Discard a :c:type:`PyBytesWriter` created by :c:func:`PyBytesWriter_Create`.
@@ -128,7 +129,9 @@ High-level API
 
 .. c:function:: int PyBytesWriter_WriteBytes(PyBytesWriter *writer, const void *bytes, Py_ssize_t size)
 
-   Write *size* bytes of *bytes* into the *writer*.
+   Grow the *writer* internal buffer by *size* bytes,
+   write *size* bytes of *bytes* at the *writer* end,
+   and add *size* to the *writer* size.
 
    If *size* is equal to ``-1``, call ``strlen(bytes)`` to get the
    string length.
@@ -138,8 +141,9 @@ High-level API
 
 .. c:function:: int PyBytesWriter_Format(PyBytesWriter *writer, const char *format, ...)
 
-   Similar to ``PyBytes_FromFormat()``, but write the output directly
-   into the writer.
+   Similar to ``PyBytes_FromFormat()``, but write the output directly at
+   the writer end. Grow the writer internal buffer on demand.
+   Then add the written size to the writer size.
 
    On success, return ``0``.
    On error, set an exception and return ``-1``.
@@ -153,7 +157,7 @@ Getters
 
 .. c:function:: void* PyBytesWriter_GetData(PyBytesWriter *writer)
 
-   Get the writer data.
+   Get the writer data: start of the internal buffer.
 
    The pointer is valid until :c:func:`PyBytesWriter_Finish` or
    :c:func:`PyBytesWriter_Discard` is called on *writer*.
@@ -182,16 +186,22 @@ Low-level API
    On success, return ``0``.
    On error, set an exception and return ``-1``.
 
-   *size* must be positive or zero.
+   *size* can be negative to shrink the writer.
 
 .. c:function:: void* PyBytesWriter_GrowAndUpdatePointer(PyBytesWriter *writer, Py_ssize_t size, void *buf)
 
    Similar to :c:func:`PyBytesWriter_Grow`, but update also the *buf*
    pointer.
 
+   The *buf* pointer is moved if the internal buffer is moved in memory.
+   The *buf* relative position within the internal buffer is left
+   unchanged.
+
    On error, set an exception and return ``NULL``.
 
-   Pseudo-code::
+   *buf* must not be ``NULL``.
+
+   Function pseudo-code::
 
        Py_ssize_t pos = (char*)buf - (char*)PyBytesWriter_GetData(writer);
        if (PyBytesWriter_Grow(writer, size) < 0) {
@@ -207,6 +217,10 @@ Overallocation
 overallocate the internal buffer to reduce the number of ``realloc()``
 calls and so reduce memory copies.
 
+:c:func:`PyBytesWriter_Finish` trims overallocations: it shrinks the
+internal buffer to the exact size when creating the final :class:`bytes`
+object.
+
 
 Thread safety
 -------------
@@ -273,8 +287,8 @@ Example creating the bytes string ``b"abc"``, with a fixed size of 3 bytes::
         return PyBytesWriter_Finish(writer);
     }
 
-GrowAndUpdatePointer() example
-------------------------------
+``GrowAndUpdatePointer()`` example
+----------------------------------
 
 Example using a pointer to write bytes and to track the written size.
 
@@ -310,22 +324,84 @@ Create the bytes string ``b"Hello World"``::
     }
 
 
+Update ``PyBytes_FromStringAndSize()`` code
+-------------------------------------------
+
+Example of code using the soft deprecated
+``PyBytes_FromStringAndSize(NULL, size)`` API::
+
+    PyObject *result = PyBytes_FromStringAndSize(NULL, num_bytes);
+    if (result == NULL) {
+        return NULL;
+    }
+    if (copy_bytes(PyBytes_AS_STRING(result), start, num_bytes) < 0) {
+        Py_CLEAR(result);
+    }
+    return result;
+
+It can now be updated to::
+
+    PyBytesWriter *writer = PyBytesWriter_Create(num_bytes);
+    if (writer == NULL) {
+        return NULL;
+    }
+    if (copy_bytes(PyBytesWriter_GetData(writer), start, num_bytes) < 0) {
+        PyBytesWriter_Discard(writer);
+        return NULL;
+    }
+    return PyBytesWriter_Finish(writer);
+
+
+Update ``_PyBytes_Resize()`` code
+---------------------------------
+
+Example of code using the soft deprecated ``_PyBytes_Resize()`` API::
+
+    PyObject *v = PyBytes_FromStringAndSize(NULL, size);
+    if (v == NULL) {
+        return NULL;
+    }
+    char *p = PyBytes_AS_STRING(v);
+
+    // ... fill bytes into 'p' ...
+
+    if (_PyBytes_Resize(&v, (p - PyBytes_AS_STRING(v)))) {
+        return NULL;
+    }
+    return v;
+
+It can now be updated to::
+
+    PyBytesWriter *writer = PyBytesWriter_Create(size);
+    if (writer == NULL) {
+        return NULL;
+    }
+    char *p = PyBytesWriter_GetData(writer);
+
+    // ... fill bytes into 'p' ...
+
+    return PyBytesWriter_FinishWithPointer(writer, p);
+
+
 Reference Implementation
 ========================
 
 `Pull request gh-131681 <https://github.com/python/cpython/pull/131681>`__.
 
-The implementation allocates internally a :class:`bytes` object, so
-:c:func:`PyBytesWriter_Finish` just returns the object without having
-to copy memory.
+Notes on the CPython reference implementation which are not part of the
+Specification:
 
-For strings up to 256 bytes, a small internal raw buffer of bytes is
-used. It avoids having to resize a :class:`bytes` object which is
-inefficient. At the end, :c:func:`PyBytesWriter_Finish` creates the
-:class:`bytes` object from this small buffer.
+* The implementation allocates internally a :class:`bytes` object, so
+  :c:func:`PyBytesWriter_Finish` just returns the object without having
+  to copy memory.
 
-A free list is used to reduce the cost of allocating a
-:c:type:`PyBytesWriter` on the heap memory.
+* For strings up to 256 bytes, a small internal raw buffer of bytes is
+  used. It avoids having to resize a :class:`bytes` object which is
+  inefficient. At the end, :c:func:`PyBytesWriter_Finish` creates the
+  :class:`bytes` object from this small buffer.
+
+* A free list is used to reduce the cost of allocating a
+  :c:type:`PyBytesWriter` on the heap memory.
 
 
 Backwards Compatibility
@@ -334,6 +410,10 @@ Backwards Compatibility
 There is no impact on the backward compatibility, only new APIs are
 added.
 
+``PyBytes_FromStringAndSize(NULL, size)`` and ``_PyBytes_Resize()`` APIs
+are soft deprecated. No new warnings is emitted when these functions are
+used and they are not planned for removal.
+
 
 Prior Discussions
 =================