PEP 768: Add some clarifications for buffer size

Author: pablogsal

peps/pep-0768.rst

@@ -132,24 +132,27 @@ Runtime State Extensions
 
 A new structure is added to PyThreadState to support remote debugging:
 
+debugging:
+
 .. code-block:: C
 
     typedef struct _remote_debugger_support {
         int debugger_pending_call;
-        char debugger_script[MAX_SCRIPT_SIZE];
+        char debugger_script_path[MAX_SCRIPT_PATH_SIZE];
+        char *debugger_buffer;
+        Py_ssize_t debugger_buffer_size;
     } _PyRemoteDebuggerSupport;
 
-
 This structure is appended to ``PyThreadState``, adding only a few fields that
 are **never accessed during normal execution**. The ``debugger_pending_call`` field
 indicates when a debugger has requested execution, while ``debugger_script``
 provides Python code to be executed when the interpreter reaches a safe point.
 
-The value for ``MAX_SCRIPT_SIZE`` will be a trade-off between binary size and
-how big debugging scripts can be. As most of the logic should be in libraries
-and arbitrary code can be executed with very short amount of Python we are
-proposing to start with 4kb initially. This value can be extended in the future
-if we ever need to.
+The value for ``MAX_SCRIPT_PATH_SIZE`` will be a trade-off between binary size
+and how big debugging scripts paths can be. To limit the memory overhead per
+thread we will be limiting this to 512 bytes. This size will also be provided as
+part of the debugger support structure so debuggers know how much they can
+write.  This value can be extended in the future if we ever need to.
 
 
 Debug Offsets Table
@@ -201,7 +204,7 @@ When a debugger wants to attach to a Python process, it follows these steps:
 
    - Write a filename containing Python code to be executed into the
      ``debugger_script`` field in ``_PyRemoteDebuggerSupport``.
-   - Set ``debugger_pending_call`` flag in ``_PyRemoteDebuggerSupport``
+   - Set ``debugger_pending_call`` flag in ``_PyRemoteDebuggerSupport`` to 1
    - Set ``_PY_EVAL_PLEASE_STOP_BIT`` in the ``eval_breaker`` field
 
 Once the interpreter reaches the next safe point, it will execute the script
@@ -224,6 +227,9 @@ the interpreter will execute the provided debugging code at the next safe point.
 This all happens in a completely safe context, since the interpreter is
 guaranteed to be in a consistent state whenever the eval breaker is checked.
 
+The only valid values for ``debugger_pending_call`` will initially be 0 and 1
+and other values are reserved for future use.
+
 An audit event will be raised before the code is executed, allowing this mechanism
 to be audited or disabled if desired by a system's administrator.
 
@@ -464,6 +470,21 @@ in the file path to point to somewhere attacker controlled, this would allow
 them to force their malicious code to be executed rather than the code the
 debugger intends to run.
 
+Using a Single Runtime Buffer
+-----------------------------------
+
+During the review of this PEP it has been suggested using a single
+shared buffer at the runtime level for all debugger communications. While this
+appeared simpler and requring less memory , we discovered it would actually prevent scenarios
+where multiple debuggers need to coordinate operations across different threads,
+or where a single debugger needs to orchestrate complex debugging operations. A
+single shared buffer would force serialization of all debugging operations,
+making it impossible for debuggers to work independently on different threads.
+
+The per-thread buffer approach, despite its memory overhead in highly threaded
+applications, enables these important debugging scenarios by allowing each
+debugger to communicate independently with its target thread. 
+
 Thanks
 ======