@@ -14,47 +14,25 @@ Post-History: `10-Mar-2025 <https://discuss.python.org/t/83959>`__,
1414Abstract
1515========
1616
17- In Python, threads are able to interact with an interpreter (e.g., invoke the
18- bytecode loop) through an :term: `attached thread state `. On with-GIL builds,
19- only one thread can hold an attached thread state at once, which means that
20- the thread holds the :term: `GIL `. On free-threaded builds, there can be
21- an unbounded number of thread states attached, allowing for parallelism (because
22- multiple threads can invoke the interpreter at once).
23-
24- With that in mind, attachment of thread states is a bit problematic in the C API.
25- The C API currently provides two ways to acquire and attach a thread state for
26- an interpreter:
27-
28- - :c:func: `PyGILState_Ensure ` & :c:func: `PyGILState_Release `.
29- - :c:func: `PyThreadState_New ` & :c:func: `PyThreadState_Swap ` (significantly
30- less common).
31-
32- The former, :c:func: `PyGILState_Ensure ` and :c:func: `PyGILState_Release `,
33- are the most common way to do this and have been the standard for over twenty
34- years (:pep: `311 `), but have a number of issues that have arisen over time:
35-
36- - Subinterpreters tend to have trouble with them, because in threads that
37- haven't ever had an attached thread state, :c:func: `PyGILState_Ensure `
38- will assume that the main interpreter was requested. This makes it
39- impossible for the thread to interact with the subinterpreter!
40- - The phrase "GIL" is confusing for developers of free-threaded
41- extensions, because there's no GIL there, right? Even on free-threaded
42- builds, threads still needs a thread state to interact with the interpreter,
43- it's just that they don't have to wait on one-another to do so. These days,
44- the important thing that :c:func: `PyGILState_Ensure ` does is get attach a
45- thread state, and acquiring the GIL is somewhat incidental.
46-
47- The other option, :c:func: `PyThreadState_New ` and :c:func: `PyThreadState_Swap `,
48- do solve those issues, but come with an additional problem with how thread state
49- attachment works in the C API (that ``PyGILState `` also includes): if the
50- thread is not the main thread, then the interpreter will randomly hang the
51- thread during attachment if it starts finalizing. This is a problem for large
52- applications that want to use their thread in addition to calling Python.
53-
54- This PEP intends to solve these issues by providing :c:func: `PyThreadState_Ensure `
55- and :c:func: `PyThreadState_Release ` as replacements for the existing functions,
56- accompanied by some interpreter reference counting APIs that let thread states
57- be acquired and attached in a thread-safe and predictable manner.
17+ In the C API, threads are able to interact with an interpreter by holding an
18+ :term: `attached thread state ` for the current thread. This works well, but
19+ can get complicated when it comes to creating and attaching :term: `thread states `
20+ in a thread-safe manner.
21+
22+ Specifically, the C API doesn't have any way to ensure that an interpreter
23+ is in a state where it can be called when creating and/or attaching a thread
24+ state. As such, attachment might hang the thread, or in subinterpreters, it
25+ might flat-out crash due to the interpreter's structure being deallocated.
26+ This can be a frustrating issue to deal with in large applications that
27+ want to execute Python code alongside some other native code.
28+
29+ In addition, assumptions about which interpreter to use tend to be wrong
30+ inside of subinterpreters, primarily because :c:func: `PyGILState_Ensure `
31+ always creates a thread state for the main interpreter in threads where
32+ Python hasn't ever run.
33+
34+ This PEP intends to solve these kinds issues by *reimagining * how we approach
35+ thread states in the C API.
5836
5937Terminology
6038===========
0 commit comments