Skip to content

Commit a39c535

Browse files
Fix SIGSEGV/SIGABRT during interpreter shutdown on Python < 3.11
During interpreter finalization (Py_FinalizeEx), active greenlets being deallocated would trigger g_switch() to throw GreenletExit. This performs a stack switch and executes Python code in a partially-torn-down interpreter, causing: - SIGSEGV (signal 11) on greenlet 3.x - SIGABRT (signal 6 / "Accessing state after destruction") on greenlet 2.x On Python >= 3.11, CPython's restructured finalization internals (frame representation, data stack management, recursion tracking) make g_switch() during finalization safe. On Python < 3.11, this was not the case. This commit adds two guards, compiled only on Python < 3.11 (!GREENLET_PY311): 1. In _green_dealloc_kill_started_non_main_greenlet (PyGreenlet.cpp): When the interpreter is finalizing, call murder_in_place() directly instead of attempting g_switch(). This marks the greenlet as dead without throwing GreenletExit, avoiding the crash at the cost of not running cleanup code inside the greenlet. 2. In ~ThreadState (TThreadState.hpp): When the interpreter is finalizing, skip the GC-based leak detection that calls PyImport_ImportModule("gc"), which is unsafe when the import machinery is partially torn down. Only perform minimal safe cleanup (clearing strong references). On Python >= 3.11, no changes are made — the existing behavior (throwing GreenletExit via g_switch, running cleanup code) continues to work correctly during finalization. Also adds test_interpreter_shutdown.py with 9 subprocess-based tests covering: - Single/multiple/nested/threaded/deeply-nested active greenlets at shutdown (no-crash safety on all Python versions) - Version-aware behavioral tests verifying that GreenletExit cleanup code runs on Python >= 3.11 but is correctly skipped on < 3.11 - Edge cases: active exception context, stress test with 50 greenlets Fixes #411 See also #351, #376 Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent d4606ef commit a39c535

3 files changed

Lines changed: 356 additions & 0 deletions

File tree

src/greenlet/PyGreenlet.cpp

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,6 +189,27 @@ green_clear(PyGreenlet* self)
189189
static int
190190
_green_dealloc_kill_started_non_main_greenlet(BorrowedGreenlet self)
191191
{
192+
// During interpreter finalization, we cannot safely throw GreenletExit
193+
// into the greenlet. Doing so calls g_switch(), which performs a stack
194+
// switch and runs Python code via _PyEval_EvalFrameDefault. On Python
195+
// < 3.11, executing Python code in a partially-torn-down interpreter
196+
// leads to SIGSEGV (greenlet 3.x) or SIGABRT (greenlet 2.x).
197+
//
198+
// Python 3.11+ restructured interpreter finalization internals (frame
199+
// representation, data stack management, recursion tracking) so that
200+
// g_switch() during finalization is safe. On older Pythons, we simply
201+
// mark the greenlet dead without throwing, which avoids the crash at
202+
// the cost of not running any cleanup code inside the greenlet.
203+
//
204+
// See: https://github.com/python-greenlet/greenlet/issues/411
205+
// https://github.com/python-greenlet/greenlet/issues/351
206+
#if !GREENLET_PY311
207+
if (_Py_IsFinalizing()) {
208+
self->murder_in_place();
209+
return 1;
210+
}
211+
#endif
212+
192213
/* Hacks hacks hacks copied from instance_dealloc() */
193214
/* Temporarily resurrect the greenlet. */
194215
assert(self.REFCNT() == 0);

src/greenlet/TThreadState.hpp

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -384,6 +384,26 @@ class ThreadState {
384384
return;
385385
}
386386

387+
// During interpreter finalization, Python APIs like
388+
// PyImport_ImportModule are unsafe (the import machinery may
389+
// be partially torn down). On Python < 3.11, perform only the
390+
// minimal cleanup that is safe: clear our strong references so
391+
// we don't leak, but skip the GC-based leak detection.
392+
//
393+
// Python 3.11+ restructured interpreter finalization so that
394+
// these APIs remain safe during shutdown.
395+
#if !GREENLET_PY311
396+
if (_Py_IsFinalizing()) {
397+
this->tracefunc.CLEAR();
398+
if (this->current_greenlet) {
399+
this->current_greenlet->murder_in_place();
400+
this->current_greenlet.CLEAR();
401+
}
402+
this->main_greenlet.CLEAR();
403+
return;
404+
}
405+
#endif
406+
387407
// We should not have an "origin" greenlet; that only exists
388408
// for the temporary time during a switch, which should not
389409
// be in progress as the thread dies.
Lines changed: 315 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,315 @@
1+
# -*- coding: utf-8 -*-
2+
"""
3+
Tests for greenlet behavior during interpreter shutdown (Py_FinalizeEx).
4+
5+
Prior to the safe finalization fix, active greenlets being deallocated
6+
during interpreter shutdown could trigger SIGSEGV or SIGABRT on Python
7+
< 3.11, because green_dealloc attempted to throw GreenletExit via
8+
g_switch() into a partially-torn-down interpreter.
9+
10+
The fix checks _Py_IsFinalizing() and calls murder_in_place() instead
11+
of g_switch() when the interpreter is shutting down.
12+
13+
On Python >= 3.11, the interpreter's finalization internals are safe
14+
enough that g_switch() works correctly, so GreenletExit IS thrown and
15+
cleanup code inside the greenlet runs normally.
16+
17+
These tests verify:
18+
1. No crashes on ANY Python version (the core safety guarantee).
19+
2. On Python >= 3.11, GreenletExit cleanup code runs during shutdown.
20+
3. On Python < 3.11, greenlets are killed in place (no cleanup) to
21+
avoid the crash — this is the expected trade-off.
22+
"""
23+
import sys
24+
import subprocess
25+
import unittest
26+
import textwrap
27+
28+
from greenlet.tests import TestCase
29+
30+
PY311 = sys.version_info[:2] >= (3, 11)
31+
32+
33+
class TestInterpreterShutdown(TestCase):
34+
35+
def _run_shutdown_script(self, script_body):
36+
"""
37+
Run a Python script in a subprocess that exercises greenlet
38+
during interpreter shutdown. Returns (returncode, stdout, stderr).
39+
"""
40+
full_script = textwrap.dedent(script_body)
41+
result = subprocess.run(
42+
[sys.executable, '-c', full_script],
43+
capture_output=True,
44+
text=True,
45+
timeout=30,
46+
)
47+
return result.returncode, result.stdout, result.stderr
48+
49+
# -----------------------------------------------------------------
50+
# Core safety tests: no crashes on any Python version
51+
# -----------------------------------------------------------------
52+
53+
def test_active_greenlet_at_shutdown_no_crash(self):
54+
"""
55+
An active (suspended) greenlet that is deallocated during
56+
interpreter shutdown should not crash the process.
57+
58+
Before the fix, this would SIGSEGV on Python < 3.11 because
59+
_green_dealloc_kill_started_non_main_greenlet tried to call
60+
g_switch() during Py_FinalizeEx.
61+
"""
62+
rc, stdout, stderr = self._run_shutdown_script("""\
63+
import greenlet
64+
65+
def worker():
66+
greenlet.getcurrent().parent.switch("from worker")
67+
return "done"
68+
69+
g = greenlet.greenlet(worker)
70+
result = g.switch()
71+
assert result == "from worker", result
72+
print("OK: exiting with active greenlet")
73+
""")
74+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
75+
self.assertIn("OK: exiting with active greenlet", stdout)
76+
77+
def test_multiple_active_greenlets_at_shutdown(self):
78+
"""
79+
Multiple suspended greenlets at shutdown should all be cleaned
80+
up without crashing.
81+
"""
82+
rc, stdout, stderr = self._run_shutdown_script("""\
83+
import greenlet
84+
85+
def worker(name):
86+
greenlet.getcurrent().parent.switch(f"hello from {name}")
87+
return "done"
88+
89+
greenlets = []
90+
for i in range(10):
91+
g = greenlet.greenlet(worker)
92+
result = g.switch(f"g{i}")
93+
greenlets.append(g)
94+
95+
print(f"OK: {len(greenlets)} active greenlets at shutdown")
96+
""")
97+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
98+
self.assertIn("OK: 10 active greenlets at shutdown", stdout)
99+
100+
def test_nested_greenlets_at_shutdown(self):
101+
"""
102+
Nested (chained parent) greenlets at shutdown should not crash.
103+
"""
104+
rc, stdout, stderr = self._run_shutdown_script("""\
105+
import greenlet
106+
107+
def inner():
108+
greenlet.getcurrent().parent.switch("inner done")
109+
110+
def outer():
111+
g_inner = greenlet.greenlet(inner)
112+
g_inner.switch()
113+
greenlet.getcurrent().parent.switch("outer done")
114+
115+
g = greenlet.greenlet(outer)
116+
result = g.switch()
117+
assert result == "outer done", result
118+
print("OK: nested greenlets at shutdown")
119+
""")
120+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
121+
self.assertIn("OK: nested greenlets at shutdown", stdout)
122+
123+
def test_threaded_greenlets_at_shutdown(self):
124+
"""
125+
Greenlets in worker threads that are still referenced at
126+
shutdown should not crash.
127+
"""
128+
rc, stdout, stderr = self._run_shutdown_script("""\
129+
import greenlet
130+
import threading
131+
132+
results = []
133+
134+
def thread_worker():
135+
def greenlet_func():
136+
greenlet.getcurrent().parent.switch("from thread greenlet")
137+
return "done"
138+
139+
g = greenlet.greenlet(greenlet_func)
140+
val = g.switch()
141+
results.append((g, val))
142+
143+
threads = []
144+
for _ in range(3):
145+
t = threading.Thread(target=thread_worker)
146+
t.start()
147+
threads.append(t)
148+
149+
for t in threads:
150+
t.join()
151+
152+
print(f"OK: {len(results)} threaded greenlets at shutdown")
153+
""")
154+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
155+
self.assertIn("OK: 3 threaded greenlets at shutdown", stdout)
156+
157+
# -----------------------------------------------------------------
158+
# Behavioral tests: verify cleanup semantics per Python version
159+
# -----------------------------------------------------------------
160+
161+
def test_greenlet_cleanup_runs_on_supported_python(self):
162+
"""
163+
On Python >= 3.11, GreenletExit IS thrown into active greenlets
164+
during shutdown, so cleanup code (try/except/finally) runs.
165+
166+
On Python < 3.11, the greenlet is killed in place without
167+
throwing GreenletExit (to avoid the crash), so cleanup code
168+
does NOT run — this is the expected trade-off.
169+
170+
This test documents and verifies both behaviors.
171+
"""
172+
rc, stdout, stderr = self._run_shutdown_script("""\
173+
import sys
174+
import greenlet
175+
176+
cleanup_ran = False
177+
178+
def worker():
179+
try:
180+
greenlet.getcurrent().parent.switch("suspended")
181+
except greenlet.GreenletExit:
182+
# This runs only if GreenletExit is thrown into us
183+
# (which requires g_switch, safe only on 3.11+)
184+
print("CLEANUP: GreenletExit caught")
185+
raise # re-raise to finish normally
186+
187+
g = greenlet.greenlet(worker)
188+
result = g.switch()
189+
assert result == "suspended"
190+
print("OK: about to shut down")
191+
# g is active; interpreter shutdown will dealloc it
192+
""")
193+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
194+
self.assertIn("OK: about to shut down", stdout)
195+
196+
if PY311:
197+
# On 3.11+, g_switch works during finalization, so
198+
# GreenletExit is thrown and the cleanup code runs.
199+
self.assertIn("CLEANUP: GreenletExit caught", stdout)
200+
else:
201+
# On < 3.11, murder_in_place() is used to avoid the crash.
202+
# The greenlet is killed without throwing, so cleanup
203+
# code does NOT run. This is the safe trade-off.
204+
self.assertNotIn("CLEANUP: GreenletExit caught", stdout)
205+
206+
def test_finally_block_during_shutdown(self):
207+
"""
208+
Verify try/finally behavior during interpreter shutdown.
209+
210+
On Python >= 3.11, finally blocks in active greenlets run.
211+
On Python < 3.11, they don't (murder_in_place skips them).
212+
"""
213+
rc, stdout, stderr = self._run_shutdown_script("""\
214+
import sys
215+
import greenlet
216+
217+
def worker():
218+
try:
219+
greenlet.getcurrent().parent.switch("suspended")
220+
finally:
221+
print("FINALLY: cleanup executed")
222+
223+
g = greenlet.greenlet(worker)
224+
result = g.switch()
225+
assert result == "suspended"
226+
print("OK: about to shut down")
227+
""")
228+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
229+
self.assertIn("OK: about to shut down", stdout)
230+
231+
if PY311:
232+
self.assertIn("FINALLY: cleanup executed", stdout)
233+
else:
234+
self.assertNotIn("FINALLY: cleanup executed", stdout)
235+
236+
def test_many_greenlets_with_cleanup_at_shutdown(self):
237+
"""
238+
Stress test: many active greenlets with cleanup code at shutdown.
239+
Ensures no crashes regardless of deallocation order.
240+
"""
241+
rc, stdout, stderr = self._run_shutdown_script("""\
242+
import sys
243+
import greenlet
244+
245+
cleanup_count = 0
246+
247+
def worker(idx):
248+
global cleanup_count
249+
try:
250+
greenlet.getcurrent().parent.switch(f"ready-{idx}")
251+
except greenlet.GreenletExit:
252+
cleanup_count += 1
253+
raise
254+
255+
greenlets = []
256+
for i in range(50):
257+
g = greenlet.greenlet(worker)
258+
result = g.switch(i)
259+
greenlets.append(g)
260+
261+
print(f"OK: {len(greenlets)} greenlets about to shut down")
262+
# Note: we can't easily print cleanup_count during shutdown
263+
# since it happens after the main module's code runs.
264+
""")
265+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
266+
self.assertIn("OK: 50 greenlets about to shut down", stdout)
267+
268+
def test_deeply_nested_greenlets_at_shutdown(self):
269+
"""
270+
Deeply nested greenlet parent chains at shutdown.
271+
Tests that the deallocation order doesn't cause issues.
272+
"""
273+
rc, stdout, stderr = self._run_shutdown_script("""\
274+
import greenlet
275+
276+
def level(depth, max_depth):
277+
if depth < max_depth:
278+
g = greenlet.greenlet(level)
279+
g.switch(depth + 1, max_depth)
280+
greenlet.getcurrent().parent.switch(f"depth-{depth}")
281+
282+
g = greenlet.greenlet(level)
283+
result = g.switch(0, 10)
284+
print(f"OK: nested to depth 10, got {result}")
285+
""")
286+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
287+
self.assertIn("OK: nested to depth 10", stdout)
288+
289+
def test_greenlet_with_traceback_at_shutdown(self):
290+
"""
291+
A greenlet that has an active exception context when it's
292+
suspended should not crash during shutdown cleanup.
293+
"""
294+
rc, stdout, stderr = self._run_shutdown_script("""\
295+
import greenlet
296+
297+
def worker():
298+
try:
299+
raise ValueError("test error")
300+
except ValueError:
301+
# Suspend while an exception is active on the stack
302+
greenlet.getcurrent().parent.switch("suspended with exc")
303+
return "done"
304+
305+
g = greenlet.greenlet(worker)
306+
result = g.switch()
307+
assert result == "suspended with exc"
308+
print("OK: greenlet with active exception at shutdown")
309+
""")
310+
self.assertEqual(rc, 0, f"Process crashed (rc={rc}):\n{stdout}{stderr}")
311+
self.assertIn("OK: greenlet with active exception at shutdown", stdout)
312+
313+
314+
if __name__ == '__main__':
315+
unittest.main()

0 commit comments

Comments
 (0)