Meta-Issue: Mutli-threading leads to segfaults

[axsk]

This seems to be coming up quite often, in order to track this issue I took the liberty to create this meta-issue linking all other related issues I could find:

#1095
#1090
#1088
#1083
#1078
#1077
#1072

Affected repos:
timholy/Revise.jl#989

Possible also related (crashes/segfaults)
#1082
#1080

[axsk]

Seeing that the last change to this repo is older then two years, is it correct to assume it has become unmaintained and no fix is to be expected?

[topolarity]

Reproducer:

using Base.Threads
using PyCall
o = PyObject(randn(3,3))
t = Threads.@spawn begin
    println(Core.stderr, propertynames(o))
end
wait(t)

$ julia +1.12 --project=. mwe.jl

[119772] signal 11 (1): Segmentation fault
in expression starting at /home/topolarity/repos/claude/PyCall.jl/mwe.jl:7
_PyType_Lookup at /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0 (unknown line)
_PyObject_LookupSpecial at /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0 (unknown line)
PyObject_Dir at /usr/lib/x86_64-linux-gnu/libpython3.12.so.1.0 (unknown line)

The issue seems to be that PyCall.jl does not manage thread states or the GIL: https://docs.python.org/3/c-api/threads.html

Here's a "fixed" version of the example:

using Base.Threads
using PyCall
ccall(:PyEval_InitThreads, Cvoid, ()) # enable threads / GIL
o = PyObject(randn(3,3))
tstate = ccall(:PyEval_SaveThread, Ptr{Cvoid}, ()) # Release GIL

t = Threads.@spawn begin
    GIL = ccall(:PyGILState_Ensure, Ptr{Cvoid}, ())
    println(Core.stderr, propertynames(o))
    ccall(:PyGILState_Release, Cvoid, (Ptr{Cvoid},), GIL)
end
wait(t)

ccall(:PyEval_RestoreThread, Cvoid, (Ptr{Cvoid},), tstate) # Re-acquire GIL

[topolarity]

Equivalent fix in PythonCall.jl: JuliaPy/PythonCall.jl#677

Looks like they opt to leave the GIL-management to users for most of the API. For propertynames (which is called by the REPL tab-complete), they provide a mechanism to request execution on the main thread.