Add Free-Threaded Python Support in 3.14

[mkmoisen]

With the release of Python 3.14, free threaded support (i.e., GIL-less python) is now a sensible option.

Third party packages that use C-Extensions like confluent_kafka however need to have code change to support free-threaded mode.

Here is the documentation on how to change C-Extensions to support free threaded mode:

https://docs.python.org/3.14/howto/free-threading-extensions.html#freethreading-extensions-howto

My understanding is that if a third party application does not support free-threaded python, then applications which import these applications will revert to using the GIL. This means that users are stuck with the GIL until all third party applications have ported to free-threaded compatibility.

May we please put this on the road map for confluent_kafka?

Thank you.

[MSeal]

Thanks for raising the issue Matthew Moisen (@mkmoisen) -- I've been a preliminary pass of the C code change requirements. It seems like the code changes will be minimal but testing this will require a bit of preparation work to ensure no deadlocks of segfaults are introduced against the librdkafka layer. I've added it to the roadmap, and will discuss with the team next week.

[cclauss]

Is this fixed in the new v2.12.1 release?

• https://github.com/confluentinc/confluent-kafka-python/releases

[dpdani]

As additional documentation, you can also take a look at https://py-free-threading.github.io/porting-extensions/

Apart from that page in particular, you may also find the rest of the site helpful.

I reckon the underlying librdkafka library is thread-safe, so most of the problems may arise from possible concurrent mutations on the Handle object, which should be solvable with the new critical section API.

I don't know a lot about the internals of this library but I do know free threading quite well, let me know if I can be of help in any way 🙏

[MSeal]

Christian Clauss (@cclauss) no, there is a wheel compiled for free-threading but the C-bindings aren't aware of this yet. Daniele Parmeggiani (@dpdani) if you want to take a pass on updating the C-bindings, we're happy to have the contributions. The librdkafka is thread safe outside of some bug reports on specific call sequences that might need a patch, but testing here is the biggest concern as we don't have many multi-thread test examples at the Python level to feel confident we didn't miss anything.

[dpdani]

Matthew Seal (@MSeal) sorry, didn't mean to disappear. I haven't yet found the time to go over this and I don't think I will in the near future.
If you have any specific questions about free threading, feel free to ask!

[robotechdy]

Hi Matthew Seal (@MSeal), thanks for looking into this.

confluent-kafka-python is a key dependency for many of us, so free-threaded support in 3.14 matters a lot—otherwise import forces a GIL fallback.

Is there a rough plan/timeline for getting the C bindings “free-threading aware”? If it helps, I can contribute a small threaded repro/test (produce+poll, poll+close, callbacks under load) to reduce the risk of deadlocks/segfaults.

[ojasvajain]

We have created ticket 4251 to track this internally.

[robotechdy]

hello, Ojasva Jain (@ojasvajain) is there any update for this issue? :)