@@ -76,19 +76,20 @@ the magic immortal number. But again, this isn't totally safe; for example, due
7676to the current string interning implementation, immortalizing a string can cause
7777crashes. To make matters worse, there's no mechanism in place to deallocate an
7878object once it has become immortal; this is limiting for potential optimizations
79- using immortality.
79+ using immortality, because immortal objects stored on the heap (and the memory
80+ that they own) are leaked.
8081
8182What Problem Does Immortality Solve?
8283************************************
8384
8485Currently, the main way to share objects between subinterpreters is via
8586serialization. This works OK, but can be limited for many objects.
86- Immortality is a good precursor to directly sharing objects across multiple
87+ Immortality is a necessary precursor to directly sharing objects across multiple
8788interpreters, as it provides a way to make reference counting thread-safe
8889without atomic operations on the reference count field (which has proven to
8990drastically hurt performance).
9091
91- Subinterpreters aside, immortality is just generally a powerful tool to wield ,
92+ Subinterpreters aside, immortality is just generally a powerful tool,
9293especially for CPython maintainers. The authors of this PEP do not expect that
9394immortality will become a common tool for Python code, but the complexity of
9495the approach on its own warrants a PEP.
@@ -100,10 +101,12 @@ other areas of CPython, or in third-party libraries:
100101 build are still subject to reference count contention. It is possible to help
101102 mitigate this problem through deferred reference counting in the C API
102103 (:c:func: `PyUnstable_Object_EnableDeferredRefcount `), but that has the downside
103- of damaging single-threaded performance.
104+ of damaging single-threaded performance. Immortality is able to avoid reference
105+ count contention while also keeping good single-threaded performance.
104106- Immortality can help memory usage by avoiding copy-on-write operations in
105107 child processes. Instagram has been `doing this in Cinder `_ for years,
106- and it would be nice to let CPython also take advantage of it.
108+ and it would be nice to let CPython also take advantage of it. This is
109+ mainly an incidental goal of this PEP.
107110
108111.. _doing this in Cinder : https://engineering.fb.com/2023/08/15/developer-tools/immortal-objects-for-python-instagram-meta/
109112
@@ -113,10 +116,10 @@ Immortal Objects Must Remain in CPython
113116
114117In the past, the main pushback to exposing APIs for immortality is that
115118it's exposing an implementation detail, and would force CPython to keep
116- immortality forever. Unfortunately, we've already reached that point: too
117- many users, and `CPython itself `_, omit reference counting for known immortal
118- objects, such as :c:data: `Py_None `. Since there's no good way to deprecate
119- that, CPython will always need some form of immortality to retain
119+ immortality forever. Unfortunately, it's likely that point has already been
120+ reached: too many users, and `CPython itself `_, omit reference counting for
121+ known immortal objects, such as :c:data: `Py_None `. Since there's no good way
122+ to deprecate that, CPython will always need some form of immortality to retain
120123compatibility. That said, this proposal still keeps all new APIs unstable.
121124
122125.. _CPython itself : https://github.com/python/cpython/issues/103906
@@ -132,8 +135,9 @@ which will be discussed further in a moment. As of writing, the rule
132135is that you cannot modify the reference count of an object created
133136in a different interpreter, and since you can't modify the reference
134137count, you can't do anything with the object. Since immortality prevents
135- reference counting operations, this PEP hopes to solve that issue, allowing
136- for an object proxy that can safely be reference counted in all interpreters.
138+ reference counting operations, this PEP will solve that issue, allowing
139+ for an object proxy that can safely be used with reference counting APIs
140+ in all interpreters.
137141
138142Reference Count Modifications Must be Per-interpreter
139143-----------------------------------------------------
@@ -143,7 +147,7 @@ by the interpreter's GIL, meaning that trying to modify the reference count of
143147an object that belongs to another interpreter could cause data races.
144148
145149This isn't a problem on the free-threaded build as reference count operations
146- are atomic, but subinterpreters are supported on the with- GIL build. An
150+ are atomic, but subinterpreters are supported on the GIL-enabled build. An
147151alternative option could be to make reference counting also atomic on the
148152GIL-enabled build, but that has been shown to damage single-threaded performance
149153or compromise compatibility with the stable ABI.
@@ -169,6 +173,16 @@ In doing so, an object will release its memory back to CPython's object allocato
169173allocator is per-interpreter, so it is unsafe to try to release an object's
170174memory in an interpreter different from the one that created it.
171175
176+ Immortality Has No Impact on Reference Counting Performance
177+ -----------------------------------------------------------
178+
179+ If the primary issue with cross-interpreter reference counting is thread-safety,
180+ why not make reference counting atomic? Unfortunately, atomic operations are
181+ slower than their non-atomic counterparts, so this would hurt overalll
182+ performance. :pep: `703 ` managed to solve this issue using biased reference
183+ counting, splitting the reference count field into two, but at the cost of
184+ breaking the stable C API. Atomic reference counting on its own also doesn't
185+ solve the per-interpreter deallocator issue as mentioned previously.
172186
173187Immortality is a Powerful Tool to Wield
174188***************************************
@@ -604,7 +618,7 @@ Backwards Compatibility
604618The `Immortalization Contract `_ is very slightly backwards-incompatible.
605619The contract is based on best practices in the C API; it doesn't have
606620anything that's really new, it's just based around what objects currently
607- do. That being said, there was nothing enforcing before that objects follow
621+ do. That being said, there was nothing previously enforcing that objects follow
608622these assumptions, but the authors of this PEP have never seen code in the
609623wild that breaks the immortalization contract.
610624
0 commit comments