Parent: #3980
Scenario
A consumer process is killed in a way that bypasses the terminate/2 callback:
:kill signal: Even with Process.flag(:trap_exit, true) (consumer.ex:108), a :kill signal terminates the process immediately without calling terminate. From Erlang docs: "If the process receives a kill signal, it terminates, regardless of the trap_exit flag."
:brutal_kill supervisor shutdown: If a supervisor is configured with shutdown: :brutal_kill, child processes receive :kill.
What happens
- A transaction arrives.
ShapeLogCollector.publish → ConsumerRegistry.publish → broadcast delivers the event to the consumer.
- The consumer processes the event, replies
:ok. FlushTracker.handle_txn_fragment records the shape in last_flushed and min_incomplete_flush_tree.
- The consumer is killed (
:kill signal, etc.) before the storage flush callback fires and notify_flushed is called.
terminate/2 does NOT run. No cleanup happens. No handle_writer_termination, no remove_shape_async, no FlushTracker.handle_shape_removed.
- The shape's entry in FlushTracker becomes the permanent minimum, blocking
last_global_flushed_offset from advancing.
Fix
This scenario can only be addressed by an active detection mechanism in ShapeLogCollector (see parent issue #3980). The terminate callback path is insufficient by definition — no amount of improvement to terminate or handle_writer_termination can help when the callback doesn't execute.
Parent: #3980
Scenario
A consumer process is killed in a way that bypasses the
terminate/2callback::killsignal: Even withProcess.flag(:trap_exit, true)(consumer.ex:108), a:killsignal terminates the process immediately without callingterminate. From Erlang docs: "If the process receives a kill signal, it terminates, regardless of the trap_exit flag.":brutal_killsupervisor shutdown: If a supervisor is configured withshutdown: :brutal_kill, child processes receive:kill.What happens
ShapeLogCollector.publish→ConsumerRegistry.publish→broadcastdelivers the event to the consumer.:ok.FlushTracker.handle_txn_fragmentrecords the shape inlast_flushedandmin_incomplete_flush_tree.:killsignal, etc.) before the storage flush callback fires andnotify_flushedis called.terminate/2does NOT run. No cleanup happens. Nohandle_writer_termination, noremove_shape_async, noFlushTracker.handle_shape_removed.last_global_flushed_offsetfrom advancing.Fix
This scenario can only be addressed by an active detection mechanism in ShapeLogCollector (see parent issue #3980). The terminate callback path is insufficient by definition — no amount of improvement to
terminateorhandle_writer_terminationcan help when the callback doesn't execute.