From f6941f67f3b50147831d66fd51060f6907d191a9 Mon Sep 17 00:00:00 2001 From: Ben Nemec Date: Fri, 7 Nov 2025 14:14:05 -0600 Subject: [PATCH] OCPBUGS-59925: Fix keepalived SIGTERM handling Previously the SIGTERM that is sent when kubelet shuts down the pod was being ignored, which caused keepalived to be SIGKILLed and not able to send the priority 0 message so another node takes the VIP immediately. This caused ~5 seconds of delay when keepalived and haproxy were restarted during upgrades, and if this happened on a node where the local apiserver is also unavailable it causes a temporary API outage. There appears to have been two reasons for this: 1) The socat call blocks SIGTERM handling in the bash script 2) Sending SIGTERM to keepalived without waiting for it to complete can cause the container to exit before priority 0 is sent Using the "wait" command seems to make this work as expected. Now socat is started in the background and wait is used to keep the script from exiting. A delay is also added to allow keepalived time to shut down cleanly. Timeouts are not needed for any of these calls because kubelet will already send SIGKILL after the grace period expires. --- templates/common/on-prem/files/keepalived.yaml | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/templates/common/on-prem/files/keepalived.yaml b/templates/common/on-prem/files/keepalived.yaml index 36568e4e6b..8e898de542 100644 --- a/templates/common/on-prem/files/keepalived.yaml +++ b/templates/common/on-prem/files/keepalived.yaml @@ -87,6 +87,8 @@ contents: { if pid=$(pgrep -o keepalived); then kill -s SIGTERM "$pid" + # Give keepalived time to shut down + while pgrep -o keepalived; do sleep 1; done fi } @@ -146,7 +148,8 @@ contents: fi rm -f "$keepalived_sock" - socat UNIX-LISTEN:${keepalived_sock},fork system:'bash -c msg_handler' + socat UNIX-LISTEN:${keepalived_sock},fork system:'bash -c msg_handler' & + wait resources: requests: cpu: 100m