Force transition instances in inconsistent state#618
Merged
gabriel-samfira merged 1 commit intocloudbase:mainfrom Feb 11, 2026
Merged
Force transition instances in inconsistent state#618gabriel-samfira merged 1 commit intocloudbase:mainfrom
gabriel-samfira merged 1 commit intocloudbase:mainfrom
Conversation
If GARM is killed or restarted while creating a runner, there is a chance that runners remain in creating or deleting state. We've started checking state transitions in GARM and allow a transition when the new state makes sense in normal circumstances. However, when recovering from a crash, we may be in an inconsisten state from which we need to recover. This change added a ForceUpdateInstance() function that ignores state transition inconsistencies. For now, we only use it when spinning up a scale set and check for instance states. This change also fixes a locking issue. Signed-off-by: Gabriel Adrian Samfira <gsamfira@cloudbasesolutions.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
If GARM is killed or restarted while creating a runner, there is a chance that runners remain in creating or deleting state. We've started checking state transitions in GARM and allow a transition when the new state makes sense in normal circumstances. However, when recovering from a crash, we may be in an inconsisten state from which we need to recover.
This change added a ForceUpdateInstance() function that ignores state transition inconsistencies. For now, we only use it when spinning up a scale set and check for instance states.
This change also fixes a locking issue.
Fixes: #610
Includes: #598