feat: add early exit for concurrency #242
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Add OrphanedChildException (BaseException) to terminate orphaned children when parent completes early - Modify ThreadPoolExecutor to shutdown without waiting (wait=False) when completion criteria met - Raise exception when orphaned children attempt to checkpoint, preventing subsequent operations from executing - Update state.py to reject orphaned child checkpoints with exception instead of silent return - Add comprehensive tests for early exit behavior and orphaned child handling When min_successful or error threshold is reached in parallel/map operations, the parent now returns immediately without waiting for remaining branches to complete. Orphaned branches are terminated on their next checkpoint attempt, preventing wasted work and ensuring correct semantics for completion criteria.
wangyb-A
approved these changes
Dec 9, 2025
wangyb-A
pushed a commit
to aws/aws-durable-execution-sdk-python-testing
that referenced
this pull request
Dec 9, 2025
- Update test case after we add early exit for concurrency - aws/aws-durable-execution-sdk-python#242
wangyb-A
pushed a commit
to aws/aws-durable-execution-sdk-python-testing
that referenced
this pull request
Dec 9, 2025
- Update test case after we add early exit for concurrency - aws/aws-durable-execution-sdk-python#242
wangyb-A
pushed a commit
to aws/aws-durable-execution-sdk-python-testing
that referenced
this pull request
Dec 9, 2025
- Update test case after we add early exit for concurrency - aws/aws-durable-execution-sdk-python#242
ghost
reviewed
Dec 9, 2025
ghost
left a comment
ghost
left a comment
There was a problem hiding this comment.
How does this work with at most once semantics?
|
Okay at most once is preserved because on termination we store any running threads as started, and on replay we won't run them again. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available:
Description of changes:
When min_successful or error threshold is reached in parallel/map operations, the parent now returns immediately without waiting for remaining branches to complete. Orphaned branches are terminated on their next checkpoint attempt, preventing wasted work and ensuring correct semantics for completion criteria.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.