[FLINK-38539][table] Fix integer overflow in AbstractBytesMultiMap wh…#27510
Open
nateab wants to merge 2 commits intoapache:masterfrom
Open
[FLINK-38539][table] Fix integer overflow in AbstractBytesMultiMap wh…#27510nateab wants to merge 2 commits intoapache:masterfrom
nateab wants to merge 2 commits intoapache:masterfrom
Conversation
Collaborator
e235731 to
cbcb3fa
Compare
davidradl
reviewed
Feb 3, 2026
| * handling path works correctly when the map is filled to capacity. | ||
| */ | ||
| @Test | ||
| void testThrowsEOFExceptionWhenFull() throws Exception { |
Contributor
There was a problem hiding this comment.
Can we have a test with exactly the maximum, one under and one over (or as close as is feasible with the 4 byte pointer) to make sure that the boundary is exactly policed as expected please.
Contributor
Author
There was a problem hiding this comment.
Sure I've added flink-table/flink-table-runtime/src/test/java/org/apache/flink/table/runtime/util/collections/binary/AbstractBytesMultiMapBoundaryTest.java to test this now!
…en data exceeds 2GB This fixes an IndexOutOfBoundsException in AbstractBytesMultiMap that occurs when window aggregate queries process large amounts of data with the memory state backend. When the key/value area exceeds ~2GB (Integer.MAX_VALUE bytes), casting the long offset to int overflows to a negative value, causing invalid array access. The fix adds bounds checking in writePointer(), skipPointer(), and appendRecord() to validate offsets before casting long to int, ensuring EOFException is thrown with a clear warning message before any overflow can occur. Changes: - Added bounds checking in writePointer() to validate offset before casting - Added bounds checking in skipPointer() to validate offset before casting - Added bounds checking in appendRecord() to validate key area offset - Added AbstractBytesMultiMapBoundaryTest with boundary tests at Integer.MAX_VALUE: - Test at MAX - 4 (should succeed, pointer fits) - Test at MAX - 3 with alignment skip (should succeed) - Test at exactly MAX (should succeed, boundary is >) - Test at MAX + 1 (should throw EOFException) - Test at 2*MAX (should throw EOFException) - Added testThrowsEOFExceptionWhenFull() to BytesMultiMapTestBase
cbcb3fa to
2ff9b1e
Compare
… reflection The previous test replicated the overflow check logic locally instead of calling the actual writePointer/skipPointer methods, which meant it passed regardless of whether the production fix was present. Rewrite to use reflection to invoke the real AbstractBytesMultiMap methods, properly verifying the overflow protection works.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…en data exceeds 2GB
What is the purpose of the change
This pull request fixes an IndexOutOfBoundsException in AbstractBytesMultiMap that occurs when window aggregate queries (using TUMBLE TVF with LAST_VALUE aggregations) process large amounts of data with the
memory state backend. When the value/key area exceeds ~2GB (Integer.MAX_VALUE bytes), casting the long offset to int overflows to a negative value, causing invalid array access.
Brief change log
Verifying this change
This change added tests and can be verified as follows:
The fix adds validation that throws EOFException earlier in the code path. The existing EOFException handling in RecordsWindowBuffer.addElement() already properly handles this case by flushing the buffer and
retrying.
##Does this pull request potentially affect one of the following parts:
Documentation