Core: Fix identity transform's predicate projection#15665
Closed
ldudas-marx wants to merge 32 commits intoapache:mainfrom
Closed
Core: Fix identity transform's predicate projection#15665ldudas-marx wants to merge 32 commits intoapache:mainfrom
ldudas-marx wants to merge 32 commits intoapache:mainfrom
Conversation
The Identity transformation projection functions left out the predicate's term part if it was not a reference. This lead to type mismatch errors or incorrect results. In this case unfortunately we cannot safely project the term part as not all visitors support non-reference terms. Refactored TestProjection tests to use the ParameterizedTest feature for better visibility. Closes apache#15502 Co-authored-by: Claude (claude-sonnet-4.6) <noreply@anthropic.com>
Contributor
|
@manuzhang @nastra can you trigger the CI? I understand it doesn't get triggered for new contributors? |
…sm and distributionMode (apache#15740)
…eParallelism and distributionMode (apache#15762)
Co-authored-by: Codex <codex@openai.com> Co-authored-by: Maximilian Michels <mxm@apache.org>
…hotProducer (apache#15653) * Add manifest partition pruning to DV validation
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 7.3.1 to 7.6.0. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@5a095e7...37802ad) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 7.6.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…che#15798) Three test methods in TestReplacePartitions operate on separate unpartitioned or all-void tables but were passing the wrong table instance to commit() and validation helpers: - testReplaceWithUnpartitionedTable used commit(table, ...) instead of commit(unpartitioned, ...) for operations on the unpartitioned table - testReplaceAndMergeWithUnpartitionedTable had the same issue - validateSnapshot and validateManifestEntries were called without specifying which table to use for reading manifests, defaulting to the partitioned test table instead of the actual target table The commit(table, ...) bug was introduced in apache#6650 when the test was refactored from direct toBranch().commit() calls to use the commit() helper. The validation calls for the all-void unpartitioned table (added in apache#14186) also used the default overloads that read manifests via the wrong table's specs. This adds Table-accepting overloads for validateSnapshot and validateManifestEntries in TestBase and updates the affected tests to pass the correct table instance.
…#15804) The 3-arg VectorHolder constructor (FieldVector, NestedField, NullabilityHolder) was package-private but all callers are within the same top-level class (PositionVectorHolder inner class and vectorHolder() factory method). Since Java static inner classes can access private members of the enclosing class, this constructor can safely be made private to reduce its visibility scope.
…15748) * [Core] Fix NPE of generateRandomMetrics in FileGenerationUtil * Address review feedback
…ing to 2.0.3 (apache#15440) * Bump aircompressor to 2.0.3 globally to fix CVE-2025-67721 Update the central version in libs.versions.toml from 0.27 to 2.0.3 and update the version reference in open-api/LICENSE. * Enforce aircompressor 2.0.3 globally via dependency substitution Add a substitute rule in build.gradle to enforce aircompressor 2.0.3 across all modules, following the same pattern used for lz4-java.
…5823) * Remove Spark 2 test assumptions for write projection * apply formatting * remove dead code for spark 2 * revert partial column fix for 3.4, 3.5 and 4.0 * remove spark 2 testing in 3.4, 3.5 and 4.0
flinkVersions=1.14 is End of Life and was removed from the Iceberg build. Any new contributor copying this command gets a hard Gradle exception. Update to current supported versions (1.20, 2.0) and update Spark versions to more current ones (3.5, 4.0).
Author
|
I messed up the git commands I close this and open a new PR |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Identity transformation projection functions left out the predicate's term part if it was not a reference. This lead to type mismatch errors or incorrect results. In this case unfortunately we cannot safely project the term part as not all visitors support non-reference terms.
Refactored TestProjection tests to use the ParameterizedTest feature for better visibility.
Closes #15502
Coded with the help of Github Copilot and claude-sonnet-4.6 (code completion)