Skip to content

[CALCITE-7405] Pre-process expressions for correlations before building projection in SqlToRelConverter#4779

Open
ian-bertolacci wants to merge 1 commit intoapache:mainfrom
ian-bertolacci:CALCITE-7405-use-correct-input-for-convertNonAggregateSelectList
Open

[CALCITE-7405] Pre-process expressions for correlations before building projection in SqlToRelConverter#4779
ian-bertolacci wants to merge 1 commit intoapache:mainfrom
ian-bertolacci:CALCITE-7405-use-correct-input-for-convertNonAggregateSelectList

Conversation

@ian-bertolacci
Copy link

@ian-bertolacci ian-bertolacci commented Jan 28, 2026

To circumvent the bloat optimizations, it is necessary to provide the correlation variables (if they exist) to the builder.
The main change here is to accomplish the logic done in SqlToRelConverter.getCorrelations without having built the Project node.
In this way we (a) extract the correlation id to use for the RelBuilder, and (b) normalize/resolve the correlation ids and rewrite the expressions; all before invoking the RelBuilder.

I've done my best to reuse existing logic. SqlToRelConverter.getCorrelations and SqlToRelConverter.massageExpressionsForCorrelation require a lot of the same logic for resolving the correlation variables for the current scope, so that was moved to a common utility method (SqlToRelConverter.getCorrelationInfo).

Copy link
Member

@xuzifu666 xuzifu666 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to fix CI error first.

/.vscode/*

# IDEA
/out
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why change .gitignore file?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When I run with intellij, git detected a ton of extra files.

@ian-bertolacci ian-bertolacci force-pushed the CALCITE-7405-use-correct-input-for-convertNonAggregateSelectList branch 2 times, most recently from 6338b8f to 3784f65 Compare January 29, 2026 23:42
…ng projection in SqlToRelConverter, simplify some gitignore paths
@ian-bertolacci ian-bertolacci force-pushed the CALCITE-7405-use-correct-input-for-convertNonAggregateSelectList branch from 3784f65 to 5314e24 Compare January 30, 2026 00:01
@sonarqubecloud
Copy link

@ian-bertolacci ian-bertolacci changed the title [CALCITE-7405] Use working Project input instead of Blackboard root when rebuilding Project for correlated subqueries [CALCITE-7405] Pre-process expressions for correlations before building projection in SqlToRelConverter Jan 30, 2026
@silundong
Copy link
Contributor

If we create a temporary Project with LogicalProject.create before correlation detection, instead of using RelBuilder.projectNamed, would that resolve the issue? That could be much simpler.

@ian-bertolacci
Copy link
Author

@silundong

If we create a temporary Project with LogicalProject.create before correlation detection, instead of using RelBuilder.projectNamed, would that resolve the issue? That could be much simpler.

Thats not an unreasonable suggestion, however I expect that change would also create a lot of test changes, both in this project and anything downstream.
I'll check and see (though it'll take me a few days).

@xiedeyantu
Copy link
Member

I prefer a simpler, more intuitive processing logic, even if it affects some of the original plans. As you mentioned, are there other places where similar issues arise, and would it be better to have a unified logic for handling them?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants