Skip to content

fix missing window expressions when unparsing plans without outer projections#21801

Merged
kosiew merged 5 commits into
apache:mainfrom
nathanb9:fix/window-unparser-without-outer-projection
May 15, 2026
Merged

fix missing window expressions when unparsing plans without outer projections#21801
kosiew merged 5 commits into
apache:mainfrom
nathanb9:fix/window-unparser-without-outer-projection

Conversation

@nathanb9
Copy link
Copy Markdown
Contributor

@nathanb9 nathanb9 commented Apr 23, 2026

Summary

Addressing: #21746

Fixes a problem withplan_to_sql where window expressions could be dropped when unparsing plans that have a Window node without an outer Projection

PR solution

When unparsing a LogicalPlan::Window, if no projection has already reconstructed the output, the unparser now appends the window expressions itself.

This preserves window outputs for plans like:

  • Window(TableScan)
  • Window(Aggregate(TableScan))
  • Basically stacked Window nodes without an outer projection

Additionally:

  • DISTINCT, LIMIT, SORT, UNION, and child Projection boundaries are preserved by deriving the window input before appending window output.
  • Window expressions over derived inputs are rewritten to reference the derived input alias.
  • Aggregate unprojection now stops at Subquery / SubqueryAlias boundaries, so outer windows reference derived columns instead of incorrectly looking through the subquery.

@github-actions github-actions Bot added the sql SQL Planner label Apr 23, 2026
@nathanb9 nathanb9 changed the title preserve window expressions when unparsing plans without outer projections fix missing window expressions when unparsing plans without outer projections Apr 23, 2026
@nathanb9 nathanb9 marked this pull request as ready for review April 23, 2026 13:32
Copy link
Copy Markdown
Contributor

@kosiew kosiew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nathanb9
Thanks for working on this.
Looks 👍 to me

.alias("row_idx");
let plan = table_scan(Some("gas"), &schema, None)?
.aggregate(vec![col("time")], vec![sum(col("value")).alias("sum_n")])?
.window(vec![window_expr])?
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried to access the row_idx column here in a filter just by inserting this expression:

.filter(col("row_idx").eq(lit(0i64)))?

and the unparser fails with this error: Error: Internal("Tried to unproject agg expr for column 'row_idx' that was not found in the provided Aggregate!")

when accessing the "sum_n" column there is no problem.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea makes sense just fixed:

  • For Filter nodes, the unparser now checks for windows first when the dialect supports QUALIFY.
  • If there is also an aggregate underneath, aggregate aliases inside the QUALIFY expression are still unprojected correctly.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@PiotrSrebrny
Does 5e8b64a fix your issue?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I tested on my side and it fixes the issue. Thank you so much!

Copy link
Copy Markdown

@PiotrSrebrny PiotrSrebrny left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works pretty well, but there is still some issue. Please have a look at my comment.

@kosiew kosiew added this pull request to the merge queue May 15, 2026
Merged via the queue into apache:main with commit 3cba937 May 15, 2026
60 of 61 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

sql SQL Planner

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants