[fix](prune nested column) fix prune nested column maybe throw NullPointerException by 924060929 · Pull Request #60395 · apache/doris

924060929 · 2026-01-30T11:08:04Z

What problem does this PR solve?

fix prune nested column maybe throw NullPointerException, introduced by #58370

select k1 as k2 -- can not find s.k2, we should replace k2 to k1
from
(
  select struct_element(s, 'k1') k1
  from tbl
)t

java.lang.NullPointerException: Cannot invoke "org.apache.doris.nereids.rules.rewrite.NestedColumnPruning$DataTypeAccessTree.setAccessByPath(java.util.List, int, org.apache.doris.thrift.TAccessPathType)" because the return value of "java.util.Map.get(Object)" is null
	at org.apache.doris.nereids.rules.rewrite.NestedColumnPruning$DataTypeAccessTree.setAccessByPath(NestedColumnPruning.java:431)
	at org.apache.doris.nereids.rules.rewrite.NestedColumnPruning.pruneDataType(NestedColumnPruning.java:128)
	at org.apache.doris.nereids.rules.rewrite.NestedColumnPruning.rewriteRoot(NestedColumnPruning.java:86)
	at org.apache.doris.nereids.jobs.rewrite.CustomRewriteJob.execute(CustomRewriteJob.java:61)
	at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:167)
	at org.apache.doris.nereids.jobs.executor.Rewriter.lambda$execute$23(Rewriter.java:963)
	at org.apache.doris.nereids.util.MoreFieldsThread.keepFunctionSignature(MoreFieldsThread.java:127)
	at org.apache.doris.nereids.util.MoreFieldsThread.keepFunctionSignature(MoreFieldsThread.java:116)
	at org.apache.doris.nereids.jobs.executor.Rewriter.execute(Rewriter.java:962)
	at org.apache.doris.nereids.NereidsPlanner.lambda$rewrite$5(NereidsPlanner.java:434)
	at org.apache.doris.nereids.NereidsPlanner.keepOrShowPlanProcess(NereidsPlanner.java:1109)
	at org.apache.doris.nereids.NereidsPlanner.rewrite(NereidsPlanner.java:433)
	at org.apache.doris.nereids.NereidsPlanner.planWithoutLock(NereidsPlanner.java:297)
	at org.apache.doris.nereids.NereidsPlanner.planWithLock(NereidsPlanner.java:263)
	at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:162)
	at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:748)
	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:541)
	at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:500)
	at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:485)
	at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:311)
	at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:198)
	at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:231)
	at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:259)
	at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:403)
	at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

hello-stephen · 2026-01-30T11:08:14Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

924060929 · 2026-01-30T11:13:19Z

run buildall

924060929 · 2026-01-30T11:16:10Z

run buildall

doris-robot · 2026-01-30T11:53:43Z

TPC-H: Total hot run time: 32126 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 92e36c9317a151c4bc912c52e735aa1f8c3f19ba, data reload: false

------ Round 1 ----------------------------------
q1	17626	5266	5109	5109
q2	2032	307	193	193
q3	10190	1290	766	766
q4	10212	866	323	323
q5	7556	2179	1902	1902
q6	204	204	154	154
q7	894	716	617	617
q8	9299	1375	1115	1115
q9	5292	4832	4919	4832
q10	6816	1945	1578	1578
q11	502	288	274	274
q12	334	378	225	225
q13	17791	4064	3278	3278
q14	245	252	226	226
q15	908	825	815	815
q16	689	676	620	620
q17	646	808	536	536
q18	6779	6504	6520	6504
q19	1247	995	607	607
q20	403	344	235	235
q21	2570	2008	1933	1933
q22	355	312	284	284
Total cold run time: 102590 ms
Total hot run time: 32126 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5360	5325	5347	5325
q2	260	335	254	254
q3	2217	2706	2238	2238
q4	1364	1804	1345	1345
q5	4422	4263	4345	4263
q6	224	185	143	143
q7	2379	2174	1966	1966
q8	2633	2467	2422	2422
q9	7508	7417	7665	7417
q10	2836	3159	2585	2585
q11	537	469	437	437
q12	704	766	659	659
q13	3877	4768	3480	3480
q14	313	336	312	312
q15	872	817	805	805
q16	685	729	671	671
q17	1100	1374	1399	1374
q18	8273	7953	7804	7804
q19	882	951	875	875
q20	2071	2171	2082	2082
q21	4693	4236	4142	4142
q22	564	543	542	542
Total cold run time: 53774 ms
Total hot run time: 51141 ms

doris-robot · 2026-01-30T12:10:30Z

ClickBench: Total hot run time: 28.32 s

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 92e36c9317a151c4bc912c52e735aa1f8c3f19ba, data reload: false

query1	0.06	0.05	0.05
query2	0.09	0.04	0.04
query3	0.27	0.08	0.08
query4	1.60	0.12	0.11
query5	0.27	0.24	0.24
query6	1.16	0.68	0.66
query7	0.03	0.03	0.03
query8	0.06	0.04	0.04
query9	0.57	0.49	0.50
query10	0.55	0.56	0.56
query11	0.14	0.09	0.11
query12	0.15	0.11	0.11
query13	0.66	0.64	0.63
query14	1.10	1.07	1.08
query15	0.89	0.88	0.87
query16	0.41	0.39	0.39
query17	1.18	1.16	1.11
query18	0.23	0.22	0.21
query19	2.15	2.03	2.00
query20	0.02	0.01	0.01
query21	15.41	0.26	0.15
query22	5.25	0.05	0.04
query23	16.02	0.29	0.10
query24	1.47	1.32	0.29
query25	0.11	0.07	0.06
query26	0.15	0.14	0.13
query27	0.06	0.05	0.07
query28	4.93	1.14	0.96
query29	12.60	3.86	3.13
query30	0.27	0.12	0.11
query31	2.82	0.63	0.41
query32	3.23	0.59	0.49
query33	3.20	3.31	3.27
query34	15.98	5.39	4.71
query35	4.84	4.83	4.77
query36	0.65	0.50	0.48
query37	0.12	0.07	0.07
query38	0.07	0.04	0.03
query39	0.04	0.03	0.04
query40	0.18	0.16	0.15
query41	0.09	0.03	0.03
query42	0.05	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 99.18 s
Total hot run time: 28.32 s

hello-stephen · 2026-01-30T12:20:03Z

FE UT Coverage Report

Increment line coverage 100.00% (10/10) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-01-30T15:22:21Z

FE Regression Coverage Report

Increment line coverage 100.00% (10/10) 🎉
Increment coverage report
Complete coverage report

github-actions · 2026-01-31T00:18:34Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-01-31T00:18:36Z

PR approved by anyone and no changes requested.

…interException (#60395) fix prune nested column maybe throw NullPointerException, introduced by #58370 ```sql select k1 as k2 -- can not find s.k2, we should replace k2 to k1 from ( select struct_element(s, 'k1') k1 from tbl )t ``` ``` java.lang.NullPointerException: Cannot invoke "org.apache.doris.nereids.rules.rewrite.NestedColumnPruning$DataTypeAccessTree.setAccessByPath(java.util.List, int, org.apache.doris.thrift.TAccessPathType)" because the return value of "java.util.Map.get(Object)" is null at org.apache.doris.nereids.rules.rewrite.NestedColumnPruning$DataTypeAccessTree.setAccessByPath(NestedColumnPruning.java:431) at org.apache.doris.nereids.rules.rewrite.NestedColumnPruning.pruneDataType(NestedColumnPruning.java:128) at org.apache.doris.nereids.rules.rewrite.NestedColumnPruning.rewriteRoot(NestedColumnPruning.java:86) at org.apache.doris.nereids.jobs.rewrite.CustomRewriteJob.execute(CustomRewriteJob.java:61) at org.apache.doris.nereids.jobs.executor.AbstractBatchJobExecutor.execute(AbstractBatchJobExecutor.java:167) at org.apache.doris.nereids.jobs.executor.Rewriter.lambda$execute$23(Rewriter.java:963) at org.apache.doris.nereids.util.MoreFieldsThread.keepFunctionSignature(MoreFieldsThread.java:127) at org.apache.doris.nereids.util.MoreFieldsThread.keepFunctionSignature(MoreFieldsThread.java:116) at org.apache.doris.nereids.jobs.executor.Rewriter.execute(Rewriter.java:962) at org.apache.doris.nereids.NereidsPlanner.lambda$rewrite$5(NereidsPlanner.java:434) at org.apache.doris.nereids.NereidsPlanner.keepOrShowPlanProcess(NereidsPlanner.java:1109) at org.apache.doris.nereids.NereidsPlanner.rewrite(NereidsPlanner.java:433) at org.apache.doris.nereids.NereidsPlanner.planWithoutLock(NereidsPlanner.java:297) at org.apache.doris.nereids.NereidsPlanner.planWithLock(NereidsPlanner.java:263) at org.apache.doris.nereids.NereidsPlanner.plan(NereidsPlanner.java:162) at org.apache.doris.qe.StmtExecutor.executeByNereids(StmtExecutor.java:748) at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:541) at org.apache.doris.qe.StmtExecutor.queryRetry(StmtExecutor.java:500) at org.apache.doris.qe.StmtExecutor.execute(StmtExecutor.java:485) at org.apache.doris.qe.ConnectProcessor.executeQuery(ConnectProcessor.java:311) at org.apache.doris.qe.ConnectProcessor.handleQuery(ConnectProcessor.java:198) at org.apache.doris.qe.MysqlConnectProcessor.handleQuery(MysqlConnectProcessor.java:231) at org.apache.doris.qe.MysqlConnectProcessor.dispatch(MysqlConnectProcessor.java:259) at org.apache.doris.qe.MysqlConnectProcessor.processOnce(MysqlConnectProcessor.java:403) at org.apache.doris.mysql.ReadListener.lambda$handleEvent$0(ReadListener.java:52) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:840) ```

…throw NullPointerException #60395 (#60444) Cherry-picked from #60395 Co-authored-by: 924060929 <lanhuajian@selectdb.com>

924060929 added 2 commits January 30, 2026 19:05

fix

c1167f6

fix

856f226

924060929 added the dev/4.0.x label Jan 30, 2026

fix

92e36c9

englefly approved these changes Jan 31, 2026

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 31, 2026

github-actions bot added the reviewed label Jan 31, 2026

starocean999 approved these changes Feb 2, 2026

View reviewed changes

924060929 merged commit 44de710 into apache:master Feb 2, 2026
32 of 33 checks passed

924060929 deleted the fix-npe branch February 2, 2026 11:08

github-actions bot mentioned this pull request Feb 2, 2026

branch-4.0: [fix](prune nested column) fix prune nested column maybe throw NullPointerException #60395 #60444

Merged

yiguolei pushed a commit that referenced this pull request Feb 3, 2026

branch-4.0: [fix](prune nested column) fix prune nested column maybe …

811d8f1

…throw NullPointerException #60395 (#60444) Cherry-picked from #60395 Co-authored-by: 924060929 <lanhuajian@selectdb.com>

yiguolei added dev/4.0.4-merged and removed dev/4.0.x labels Feb 3, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[fix](prune nested column) fix prune nested column maybe throw NullPointerException#60395

[fix](prune nested column) fix prune nested column maybe throw NullPointerException#60395
924060929 merged 3 commits intoapache:masterfrom
924060929:fix-npe

924060929 commented Jan 30, 2026 •

edited

Loading

Uh oh!

hello-stephen commented Jan 30, 2026

Uh oh!

924060929 commented Jan 30, 2026

Uh oh!

924060929 commented Jan 30, 2026

Uh oh!

doris-robot commented Jan 30, 2026

Uh oh!

doris-robot commented Jan 30, 2026

Uh oh!

hello-stephen commented Jan 30, 2026

Uh oh!

hello-stephen commented Jan 30, 2026

Uh oh!

github-actions bot commented Jan 31, 2026

Uh oh!

github-actions bot commented Jan 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

924060929 commented Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

hello-stephen commented Jan 30, 2026

Uh oh!

924060929 commented Jan 30, 2026

Uh oh!

924060929 commented Jan 30, 2026

Uh oh!

doris-robot commented Jan 30, 2026

Uh oh!

doris-robot commented Jan 30, 2026

Uh oh!

hello-stephen commented Jan 30, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Jan 30, 2026

FE Regression Coverage Report

Uh oh!

github-actions bot commented Jan 31, 2026

Uh oh!

github-actions bot commented Jan 31, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

924060929 commented Jan 30, 2026 •

edited

Loading