Skip to content

[fix](function) support TIMESTAMPDIFF MICROSECOND in nereids#63365

Open
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:TIMESTAMPDIFF-MICROSECOND
Open

[fix](function) support TIMESTAMPDIFF MICROSECOND in nereids#63365
Mryange wants to merge 1 commit into
apache:masterfrom
Mryange:TIMESTAMPDIFF-MICROSECOND

Conversation

@Mryange
Copy link
Copy Markdown
Contributor

@Mryange Mryange commented May 18, 2026

What problem does this PR solve?

Nereids rejects TIMESTAMPDIFF(MICROSECOND, ...) during analysis.

The executable path already exists through MicroSecondsDiff, but the FE binder cannot reach it because:

  • Interval.TimeUnit does not define a standalone MICROSECOND
  • DatetimeFunctionBinder only routes TIMESTAMPDIFF up to SECOND

As a result, queries such as TIMESTAMPDIFF(MICROSECOND, MIN(t), MAX(t)) fail even for valid DATETIMEV2(6) inputs.

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented May 18, 2026

/review

@hello-stephen
Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copy link
Copy Markdown
Contributor

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code review completed. I did not find blocking issues.

Critical checkpoint conclusions:

  • Goal and proof: The PR routes Nereids TIMESTAMPDIFF(MICROSECOND, start, end) to the existing MicroSecondsDiff scalar function, matching the intended end-minus-start argument order. The added FE unit test and regression case cover literal and aggregate DATETIMEV2(6) usage.
  • Scope: The change is small and focused on binding the missing unit plus test coverage.
  • Concurrency and lifecycle: No shared mutable state, locking, threads, static initialization dependency, or lifecycle-sensitive path is introduced.
  • Config and compatibility: No config, storage format, FE-BE protocol, or rolling-upgrade compatibility concern is introduced. The scalar function and executable path already exist.
  • Parallel paths: Existing timestampdiff units remain unchanged. The Nereids binder now covers MICROSECOND for the timestampdiff path; date add/sub/floor/ceil paths are not part of this PR.
  • Error handling: Unsupported units still fail analysis; no ignored Status or exception-boundary issue is involved in this Java analyzer path.
  • Tests and results: Regression output is deterministic because the new aggregate query returns one row, and the table is dropped before use. No issue found in the added expected results.
  • Observability and performance: No new runtime-heavy path or observability need; binding adds only one switch case.
  • User focus: No additional user-provided review focus was supplied.

@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented May 18, 2026

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 31005 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18ed4b064f6fcfdfa37dd93b1ba3b18579850aeb, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17796	3991	3953	3953
q2	q3	10770	1383	769	769
q4	4695	477	340	340
q5	7543	2276	2109	2109
q6	237	178	136	136
q7	923	783	632	632
q8	9520	1680	1475	1475
q9	5107	4944	4894	4894
q10	6437	2081	1809	1809
q11	431	282	245	245
q12	624	428	295	295
q13	18084	3386	2792	2792
q14	260	256	232	232
q15	q16	780	789	710	710
q17	887	952	869	869
q18	7413	5808	5424	5424
q19	1143	1263	1084	1084
q20	503	501	310	310
q21	5940	2907	2601	2601
q22	469	369	326	326
Total cold run time: 99562 ms
Total hot run time: 31005 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4780	4507	4528	4507
q2	q3	4813	5177	4632	4632
q4	2120	2209	1396	1396
q5	4795	4807	4615	4615
q6	228	173	127	127
q7	1778	1816	1529	1529
q8	2360	2064	2070	2064
q9	7768	7542	7132	7132
q10	4458	4414	3994	3994
q11	523	375	351	351
q12	700	726	505	505
q13	2994	3384	2752	2752
q14	280	283	243	243
q15	q16	669	691	598	598
q17	1242	1235	1230	1230
q18	7183	6544	6736	6544
q19	1136	1118	1138	1118
q20	2199	2190	1924	1924
q21	5315	4591	4501	4501
q22	520	450	407	407
Total cold run time: 55861 ms
Total hot run time: 50169 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 169010 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18ed4b064f6fcfdfa37dd93b1ba3b18579850aeb, data reload: false

query5	4346	645	523	523
query6	340	224	213	213
query7	4256	582	304	304
query8	328	235	221	221
query9	8838	3972	3984	3972
query10	455	351	299	299
query11	5797	2384	2239	2239
query12	186	127	125	125
query13	1298	611	385	385
query14	5983	5360	5036	5036
query14_1	4352	4349	4366	4349
query15	224	203	181	181
query16	1024	456	463	456
query17	1171	765	620	620
query18	2668	479	360	360
query19	211	220	162	162
query20	134	128	127	127
query21	213	139	117	117
query22	13579	13532	13461	13461
query23	17321	16380	16023	16023
query23_1	16158	16151	16247	16151
query24	7507	1749	1290	1290
query24_1	1297	1282	1294	1282
query25	551	463	397	397
query26	1322	314	172	172
query27	2668	539	331	331
query28	4407	1949	1930	1930
query29	968	609	499	499
query30	302	243	199	199
query31	1112	1064	927	927
query32	86	75	71	71
query33	557	343	288	288
query34	1178	1104	637	637
query35	763	779	667	667
query36	1344	1319	1162	1162
query37	151	108	92	92
query38	3226	3163	3092	3092
query39	933	937	915	915
query39_1	870	861	863	861
query40	225	145	132	132
query41	65	63	61	61
query42	107	107	107	107
query43	323	323	285	285
query44	
query45	207	199	192	192
query46	1025	1188	726	726
query47	2326	2345	2129	2129
query48	414	405	289	289
query49	636	504	404	404
query50	992	354	253	253
query51	4534	4321	4202	4202
query52	104	108	98	98
query53	248	285	215	215
query54	324	286	267	267
query55	93	93	93	93
query56	317	339	336	336
query57	1432	1367	1325	1325
query58	315	286	276	276
query59	1575	1596	1402	1402
query60	348	330	320	320
query61	174	175	190	175
query62	679	627	551	551
query63	241	204	220	204
query64	2478	851	692	692
query65	
query66	1712	494	375	375
query67	29939	30091	29840	29840
query68	
query69	473	343	302	302
query70	1010	1037	999	999
query71	295	275	263	263
query72	2950	2715	2485	2485
query73	842	782	400	400
query74	5099	4922	4706	4706
query75	2667	2574	2250	2250
query76	2308	1139	747	747
query77	399	409	325	325
query78	12174	12151	11469	11469
query79	1520	1076	724	724
query80	1353	574	448	448
query81	517	287	235	235
query82	997	154	121	121
query83	343	269	247	247
query84	261	147	115	115
query85	933	539	447	447
query86	446	341	339	339
query87	3398	3360	3184	3184
query88	3519	2640	2618	2618
query89	443	386	338	338
query90	1884	180	166	166
query91	173	165	149	149
query92	80	72	76	72
query93	1624	1506	853	853
query94	730	339	319	319
query95	665	477	359	359
query96	1019	739	318	318
query97	2672	2662	2550	2550
query98	236	227	260	227
query99	1116	1092	984	984
Total cold run time: 253870 ms
Total hot run time: 169010 ms

@Mryange
Copy link
Copy Markdown
Contributor Author

Mryange commented May 18, 2026

run buildall

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-H: Total hot run time: 30607 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 18ed4b064f6fcfdfa37dd93b1ba3b18579850aeb, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17736	3795	3777	3777
q2	q3	10770	1348	803	803
q4	4683	468	352	352
q5	7525	2181	2089	2089
q6	260	179	138	138
q7	939	763	639	639
q8	9362	1598	1650	1598
q9	6791	4841	4848	4841
q10	6440	2075	1795	1795
q11	431	270	239	239
q12	695	426	296	296
q13	18218	3362	2799	2799
q14	263	251	228	228
q15	q16	812	774	697	697
q17	893	862	929	862
q18	6693	5625	5497	5497
q19	1209	1276	1089	1089
q20	510	411	260	260
q21	5564	2536	2300	2300
q22	420	362	308	308
Total cold run time: 100214 ms
Total hot run time: 30607 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4194	4116	4079	4079
q2	q3	4440	4892	4264	4264
q4	2081	2204	1362	1362
q5	4336	4206	4254	4206
q6	227	171	129	129
q7	1744	2036	1707	1707
q8	2518	2030	2035	2030
q9	7684	7678	7595	7595
q10	4533	4443	3992	3992
q11	596	407	365	365
q12	733	737	522	522
q13	3392	3725	2930	2930
q14	299	295	271	271
q15	q16	725	753	684	684
q17	1317	1321	1296	1296
q18	7867	7261	7241	7241
q19	1172	1070	1115	1070
q20	2194	2206	1936	1936
q21	5228	4600	4453	4453
q22	520	463	415	415
Total cold run time: 55800 ms
Total hot run time: 50547 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

TPC-DS: Total hot run time: 169141 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 18ed4b064f6fcfdfa37dd93b1ba3b18579850aeb, data reload: false

query5	4311	658	514	514
query6	329	217	200	200
query7	4225	576	289	289
query8	332	226	220	220
query9	8801	3979	4009	3979
query10	450	343	325	325
query11	5778	2352	2229	2229
query12	183	129	127	127
query13	1335	599	432	432
query14	5939	5341	5039	5039
query14_1	4350	4328	4374	4328
query15	212	203	186	186
query16	1013	456	449	449
query17	1184	745	604	604
query18	2460	503	382	382
query19	226	212	175	175
query20	138	137	136	136
query21	222	140	119	119
query22	13509	13548	13432	13432
query23	17141	16476	15979	15979
query23_1	16120	16152	16242	16152
query24	7474	1756	1284	1284
query24_1	1315	1291	1310	1291
query25	611	483	435	435
query26	1328	330	168	168
query27	2689	552	330	330
query28	4481	1984	1946	1946
query29	1010	630	525	525
query30	306	243	204	204
query31	1133	1085	949	949
query32	90	78	76	76
query33	559	363	307	307
query34	1177	1117	648	648
query35	757	784	722	722
query36	1318	1380	1177	1177
query37	150	106	86	86
query38	3224	3117	3014	3014
query39	932	932	905	905
query39_1	887	883	860	860
query40	229	148	124	124
query41	64	62	63	62
query42	109	108	105	105
query43	320	321	281	281
query44	
query45	207	199	193	193
query46	1069	1187	725	725
query47	2322	2392	2211	2211
query48	400	422	300	300
query49	651	506	419	419
query50	1043	363	242	242
query51	4379	4375	4247	4247
query52	105	115	95	95
query53	265	284	209	209
query54	310	274	252	252
query55	93	93	87	87
query56	308	317	306	306
query57	1394	1382	1315	1315
query58	293	270	271	270
query59	1565	1638	1385	1385
query60	323	323	307	307
query61	161	151	165	151
query62	664	633	557	557
query63	254	212	203	203
query64	2421	818	648	648
query65	
query66	1726	499	359	359
query67	30151	29965	29806	29806
query68	
query69	476	332	314	314
query70	1035	982	964	964
query71	311	273	268	268
query72	2970	2640	2525	2525
query73	833	721	433	433
query74	5050	4907	4749	4749
query75	2681	2627	2272	2272
query76	2300	1133	748	748
query77	391	417	337	337
query78	12054	12069	11543	11543
query79	1450	1038	729	729
query80	734	556	444	444
query81	466	280	244	244
query82	1369	156	124	124
query83	326	277	248	248
query84	261	152	114	114
query85	915	549	455	455
query86	392	330	300	300
query87	3391	3413	3223	3223
query88	3516	2675	2662	2662
query89	455	381	338	338
query90	1908	181	171	171
query91	180	169	148	148
query92	79	88	74	74
query93	1439	1413	860	860
query94	559	339	287	287
query95	672	455	344	344
query96	1004	760	359	359
query97	2699	2688	2540	2540
query98	247	233	252	233
query99	1084	1099	976	976
Total cold run time: 252942 ms
Total hot run time: 169141 ms

@hello-stephen
Copy link
Copy Markdown
Contributor

FE Regression Coverage Report

Increment line coverage 100.00% (2/2) 🎉
Increment coverage report
Complete coverage report

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants