Skip to content

[refactoring](multi-catalog)data_lake_reader_refactoring.#61783

Draft
kaka11chen wants to merge 1 commit intoapache:masterfrom
kaka11chen:data_lake_reader_refactoring
Draft

[refactoring](multi-catalog)data_lake_reader_refactoring.#61783
kaka11chen wants to merge 1 commit intoapache:masterfrom
kaka11chen:data_lake_reader_refactoring

Conversation

@kaka11chen
Copy link
Contributor

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@Thearas
Copy link
Contributor

Thearas commented Mar 26, 2026

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@kaka11chen
Copy link
Contributor Author

run buildall

@hello-stephen
Copy link
Contributor

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 78.67% (1796/2283)
Line Coverage 64.40% (32282/50128)
Region Coverage 65.28% (16163/24761)
Branch Coverage 55.78% (8620/15454)

@doris-robot
Copy link

TPC-H: Total hot run time: 26855 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit b3338a3791cc4c2c64f44abadf924a0685d12f50, data reload: false

------ Round 1 ----------------------------------
orders	Doris	NULL	NULL	0	0	0	NULL	0	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	17637	4444	4312	4312
q2	q3	10744	812	531	531
q4	4721	358	252	252
q5	8229	1251	1009	1009
q6	245	173	147	147
q7	818	845	685	685
q8	10867	1515	1330	1330
q9	6766	4771	4702	4702
q10	6379	1952	1803	1803
q11	481	248	241	241
q12	754	585	469	469
q13	18042	2697	1974	1974
q14	233	238	223	223
q15	q16	740	765	676	676
q17	724	865	440	440
q18	6010	5420	5261	5261
q19	1118	977	631	631
q20	551	497	379	379
q21	4738	2091	1506	1506
q22	392	357	284	284
Total cold run time: 100189 ms
Total hot run time: 26855 ms

----- Round 2, with runtime_filter_mode=off -----
orders	Doris	NULL	NULL	150000000	42	6422171781	NULL	22778155	NULL	NULL	2023-12-26 18:27:23	2023-12-26 18:42:55	NULL	utf-8	NULL	NULL	
============================================
q1	4635	4507	4599	4507
q2	q3	3908	4349	3821	3821
q4	901	1185	774	774
q5	4075	4475	4374	4374
q6	199	180	137	137
q7	1769	1629	1543	1543
q8	2543	2737	2611	2611
q9	7586	7501	7459	7459
q10	3821	4001	3614	3614
q11	527	434	416	416
q12	491	593	433	433
q13	2622	3018	2041	2041
q14	312	318	300	300
q15	q16	773	768	717	717
q17	1201	1406	1273	1273
q18	7332	7009	6705	6705
q19	933	912	951	912
q20	2089	2163	1985	1985
q21	4051	3624	3333	3333
q22	455	466	375	375
Total cold run time: 50223 ms
Total hot run time: 47330 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 169048 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit b3338a3791cc4c2c64f44abadf924a0685d12f50, data reload: false

query5	4335	640	512	512
query6	334	231	200	200
query7	4231	469	263	263
query8	352	258	238	238
query9	8685	2737	2732	2732
query10	531	376	369	369
query11	7004	5096	4857	4857
query12	184	121	124	121
query13	1298	475	350	350
query14	5731	3752	3472	3472
query14_1	2970	2830	2874	2830
query15	209	195	180	180
query16	998	408	450	408
query17	909	729	635	635
query18	2453	456	357	357
query19	221	214	188	188
query20	131	126	126	126
query21	217	138	113	113
query22	13219	13990	14721	13990
query23	16612	16252	16088	16088
query23_1	16410	15833	15691	15691
query24	7188	1615	1221	1221
query24_1	1193	1233	1223	1223
query25	567	455	406	406
query26	1245	261	184	184
query27	2759	478	294	294
query28	4495	1820	1825	1820
query29	803	570	472	472
query30	302	217	188	188
query31	1019	952	869	869
query32	87	69	73	69
query33	509	318	275	275
query34	869	891	521	521
query35	660	681	590	590
query36	1096	1134	987	987
query37	142	98	86	86
query38	2968	2936	2833	2833
query39	845	822	793	793
query39_1	798	798	803	798
query40	235	153	136	136
query41	63	62	58	58
query42	261	251	252	251
query43	237	251	223	223
query44	
query45	190	190	180	180
query46	878	983	600	600
query47	2128	2158	2058	2058
query48	303	315	230	230
query49	633	464	380	380
query50	705	301	217	217
query51	4052	4069	4018	4018
query52	263	270	259	259
query53	286	332	293	293
query54	312	280	259	259
query55	87	88	83	83
query56	338	314	311	311
query57	1956	1719	1779	1719
query58	282	270	274	270
query59	2767	2943	2737	2737
query60	329	340	322	322
query61	186	155	149	149
query62	619	581	537	537
query63	307	284	282	282
query64	5011	1285	1021	1021
query65	
query66	1472	451	359	359
query67	24231	24252	24407	24252
query68	
query69	410	323	294	294
query70	986	981	967	967
query71	333	315	297	297
query72	2934	2902	2645	2645
query73	549	553	321	321
query74	9618	9553	9439	9439
query75	2845	2767	2503	2503
query76	2303	1030	670	670
query77	361	373	312	312
query78	10979	11070	10440	10440
query79	3110	772	591	591
query80	1729	624	510	510
query81	576	259	231	231
query82	998	158	121	121
query83	328	260	235	235
query84	302	121	106	106
query85	909	495	448	448
query86	423	299	294	294
query87	3134	3095	2954	2954
query88	3518	2611	2623	2611
query89	425	371	350	350
query90	2024	179	177	177
query91	168	168	135	135
query92	75	73	71	71
query93	1368	858	499	499
query94	668	321	294	294
query95	579	393	323	323
query96	647	511	231	231
query97	2436	2462	2413	2413
query98	237	217	226	217
query99	1005	1001	925	925
Total cold run time: 251988 ms
Total hot run time: 169048 ms

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants