Skip to content

Conversation

@suxiaogang223
Copy link
Contributor

What problem does this PR solve?

Problem Summary:

With enable_java_support=false, Hive/HDFS access still reaches JNI and can hang in getJNIEnv, sometimes with StackOverflowError.

Release note

Add an early enable_java_support guard in HdfsMgr::get_or_create_fs to return an error before JNI initialization.

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@suxiaogang223
Copy link
Contributor Author

run buildall

@suxiaogang223
Copy link
Contributor Author

run buildall

@doris-robot
Copy link

BE UT Coverage Report

Increment line coverage 42.86% (3/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 52.76% (19277/36540)
Line Coverage 36.14% (179172/495766)
Region Coverage 32.60% (139012/426377)
Branch Coverage 33.52% (60121/179339)

@doris-robot
Copy link

TPC-H: Total hot run time: 32047 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit a142e6d6c180b971fb85167fcd41de37b389a178, data reload: false

------ Round 1 ----------------------------------
q1	17603	5292	5064	5064
q2	2092	302	187	187
q3	10230	1291	739	739
q4	10221	877	315	315
q5	7526	2115	1936	1936
q6	196	180	149	149
q7	895	743	605	605
q8	9256	1315	1110	1110
q9	5271	4831	4963	4831
q10	6846	1925	1574	1574
q11	513	305	267	267
q12	335	376	226	226
q13	17603	4029	3231	3231
q14	237	241	224	224
q15	905	834	814	814
q16	669	683	620	620
q17	635	765	513	513
q18	6768	6445	7282	6445
q19	1611	1092	641	641
q20	402	354	277	277
q21	2908	2293	1985	1985
q22	372	323	294	294
Total cold run time: 103094 ms
Total hot run time: 32047 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5561	5442	5609	5442
q2	279	351	266	266
q3	2436	2852	2515	2515
q4	1461	1886	1523	1523
q5	4641	4635	4537	4537
q6	225	181	144	144
q7	2102	1926	1802	1802
q8	2557	2395	2402	2395
q9	7865	7570	7584	7570
q10	2893	2951	2537	2537
q11	532	480	451	451
q12	628	677	533	533
q13	3566	4050	3275	3275
q14	277	289	280	280
q15	832	819	794	794
q16	637	679	639	639
q17	1079	1266	1355	1266
q18	7569	7500	7379	7379
q19	809	788	799	788
q20	1969	2028	1915	1915
q21	4455	4272	4104	4104
q22	577	547	498	498
Total cold run time: 52950 ms
Total hot run time: 50653 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 28.25 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit a142e6d6c180b971fb85167fcd41de37b389a178, data reload: false

query1	0.05	0.05	0.04
query2	0.10	0.05	0.05
query3	0.25	0.08	0.09
query4	1.61	0.11	0.11
query5	0.26	0.25	0.26
query6	1.16	0.68	0.68
query7	0.03	0.03	0.03
query8	0.05	0.04	0.04
query9	0.57	0.50	0.49
query10	0.55	0.54	0.53
query11	0.15	0.10	0.10
query12	0.14	0.10	0.11
query13	0.64	0.61	0.63
query14	1.06	1.06	1.08
query15	0.87	0.87	0.88
query16	0.42	0.37	0.38
query17	1.14	1.16	1.12
query18	0.23	0.22	0.21
query19	2.08	2.01	2.04
query20	0.02	0.02	0.01
query21	15.40	0.28	0.15
query22	5.11	0.05	0.04
query23	15.86	0.29	0.11
query24	1.00	0.39	0.28
query25	0.12	0.06	0.06
query26	0.15	0.14	0.13
query27	0.06	0.06	0.08
query28	3.48	1.17	0.96
query29	12.60	3.89	3.12
query30	0.29	0.14	0.12
query31	2.82	0.66	0.40
query32	3.23	0.60	0.49
query33	3.20	3.25	3.22
query34	16.09	5.40	4.72
query35	4.82	4.75	4.80
query36	0.65	0.50	0.49
query37	0.12	0.07	0.07
query38	0.08	0.04	0.04
query39	0.05	0.03	0.03
query40	0.19	0.16	0.15
query41	0.09	0.03	0.03
query42	0.04	0.03	0.03
query43	0.05	0.04	0.04
Total cold run time: 96.88 s
Total hot run time: 28.25 s

@hello-stephen
Copy link
Contributor

BE Regression && UT Coverage Report

Increment line coverage 42.86% (3/7) 🎉

Increment coverage report
Complete coverage report

Category Coverage
Function Coverage 73.23% (26229/35818)
Line Coverage 56.28% (278324/494533)
Region Coverage 53.87% (232004/430695)
Branch Coverage 55.66% (100222/180055)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants