[feature](maxcompute)support maxcompute ram_role_arn and ecs_ram_role authenticate. by hubgeter · Pull Request #60649 · apache/doris

hubgeter · 2026-02-10T10:15:27Z

What problem does this PR solve?

Problem Summary:
support maxcompute RAMRoleArn and ECSRAMRole authenticate, not only ak-sk.

CREATE CATALOG mc PROPERTIES (
  "type" = "max_compute",
  "mc.default.project" = "xxx",
  "mc.access_key" = "xx",
  "mc.secret_key" = "xxx",
  "mc.endpoint" = "xxx"
);

CREATE CATALOG mc PROPERTIES (
  "type" = "max_compute",
  "mc.auth.type" = "ram_role_arn",
  "mc.ram_role_arn" = "xxxx",
  "mc.default.project" = "xxx",
  "mc.access_key" = "xx",
  "mc.secret_key" = "xxx",
  "mc.endpoint" = "xxx"
);

CREATE CATALOG mc PROPERTIES (
  "type" = "max_compute",
  "mc.auth.type" = "ecs_ram_role",
  "mc.ecs_ram_role" = "xxxxxx",
  "mc.default.project" = "xxx",
  "mc.endpoint" = "xxx"
);

can ref : https://www.alibabacloud.com/help/zh/maxcompute/user-guide/configure-access-credentials#599325c7000vc
method 3 & 4.

Release note

support maxcompute RAMRoleArn and ECSRAMRole authenticate, not only ak-sk.

Check List (For Author)

Test
- Regression test
- Unit Test
- Manual test (add detailed scripts or steps below)
- No need to test or manual test. Explain why:
  - This is a refactor/code format and no logic has been changed.
  - Previous test can cover this change.
  - No code files have been changed.
  - Other reason
Behavior changed:
- No.
- Yes.
Does this need documentation?
- No.
- Yes.
  [catalog](maxcompute)add maxcompute catalog auth properties. doris-website#3491

Check List (For Reviewer who merge this PR)

Confirm the release note
Confirm test cases
Confirm document
Add branch pick label

Thearas · 2026-02-10T10:15:32Z

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

What problem was fixed (it's best to include specific error reporting information). How it was fixed.
Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
What features were added. Why was this function added?
Which code was refactored and why was this part of the code refactored?
Which functions were optimized and what is the difference before and after the optimization?

hubgeter · 2026-02-10T10:16:11Z

run buildall

doris-robot · 2026-02-10T10:41:47Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.35% (1795/2262)
Line Coverage	64.87% (31960/49269)
Region Coverage	65.57% (15948/24323)
Branch Coverage	56.11% (8480/15114)

doris-robot · 2026-02-10T13:54:06Z

BE UT Coverage Report

Increment line coverage 0.00% (0/22) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.70% (19450/36904)
Line Coverage	36.19% (181081/500346)
Region Coverage	32.57% (140598/431702)
Branch Coverage	33.62% (60949/181274)

be/src/runtime/descriptors.cpp

morningman · 2026-02-10T16:00:16Z

fe/fe-common/src/main/java/org/apache/doris/common/maxcompute/MCUtils.java

+            Account account = new AliyunAccount(accessKey, secretKey);
+            return new Odps(account);
+        } else if (authType.equalsIgnoreCase(MCProperties.AUTH_TYPE_RAM_ROLE_ARN)) {
+            String accessKey = properties.get(MCProperties.ACCESS_KEY);


why we still need ak,sk for arn type?

This is typically used to access maxcompute across Alibaba Cloud accounts, and aksk is required.

fe/fe-core/src/main/java/org/apache/doris/datasource/maxcompute/MaxComputeExternalTable.java

hubgeter · 2026-02-11T08:50:46Z

run buildall

hello-stephen · 2026-02-11T09:30:12Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.32% (1795/2263)
Line Coverage	64.82% (31955/49300)
Region Coverage	65.54% (15946/24330)
Branch Coverage	56.02% (8473/15124)

hubgeter · 2026-02-25T15:06:29Z

run buildall

hello-stephen · 2026-02-25T15:49:46Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.33% (1796/2264)
Line Coverage	64.80% (32008/49393)
Region Coverage	65.50% (15970/24383)
Branch Coverage	56.02% (8497/15168)

hello-stephen · 2026-02-25T16:30:05Z

FE UT Coverage Report

Increment line coverage 0.00% (0/6) 🎉
Increment coverage report
Complete coverage report

hello-stephen · 2026-02-25T19:08:00Z

BE UT Coverage Report

Increment line coverage 0.00% (0/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.52% (19587/37297)
Line Coverage	36.12% (182582/505531)
Region Coverage	32.48% (141804/436598)
Branch Coverage	33.42% (61453/183859)

hubgeter · 2026-02-26T02:46:18Z

run buildall

hello-stephen · 2026-02-26T03:24:59Z

Cloud UT Coverage Report

Increment line coverage 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	79.33% (1796/2264)
Line Coverage	64.80% (32005/49393)
Region Coverage	65.52% (15976/24383)
Branch Coverage	56.00% (8494/15168)

hello-stephen · 2026-02-26T04:01:10Z

FE UT Coverage Report

Increment line coverage 0.00% (0/6) 🎉
Increment coverage report
Complete coverage report

doris-robot · 2026-02-26T04:54:01Z

TPC-H: Total hot run time: 28918 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit f1fd1f3b5e5f6c7a4acb59a2dc018bc06db776ca, data reload: false

------ Round 1 ----------------------------------
============================================
q1	17638	4523	4308	4308
q2	q3	10723	794	517	517
q4	4735	371	255	255
q5	8125	1260	1041	1041
q6	255	181	149	149
q7	809	863	684	684
q8	10768	1532	1381	1381
q9	6856	4766	4813	4766
q10	6847	1902	1633	1633
q11	467	257	239	239
q12	754	576	470	470
q13	17783	4220	3418	3418
q14	233	239	215	215
q15	963	797	799	797
q16	775	732	667	667
q17	733	874	427	427
q18	6314	5439	5266	5266
q19	1151	977	636	636
q20	511	496	397	397
q21	4543	1876	1401	1401
q22	344	276	251	251
Total cold run time: 101327 ms
Total hot run time: 28918 ms

----- Round 2, with runtime_filter_mode=off -----
============================================
q1	4462	4373	4384	4373
q2	q3	1775	2183	1725	1725
q4	850	1188	758	758
q5	4046	4379	4334	4334
q6	176	173	139	139
q7	1714	1601	1494	1494
q8	2458	2721	2546	2546
q9	7970	7314	7363	7314
q10	2650	2920	2489	2489
q11	520	453	427	427
q12	506	620	457	457
q13	4070	4451	3584	3584
q14	277	297	280	280
q15	847	834	851	834
q16	733	782	804	782
q17	1205	1582	1315	1315
q18	7111	6782	6763	6763
q19	969	911	946	911
q20	2126	2329	2047	2047
q21	3976	3538	3357	3357
q22	473	435	396	396
Total cold run time: 48914 ms
Total hot run time: 46325 ms

doris-robot · 2026-02-26T05:05:24Z

TPC-DS: Total hot run time: 183895 ms

machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit f1fd1f3b5e5f6c7a4acb59a2dc018bc06db776ca, data reload: false

query5	4967	647	525	525
query6	329	232	202	202
query7	4229	467	267	267
query8	338	230	227	227
query9	8751	2683	2659	2659
query10	520	395	339	339
query11	16996	17373	17173	17173
query12	193	129	124	124
query13	1258	517	379	379
query14	6794	3480	3244	3244
query14_1	2938	2987	2954	2954
query15	216	210	187	187
query16	1034	525	485	485
query17	1310	800	702	702
query18	3248	525	368	368
query19	216	215	187	187
query20	155	138	132	132
query21	220	142	121	121
query22	5764	5414	4977	4977
query23	17302	16871	16649	16649
query23_1	16683	16684	16677	16677
query24	7218	1623	1238	1238
query24_1	1240	1266	1233	1233
query25	579	494	432	432
query26	1234	262	163	163
query27	2735	482	288	288
query28	4453	1847	1844	1844
query29	789	553	470	470
query30	325	249	211	211
query31	868	705	647	647
query32	86	73	72	72
query33	513	332	287	287
query34	916	904	551	551
query35	614	661	581	581
query36	1102	1125	939	939
query37	130	92	82	82
query38	2942	2959	2893	2893
query39	895	854	823	823
query39_1	826	888	843	843
query40	230	153	137	137
query41	65	61	57	57
query42	106	103	101	101
query43	364	376	339	339
query44	
query45	196	189	183	183
query46	893	975	603	603
query47	2153	2167	2094	2094
query48	300	314	240	240
query49	622	458	377	377
query50	673	268	209	209
query51	4075	4048	4049	4048
query52	105	106	93	93
query53	293	341	282	282
query54	287	263	258	258
query55	90	86	79	79
query56	310	328	315	315
query57	1369	1315	1283	1283
query58	294	277	276	276
query59	2608	2665	2557	2557
query60	353	340	321	321
query61	152	144	149	144
query62	630	605	547	547
query63	311	271	276	271
query64	4859	1249	995	995
query65	
query66	1398	459	351	351
query67	16243	16276	16208	16208
query68	
query69	403	315	284	284
query70	985	908	953	908
query71	326	303	301	301
query72	2805	2615	2378	2378
query73	538	548	323	323
query74	9990	9890	9745	9745
query75	2822	2752	2499	2499
query76	2304	1045	662	662
query77	342	381	320	320
query78	11264	11428	10694	10694
query79	2602	789	620	620
query80	1777	625	560	560
query81	553	277	256	256
query82	960	153	114	114
query83	339	260	241	241
query84	257	123	95	95
query85	892	471	436	436
query86	425	319	300	300
query87	3106	3070	3005	3005
query88	3499	2633	2646	2633
query89	428	366	337	337
query90	1942	181	169	169
query91	168	151	134	134
query92	84	75	74	74
query93	1164	860	498	498
query94	650	315	301	301
query95	577	396	310	310
query96	644	512	228	228
query97	2468	2559	2396	2396
query98	237	236	228	228
query99	1034	982	921	921
Total cold run time: 256783 ms
Total hot run time: 183895 ms

hello-stephen · 2026-02-26T06:22:25Z

BE UT Coverage Report

Increment line coverage 0.00% (0/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	52.51% (19560/37250)
Line Coverage	36.13% (182575/505296)
Region Coverage	32.48% (141792/436496)
Branch Coverage	33.44% (61462/183821)

hello-stephen · 2026-02-26T07:50:00Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.32% (26021/36483)
Line Coverage	54.10% (272552/503794)
Region Coverage	51.44% (226680/440662)
Branch Coverage	52.91% (97569/184407)

hello-stephen · 2026-02-26T10:44:09Z

BE Regression && UT Coverage Report

Increment line coverage 0.00% (0/25) 🎉

Increment coverage report
Complete coverage report

Category	Coverage
Function Coverage	71.41% (26054/36483)
Line Coverage	54.19% (273007/503794)
Region Coverage	51.59% (227318/440662)
Branch Coverage	53.01% (97752/184407)

github-actions · 2026-02-27T07:24:14Z

PR approved by at least one committer and no changes requested.

github-actions · 2026-02-27T07:24:17Z

PR approved by anyone and no changes requested.

…cs_ram_role.(#60649) (#60905) bp #60649

…from apache#60649

…from #60649 (#61717) ### What problem does this PR solve? Issue Number: N/A Related PR: #60649, #60905 Problem Summary: #60649 added support for MaxCompute `ram_role_arn` and `ecs_ram_role` authentication on `master`, and #60905 picked it to `branch-4.0`. `branch-4.1` is based on `branch-4.0`, but because the MaxCompute write path differs between branches, part of the write-side adaptation was not picked over. As a result, on `branch-4.1`, MaxCompute INSERT/write still relied on `access_key/secret_key` in some code paths, and could not correctly reuse the catalog `properties` for `ram_role_arn` or `ecs_ram_role` authentication. This PR supplements the missing pick for `branch-4.1`.

morningman added dev/4.0.x kind/need-document labels Feb 10, 2026

morningman reviewed Feb 10, 2026

View reviewed changes

hubgeter requested a review from CalvinKirs as a code owner February 11, 2026 08:50

hubgeter added 2 commits February 25, 2026 22:22

[feature](maxcompute)support maxcompute ram_role_arn and ecs_ram_role.

b49ea31

fix build

5181fe1

hubgeter force-pushed the mc_auth branch from ec8a6e4 to 5181fe1 Compare February 25, 2026 15:06

fix fe build

f1fd1f3

morningman approved these changes Feb 27, 2026

View reviewed changes

CalvinKirs approved these changes Feb 27, 2026

View reviewed changes

github-actions bot added the approved Indicates a PR has been approved by one committer. label Feb 27, 2026

github-actions bot added the reviewed label Feb 27, 2026

morningman merged commit 567661f into apache:master Feb 27, 2026
27 of 30 checks passed

github-actions bot added the dev/4.0.x-conflict label Feb 27, 2026

hubgeter mentioned this pull request Feb 28, 2026

branch-4.0:[feature](maxcompute)support maxcompute ram_role_arn and ecs_ram_role.(#60649) #60905

Merged

yiguolei pushed a commit that referenced this pull request Mar 3, 2026

branch-4.0:[feature](maxcompute)support maxcompute ram_role_arn and e…

c2ef7a5

…cs_ram_role.(#60649) (#60905) bp #60649

yiguolei added dev/4.0.4-merged and removed dev/4.0.x dev/4.0.x-conflict labels Mar 3, 2026

yiguolei mentioned this pull request Mar 7, 2026

4.0.4 Release Notes #61121

Open

hubgeter mentioned this pull request Mar 25, 2026

[catalog](maxcompute)add maxcompute catalog auth properties. apache/doris-website#3491

Open

8 tasks

hubgeter added a commit to hubgeter/doris that referenced this pull request Mar 25, 2026

branch-4.1:[feature](maxcompute) supplement missing writer auth pick …

52f95c8

…from apache#60649

hubgeter mentioned this pull request Mar 25, 2026

branch-4.1:[feature](maxcompute) supplement missing writer auth pick from #60649 #61717

Merged

Conversation

hubgeter commented Feb 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What problem does this PR solve?

Release note

Check List (For Author)

Check List (For Reviewer who merge this PR)

Uh oh!

Thearas commented Feb 10, 2026

Uh oh!

hubgeter commented Feb 10, 2026

Uh oh!

doris-robot commented Feb 10, 2026

Cloud UT Coverage Report

Uh oh!

doris-robot commented Feb 10, 2026

BE UT Coverage Report

Uh oh!

Uh oh!

morningman Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

hubgeter Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hubgeter commented Feb 11, 2026

Uh oh!

hello-stephen commented Feb 11, 2026

Cloud UT Coverage Report

Uh oh!

hubgeter commented Feb 25, 2026

Uh oh!

hello-stephen commented Feb 25, 2026

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Feb 25, 2026

FE UT Coverage Report

Uh oh!

hello-stephen commented Feb 25, 2026

BE UT Coverage Report

Uh oh!

hubgeter commented Feb 26, 2026

Uh oh!

hello-stephen commented Feb 26, 2026

Cloud UT Coverage Report

Uh oh!

hello-stephen commented Feb 26, 2026

FE UT Coverage Report

Uh oh!

doris-robot commented Feb 26, 2026

Uh oh!

doris-robot commented Feb 26, 2026

Uh oh!

hello-stephen commented Feb 26, 2026

BE UT Coverage Report

Uh oh!

hello-stephen commented Feb 26, 2026

BE Regression && UT Coverage Report

Uh oh!

hello-stephen commented Feb 26, 2026

BE Regression && UT Coverage Report

Uh oh!

github-actions bot commented Feb 27, 2026

Uh oh!

github-actions bot commented Feb 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

hubgeter commented Feb 10, 2026 •

edited

Loading