From a70ead628a980644895af9be840b14e63fa859cd Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Tue, 26 May 2026 11:07:59 -0700 Subject: [PATCH 1/4] DOC-2209: Recommend sts for Iceberg AWS Glue on Cloud Update the Cloud variant of the Iceberg + AWS Glue page to use `sts` instead of `aws_instance_metadata` for `iceberg_rest_catalog_credentials_source` when reusing the cluster's IAM role. On Redpanda Cloud, the cluster IAM role is reached via IRSA (STS AssumeRoleWithWebIdentity), and cloudv2#26411 re-points the Glue IAM policy attachment to fire on `sts`. The previous value is accepted at the API but no longer attaches Glue permissions on Cloud. Requires Redpanda v26.1.9. Co-Authored-By: Claude Opus 4.7 (1M context) --- modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc index 6089a55901..3b563cbee8 100644 --- a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc +++ b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc @@ -135,7 +135,7 @@ For an example cluster configuration that uses the same IAM credentials for both * If you want to configure authentication to AWS Glue separately from authentication to S3, there are equivalent credential configuration properties named `iceberg_rest_catalog_aws_*` that override the object storage credentials. These properties only apply to REST catalog authentication, and never to S3 authentication: + -- -** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`]. To use the cluster's IAM role, set the property to `aws_instance_metadata`. To use static credentials, set to `config_file`. +** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`]. To use the cluster's IAM role, set the property to `sts`. To use static credentials, set to `config_file`. ** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] (static credentials only) ** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] (static credentials only), added as a secret value (see the <> for details) ** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] @@ -217,7 +217,7 @@ rpk cluster config set \ iceberg_catalog_type=rest \ iceberg_rest_catalog_endpoint=https://glue..amazonaws.com/iceberg \ iceberg_rest_catalog_authentication_mode=aws_sigv4 \ - iceberg_rest_catalog_credentials_source=aws_instance_metadata \ + iceberg_rest_catalog_credentials_source=sts \ iceberg_rest_catalog_aws_region= \ iceberg_rest_catalog_base_location=s3:/// ---- From 39226ffa90daa189d69acea8bfe1597e55389be3 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 27 May 2026 18:40:51 -0700 Subject: [PATCH 2/4] Apply clarification per SME review --- modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc index 3b563cbee8..1ce4088df8 100644 --- a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc +++ b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc @@ -135,7 +135,8 @@ For an example cluster configuration that uses the same IAM credentials for both * If you want to configure authentication to AWS Glue separately from authentication to S3, there are equivalent credential configuration properties named `iceberg_rest_catalog_aws_*` that override the object storage credentials. These properties only apply to REST catalog authentication, and never to S3 authentication: + -- -** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`]. To use the cluster's IAM role, set the property to `sts`. To use static credentials, set to `config_file`. +** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`]. *** Set the property to `sts` if you want to use the cluster's default IAM role. +*** Set to `config_file` if you want to scope Glue access through your own IAM user and policy instead of the cluster's default IAM role, or if you want to use static credentials. ** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] (static credentials only) ** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] (static credentials only), added as a secret value (see the <> for details) ** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] From 03619bba7cd6de5949cbfdaf9a68047abe0fd3a4 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 27 May 2026 18:58:01 -0700 Subject: [PATCH 3/4] Fix list item rendering --- modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc index 1ce4088df8..036db47770 100644 --- a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc +++ b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc @@ -135,8 +135,9 @@ For an example cluster configuration that uses the same IAM credentials for both * If you want to configure authentication to AWS Glue separately from authentication to S3, there are equivalent credential configuration properties named `iceberg_rest_catalog_aws_*` that override the object storage credentials. These properties only apply to REST catalog authentication, and never to S3 authentication: + -- -** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`]. *** Set the property to `sts` if you want to use the cluster's default IAM role. -*** Set to `config_file` if you want to scope Glue access through your own IAM user and policy instead of the cluster's default IAM role, or if you want to use static credentials. +** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`] +*** Set the property to `sts` if you want to use the cluster's default IAM role. +*** Set to `config_file` if you want to scope Glue access through your own IAM user and policy instead of the cluster's default IAM role, or if you want to use static credentials. ** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] (static credentials only) ** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] (static credentials only), added as a secret value (see the <> for details) ** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] From 67b4496ae0898f6bc9d5d322a34f0ce2a48a81f3 Mon Sep 17 00:00:00 2001 From: Kat Batuigas Date: Wed, 27 May 2026 19:13:09 -0700 Subject: [PATCH 4/4] Fix object storage property xrefs --- .../manage/pages/iceberg/iceberg-topics-aws-glue.adoc | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc index 036db47770..898c1ddf80 100644 --- a/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc +++ b/modules/manage/pages/iceberg/iceberg-topics-aws-glue.adoc @@ -120,10 +120,10 @@ You must configure credentials for the AWS Glue Data Catalog integration in eith * Allow Redpanda to use the same `cloud_storage_*` credential properties configured for S3. This is the recommended approach. * If you want to configure authentication to AWS Glue separately from authentication to S3, there are equivalent credential configuration properties named `iceberg_rest_catalog_aws_*` that override the object storage credentials. These properties only apply to REST catalog authentication, and never to S3 authentication: -** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`] overrides config_ref:cloud_storage_credentials_source,true,properties/cluster-properties[`cloud_storage_credentials_source`] -** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] overrides config_ref:cloud_storage_access_key,true,properties/cluster-properties[`cloud_storage_access_key`] -** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] overrides config_ref:cloud_storage_secret_key,true,properties/cluster-properties[`cloud_storage_secret_key`] -** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] overrides config_ref:cloud_storage_region,true,properties/cluster-properties[`cloud_storage_region`] +** config_ref:iceberg_rest_catalog_credentials_source,true,properties/cluster-properties[`iceberg_rest_catalog_credentials_source`] overrides config_ref:cloud_storage_credentials_source,true,properties/object-storage-properties[`cloud_storage_credentials_source`] +** config_ref:iceberg_rest_catalog_aws_access_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_access_key`] overrides config_ref:cloud_storage_access_key,true,properties/object-storage-properties[`cloud_storage_access_key`] +** config_ref:iceberg_rest_catalog_aws_secret_key,true,properties/cluster-properties[`iceberg_rest_catalog_aws_secret_key`] overrides config_ref:cloud_storage_secret_key,true,properties/object-storage-properties[`cloud_storage_secret_key`] +** config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] overrides config_ref:cloud_storage_region,true,properties/object-storage-properties[`cloud_storage_region`] endif::[] ifdef::env-cloud[] @@ -187,7 +187,7 @@ Use your own values for the following placeholders: + -- * ``: A unique namespace for this cluster's Iceberg tables. Each Redpanda cluster that writes to the same Glue catalog must use a distinct namespace to avoid table name collisions. If omitted, the default namespace `redpanda` is used. -* ``: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in either your config_ref:cloud_storage_region,true,properties/cluster-properties[`cloud_storage_region`] or config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property. +* ``: The AWS region where your Data Catalog is located. The region in the AWS Glue endpoint must match the region specified in either your config_ref:cloud_storage_region,true,properties/object-storage-properties[`cloud_storage_region`] or config_ref:iceberg_rest_catalog_aws_region,true,properties/cluster-properties[`iceberg_rest_catalog_aws_region`] property. * `` and ``: AWS Glue requires you to specify the base location where Redpanda stores Iceberg data and metadata files. You must use an S3 URI; for example, `s3:///iceberg`. This must be the same bucket used for object storage (your `cloud_storage_bucket`). You cannot specify a different bucket for Iceberg data. + `` is a name you choose (such as `iceberg`) as the logical name for the warehouse represented by all Redpanda Iceberg topic data in the cluster.