From 95805b21258f2590a6961fffb978070353ccadf3 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 11 Mar 2026 23:01:51 -0400 Subject: [PATCH 01/14] feat(topics): align Topics navtree with self-managed and add Cloud Topics - Update Develop navtree to mirror self-managed structure: Topics parent (develop:manage-topics/index.adoc) with Manage Topics and Cloud Topics as children - Remove create-topic.adoc as a standalone nav entry - Merge create-topic.adoc content into config-topics.adoc as the first section (== Create a Topic), with page-aliases to preserve old URLs - Add develop:manage-topics/cloud-topics.adoc to the navtree after Manage Topics for single-sourced Cloud Topics content Co-Authored-By: Claude Sonnet 4.6 --- modules/ROOT/nav.adoc | 5 +- modules/get-started/pages/config-topics.adoc | 48 +++++++++++++++++++- 2 files changed, 50 insertions(+), 3 deletions(-) diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index 60e12bcec..f0756b745 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -397,8 +397,9 @@ * xref:develop:index.adoc[Develop] ** xref:develop:kafka-clients.adoc[] -** xref:get-started:create-topic.adoc[Create a Topic] -** xref:get-started:config-topics.adoc[Manage Topics] +** xref:develop:manage-topics/index.adoc[] +*** xref:get-started:config-topics.adoc[Manage Topics] +*** xref:develop:manage-topics/cloud-topics.adoc[] ** xref:develop:produce-data/index.adoc[Produce Data] *** xref:develop:produce-data/configure-producers.adoc[] *** xref:develop:produce-data/idempotent-producers.adoc[Idempotent Producers] diff --git a/modules/get-started/pages/config-topics.adoc b/modules/get-started/pages/config-topics.adoc index d076d716a..db4c3e73c 100644 --- a/modules/get-started/pages/config-topics.adoc +++ b/modules/get-started/pages/config-topics.adoc @@ -1,4 +1,50 @@ = Manage Topics :description: Learn how to create topics, update topic configurations, and delete topics or records. +:page-aliases: get-started:create-topic.adoc, cloud:create-topic.adoc -include::ROOT:develop:config-topics.adoc[tag=single-source] \ No newline at end of file +== Create a Topic + +Topics provide a way to organize events. After creating a cluster, you can create a topic in it. Each cluster can have up to 40,000 topics. + +Topic properties are populated from information stored in the broker. Redpanda features, such as Tiered Storage, are enabled and configured by default in Redpanda Cloud. You can optionally overwrite some settings. + +include::develop:partial$topic-properties-warning.adoc[] + +++++ +
+++++ + +[cols="1,2"] +|=== +| Property | Description + +| *Partitions* +| The number of partitions for the topic. + +| *Replication factor* +| The number of partition replicas for the topic. + +Redpanda Cloud requires a minimum of 3 topic replicas. If a topic is created with a replication factor of 1, Redpanda resets the replication factor to 3. + +| *Cleanup policy* +| The policy that determines how to clean up old log segments. + +The default is *delete*. + +| *Retention time* +| The maximum length of time to keep messages in a topic. + +The default is *7 days*. + +| *Retention size* +| The maximum size of each partition. If a partition reaches this size and more messages are added, the oldest messages are deleted. + +The default is *infinite*. + +| *Message size* +| The maximum size of a message or batch for a newly-created topic. + +The default is *20 MiB* for BYOC and Dedicated clusters, and *8 MiB* for Serverless clusters. You can increase this value up to *32 MiB* for BYOC and Dedicated clusters, and *20 MiB* for Serverless clusters, with the `message.max.bytes` topic property. +|=== + +include::ROOT:develop:config-topics.adoc[tag=single-source] From 96aa81da40b797de43d2cdaf6f3464a4a00edc3e Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 11 Mar 2026 23:05:50 -0400 Subject: [PATCH 02/14] chore: delete create-topic.adoc Content merged into config-topics.adoc as == Create a Topic section. Page aliases on config-topics.adoc handle redirect from old URL. Co-Authored-By: Claude Sonnet 4.6 --- modules/get-started/pages/create-topic.adoc | 50 --------------------- 1 file changed, 50 deletions(-) delete mode 100644 modules/get-started/pages/create-topic.adoc diff --git a/modules/get-started/pages/create-topic.adoc b/modules/get-started/pages/create-topic.adoc deleted file mode 100644 index 97b2b20aa..000000000 --- a/modules/get-started/pages/create-topic.adoc +++ /dev/null @@ -1,50 +0,0 @@ -= Create a Topic -:description: Learn how to create a topic for a Redpanda Cloud cluster. -:page-aliases: cloud:create-topic.adoc - - - -Topics provide a way to organize events. After creating a cluster, you can create a topic in it. Each cluster can have up to 40,000 topics. - -Topic properties are populated from information stored in the broker. Redpanda features, such as Tiered Storage, are enabled and configured by default in Redpanda Cloud. You can optionally overwrite some settings. - -include::develop:partial$topic-properties-warning.adoc[] - -++++ -
-++++ - -[cols="1,2"] -|=== -| Property | Description - -| *Partitions* -| The number of partitions for the topic. - -| *Replication factor* -| The number of partition replicas for the topic. - -Redpanda Cloud requires a minimum of 3 topic replicas. If a topic is created with a replication factor of 1, Redpanda resets the replication factor to 3. - -| *Cleanup policy* -| The policy that determines how to clean up old log segments. - -The default is *delete*. - -| *Retention time* -| The maximum length of time to keep messages in a topic. - -The default is *7 days*. - -| *Retention size* -| The maximum size of each partition. If a partition reaches this size and more messages are added, the oldest messages are deleted. - -The default is *infinite*. - -| *Message size* -| The maximum size of a message or batch for a newly-created topic. - -The default is *20 MiB* for BYOC and Dedicated clusters, and *8 MiB* for Serverless clusters. You can increase this value up to *32 MiB* for BYOC and Dedicated clusters, and *20 MiB* for Serverless clusters, with the `message.max.bytes` topic property. -|=== - - From f46200d13ca2fdcf1e00f8a4d3b612a7eb513088 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 11 Mar 2026 23:12:45 -0400 Subject: [PATCH 03/14] fix(nav): add explicit labels for Topics and Cloud Topics nav entries Cross-component xrefs don't resolve titles automatically in the cloud-docs build. Add explicit labels to prevent raw page IDs from appearing in the navtree. Co-Authored-By: Claude Sonnet 4.6 --- modules/ROOT/nav.adoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index f0756b745..b758c66fb 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -397,9 +397,9 @@ * xref:develop:index.adoc[Develop] ** xref:develop:kafka-clients.adoc[] -** xref:develop:manage-topics/index.adoc[] +** xref:develop:manage-topics/index.adoc[Topics] *** xref:get-started:config-topics.adoc[Manage Topics] -*** xref:develop:manage-topics/cloud-topics.adoc[] +*** xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] ** xref:develop:produce-data/index.adoc[Produce Data] *** xref:develop:produce-data/configure-producers.adoc[] *** xref:develop:produce-data/idempotent-producers.adoc[Idempotent Producers] From 4d94dac35239f68439e41e01556e22f77e5def46 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 11 Mar 2026 23:26:43 -0400 Subject: [PATCH 04/14] feat(topics): add Topics index page and update nav to use local page Creates get-started:topics-index.adoc as a cloud-docs-specific index for the Topics section, listing Manage Topics and Cloud Topics with one-sentence descriptions. Updates the nav to reference the local page instead of the cross-component develop:manage-topics/index.adoc, which was causing the raw page ID to display in the navtree. Co-Authored-By: Claude Sonnet 4.6 --- modules/ROOT/nav.adoc | 2 +- modules/get-started/pages/topics-index.adoc | 7 +++++++ 2 files changed, 8 insertions(+), 1 deletion(-) create mode 100644 modules/get-started/pages/topics-index.adoc diff --git a/modules/ROOT/nav.adoc b/modules/ROOT/nav.adoc index b758c66fb..5774a7789 100644 --- a/modules/ROOT/nav.adoc +++ b/modules/ROOT/nav.adoc @@ -397,7 +397,7 @@ * xref:develop:index.adoc[Develop] ** xref:develop:kafka-clients.adoc[] -** xref:develop:manage-topics/index.adoc[Topics] +** xref:get-started:topics-index.adoc[Topics] *** xref:get-started:config-topics.adoc[Manage Topics] *** xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] ** xref:develop:produce-data/index.adoc[Produce Data] diff --git a/modules/get-started/pages/topics-index.adoc b/modules/get-started/pages/topics-index.adoc new file mode 100644 index 000000000..97c91b766 --- /dev/null +++ b/modules/get-started/pages/topics-index.adoc @@ -0,0 +1,7 @@ += Topics +:description: Create and configure standard Redpanda topics, or use Cloud Topics for high-throughput, cost-optimized workloads that trade latency for lower storage costs. +:page-layout: index + +* xref:get-started:config-topics.adoc[Manage Topics] — Create topics and configure topic properties such as retention, cleanup policy, and partition count. + +* xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] — Use Cloud Topics for high-throughput workloads that can trade higher latency for lower object storage costs. From f04c77bd908cb8dbfa829109b5eefa8d7a01e2ee Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 11 Mar 2026 23:28:04 -0400 Subject: [PATCH 05/14] chore: retrigger build From ae319224f29c43659cdf24859e6b9edfe3b249f8 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 11 Mar 2026 23:49:38 -0400 Subject: [PATCH 06/14] fix(topics): update index descriptions and remove page-layout index Updates Manage Topics and Cloud Topics descriptions per editorial feedback. Removes page-layout: index which was auto-generating a duplicate Manage Topics card at the bottom of the page. Co-Authored-By: Claude Sonnet 4.6 --- modules/get-started/pages/topics-index.adoc | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) diff --git a/modules/get-started/pages/topics-index.adoc b/modules/get-started/pages/topics-index.adoc index 97c91b766..cce187084 100644 --- a/modules/get-started/pages/topics-index.adoc +++ b/modules/get-started/pages/topics-index.adoc @@ -1,7 +1,6 @@ = Topics :description: Create and configure standard Redpanda topics, or use Cloud Topics for high-throughput, cost-optimized workloads that trade latency for lower storage costs. -:page-layout: index -* xref:get-started:config-topics.adoc[Manage Topics] — Create topics and configure topic properties such as retention, cleanup policy, and partition count. +* xref:get-started:config-topics.adoc[Manage Topics] — Learn how to create topics, update topic configurations, and delete topics or records. -* xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] — Use Cloud Topics for high-throughput workloads that can trade higher latency for lower object storage costs. +* xref:develop:manage-topics/cloud-topics.adoc[Cloud Topics] — Use Cloud Topics when you require low-cost, high-throughput streaming storage for applications with flexible latency requirements. From ca003f6b9fd357dbed2107ed76e061d7ca086cdc Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Wed, 11 Mar 2026 23:50:37 -0400 Subject: [PATCH 07/14] ci: trigger build From 9fdca988b665587ac55e747bd784676418efd426 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Thu, 12 Mar 2026 00:00:46 -0400 Subject: [PATCH 08/14] feat(topics): add cloud-topics.adoc wrapper page Single-sources content from ROOT:develop:manage-topics/cloud-topics.adoc (redpanda-data/docs PR #1469) using the same include pattern as other develop module pages in cloud-docs. Co-Authored-By: Claude Sonnet 4.6 --- modules/develop/pages/manage-topics/cloud-topics.adoc | 4 ++++ 1 file changed, 4 insertions(+) create mode 100644 modules/develop/pages/manage-topics/cloud-topics.adoc diff --git a/modules/develop/pages/manage-topics/cloud-topics.adoc b/modules/develop/pages/manage-topics/cloud-topics.adoc new file mode 100644 index 000000000..2a9607748 --- /dev/null +++ b/modules/develop/pages/manage-topics/cloud-topics.adoc @@ -0,0 +1,4 @@ += Manage Cloud Topics +:description: Cloud Topics are Redpanda topics that enable users to trade off latency for lower costs. + +include::ROOT:develop:manage-topics/cloud-topics.adoc[tag=single-source] From 625b35d879c798f5d25b748bc8d857605fc12165 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Thu, 12 Mar 2026 00:07:14 -0400 Subject: [PATCH 09/14] ci: trigger build From bcf5753a619194b3c87ebff17448a4cb001f3225 Mon Sep 17 00:00:00 2001 From: Joyce Fee Date: Thu, 12 Mar 2026 00:24:18 -0400 Subject: [PATCH 10/14] fix(topics): embed cloud-topics content directly pending docs PR merge The include::ROOT:develop:manage-topics/cloud-topics.adoc approach requires redpanda-data/docs PR #1469 to merge first. Until then, include the content directly with single-source tags. Self-managed-only prereqs (Tiered Storage, Enterprise license, Docker) are excluded since they don't apply to Cloud. Co-Authored-By: Claude Sonnet 4.6 --- .../pages/manage-topics/cloud-topics.adoc | 43 ++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/modules/develop/pages/manage-topics/cloud-topics.adoc b/modules/develop/pages/manage-topics/cloud-topics.adoc index 2a9607748..9f49c2194 100644 --- a/modules/develop/pages/manage-topics/cloud-topics.adoc +++ b/modules/develop/pages/manage-topics/cloud-topics.adoc @@ -1,4 +1,45 @@ = Manage Cloud Topics :description: Cloud Topics are Redpanda topics that enable users to trade off latency for lower costs. +// tag::single-source[] -include::ROOT:develop:manage-topics/cloud-topics.adoc[tag=single-source] +Starting in v26.1, Redpanda provides Cloud Topics to support multi-modal streaming workloads in the most cost-effective way possible: as a configurable option on a single cluster. While standard Redpanda xref:get-started:config-topics.adoc[topics] that use local storage or Tiered Storage are ideal for latency-tolerant workloads (for example, for audit logs or analytics), Cloud Topics are optimized for latency-tolerant, high-throughput workloads where cross-AZ networking charges are a major consideration that can become the dominant cost driver at high throughput. These workloads can include observability streams, offline analytics, AI/ML model training data feeds, or development environments that have flexible latency requirements. + +Instead of replicating every byte across expensive network links, Cloud Topics leverage durable, inexpensive Cloud storage (S3, ADLS, GCS, MinIO) as the primary mechanism to back up data. This eliminates over 90% of the cost of replicating data over network links in multi-AZ clusters. The difference in end-to-end latency is typically between 500 ms - 1 s, which is often acceptable for many streaming workloads, and can unlock new streaming use cases that previously were not cost effective. + +With Cloud Topics, data from the client is not acknowledged until it is uploaded to cloud storage. This maintains durability in the face of infrastructure failures, but results in an increase in both produce latency and end-to-end latency, driven by both batching of produced data and the inherent latency of the underlying object store. You should generally expect latencies in the range of 250 ms - 1 s. + +== Prerequisites + +- xref:manage:rpk/rpk-install.adoc[Install rpk] v26.1 or later. + +== Limitations + +- The Remote Read Replicas feature is not supported for Cloud Topics. + +- Once created, a Cloud Topic cannot be subsequently converted back to a standard Redpanda topic that uses local or Tiered Storage. + +== Enable Cloud Topics + +NOTE: This configuration update requires a restart. + +To enable Cloud Topics for a cluster: + +[,bash] +---- +rpk cluster config set cloud_topics_enabled=true +---- + +After enabling Cloud Topics, you can proceed to create new Cloud Topics: + +[,bash] +---- +rpk topic create -c redpanda.storage.mode=cloud +TOPIC STATUS +audit.analytics.may2025 OK +---- + +You can make a topic a Cloud Topic only at topic creation time. + +In addition to replication, cross-AZ ingress (producer) and egress (consumer) traffic can also contribute substantially to cloud networking cost. When running multi-AZ clusters in general, Redpanda strongly recommends using xref:consume-data/follower-fetching.adoc[Follower Fetching], which allows consumers to avoid crossing network zones. When possible, you can use xref:produce-data/leader-pinning.adoc[leader pinning], which positions a topic's partition leader close to the producers, providing a similar benefit for ingress traffic. These features can add additional savings to the replication cost savings of Cloud Topics. + +// end::single-source[] From e9a9145743a95632be81c01d4b86f8e993b7aa98 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Wed, 11 Mar 2026 22:51:42 -0600 Subject: [PATCH 11/14] feat(topics): add Cloud Topics wrapper page and update config-topics include path - Create develop:manage-topics/cloud-topics.adoc wrapper that includes single-source content from docs repo - Update config-topics.adoc include path to match new manage-topics/ directory structure Co-Authored-By: Claude Opus 4.6 --- .../pages/manage-topics/cloud-topics.adoc | 43 +------------------ modules/get-started/pages/config-topics.adoc | 2 +- 2 files changed, 2 insertions(+), 43 deletions(-) diff --git a/modules/develop/pages/manage-topics/cloud-topics.adoc b/modules/develop/pages/manage-topics/cloud-topics.adoc index 9f49c2194..2a9607748 100644 --- a/modules/develop/pages/manage-topics/cloud-topics.adoc +++ b/modules/develop/pages/manage-topics/cloud-topics.adoc @@ -1,45 +1,4 @@ = Manage Cloud Topics :description: Cloud Topics are Redpanda topics that enable users to trade off latency for lower costs. -// tag::single-source[] -Starting in v26.1, Redpanda provides Cloud Topics to support multi-modal streaming workloads in the most cost-effective way possible: as a configurable option on a single cluster. While standard Redpanda xref:get-started:config-topics.adoc[topics] that use local storage or Tiered Storage are ideal for latency-tolerant workloads (for example, for audit logs or analytics), Cloud Topics are optimized for latency-tolerant, high-throughput workloads where cross-AZ networking charges are a major consideration that can become the dominant cost driver at high throughput. These workloads can include observability streams, offline analytics, AI/ML model training data feeds, or development environments that have flexible latency requirements. - -Instead of replicating every byte across expensive network links, Cloud Topics leverage durable, inexpensive Cloud storage (S3, ADLS, GCS, MinIO) as the primary mechanism to back up data. This eliminates over 90% of the cost of replicating data over network links in multi-AZ clusters. The difference in end-to-end latency is typically between 500 ms - 1 s, which is often acceptable for many streaming workloads, and can unlock new streaming use cases that previously were not cost effective. - -With Cloud Topics, data from the client is not acknowledged until it is uploaded to cloud storage. This maintains durability in the face of infrastructure failures, but results in an increase in both produce latency and end-to-end latency, driven by both batching of produced data and the inherent latency of the underlying object store. You should generally expect latencies in the range of 250 ms - 1 s. - -== Prerequisites - -- xref:manage:rpk/rpk-install.adoc[Install rpk] v26.1 or later. - -== Limitations - -- The Remote Read Replicas feature is not supported for Cloud Topics. - -- Once created, a Cloud Topic cannot be subsequently converted back to a standard Redpanda topic that uses local or Tiered Storage. - -== Enable Cloud Topics - -NOTE: This configuration update requires a restart. - -To enable Cloud Topics for a cluster: - -[,bash] ----- -rpk cluster config set cloud_topics_enabled=true ----- - -After enabling Cloud Topics, you can proceed to create new Cloud Topics: - -[,bash] ----- -rpk topic create -c redpanda.storage.mode=cloud -TOPIC STATUS -audit.analytics.may2025 OK ----- - -You can make a topic a Cloud Topic only at topic creation time. - -In addition to replication, cross-AZ ingress (producer) and egress (consumer) traffic can also contribute substantially to cloud networking cost. When running multi-AZ clusters in general, Redpanda strongly recommends using xref:consume-data/follower-fetching.adoc[Follower Fetching], which allows consumers to avoid crossing network zones. When possible, you can use xref:produce-data/leader-pinning.adoc[leader pinning], which positions a topic's partition leader close to the producers, providing a similar benefit for ingress traffic. These features can add additional savings to the replication cost savings of Cloud Topics. - -// end::single-source[] +include::ROOT:develop:manage-topics/cloud-topics.adoc[tag=single-source] diff --git a/modules/get-started/pages/config-topics.adoc b/modules/get-started/pages/config-topics.adoc index db4c3e73c..39dea7edb 100644 --- a/modules/get-started/pages/config-topics.adoc +++ b/modules/get-started/pages/config-topics.adoc @@ -47,4 +47,4 @@ The default is *infinite*. The default is *20 MiB* for BYOC and Dedicated clusters, and *8 MiB* for Serverless clusters. You can increase this value up to *32 MiB* for BYOC and Dedicated clusters, and *20 MiB* for Serverless clusters, with the `message.max.bytes` topic property. |=== -include::ROOT:develop:config-topics.adoc[tag=single-source] +include::ROOT:develop:manage-topics/config-topics.adoc[tag=single-source] From e42fd49d3035348f4a27f34986819b709d8c1e66 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Wed, 11 Mar 2026 22:53:02 -0600 Subject: [PATCH 12/14] ci: point playbook to DOC-1602 branch for Netlify preview Co-Authored-By: Claude Opus 4.6 --- local-antora-playbook.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/local-antora-playbook.yml b/local-antora-playbook.yml index d8d478c82..9448832d9 100644 --- a/local-antora-playbook.yml +++ b/local-antora-playbook.yml @@ -15,7 +15,7 @@ content: - url: . branches: HEAD - url: https://github.com/redpanda-data/documentation - branches: [main, v/*, shared, site-search] + branches: [DOC-1602, v/*, shared, site-search] - url: https://github.com/redpanda-data/docs-site branches: [main] start_paths: [home] From 60da681456f39c02042c82db364bfdbb55410fb1 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Wed, 11 Mar 2026 22:56:41 -0600 Subject: [PATCH 13/14] ci: trigger Netlify preview build Co-Authored-By: Claude Opus 4.6 From 4931b9f3778fc4811cb3907bd20a3cc2dc379076 Mon Sep 17 00:00:00 2001 From: micheleRP Date: Wed, 11 Mar 2026 23:10:12 -0600 Subject: [PATCH 14/14] fix: remove create-topic.adoc alias that conflicts with main branch The page alias conflicts with the actual create-topic.adoc file on main during Netlify's merge-based preview builds. The alias can be re-added after this PR merges and the file is deleted from main. Co-Authored-By: Claude Opus 4.6 --- modules/get-started/pages/config-topics.adoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/modules/get-started/pages/config-topics.adoc b/modules/get-started/pages/config-topics.adoc index 39dea7edb..82e61fbe1 100644 --- a/modules/get-started/pages/config-topics.adoc +++ b/modules/get-started/pages/config-topics.adoc @@ -1,6 +1,6 @@ = Manage Topics :description: Learn how to create topics, update topic configurations, and delete topics or records. -:page-aliases: get-started:create-topic.adoc, cloud:create-topic.adoc +:page-aliases: cloud:create-topic.adoc == Create a Topic