[feat][monitor] PIP-447 for Customizable Prometheus Labels for Topic Metrics#24991
Open
coderzc wants to merge 30 commits intoapache:masterfrom
Open
[feat][monitor] PIP-447 for Customizable Prometheus Labels for Topic Metrics#24991coderzc wants to merge 30 commits intoapache:masterfrom
coderzc wants to merge 30 commits intoapache:masterfrom
Conversation
coderzc
force-pushed
the
feat_pip_447
branch
from
d8179f9 to
bc8eab2
Compare
lhotari
reviewed
Jan 2, 2026
Member
lhotari
left a comment
lhotari
left a comment
There was a problem hiding this comment.
One comment about the usage of PolicyName.MAX_SUBSCRIPTIONS
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/PersistentTopics.java
Outdated
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment.
Pull request overview
This PR implements PIP-447 to add support for customizable Prometheus labels on topic metrics, allowing operators to attach custom metadata (like SLA tier, team ownership, etc.) to topics that will be exposed as Prometheus labels.
Key changes:
- Adds
customMetricLabelsfield to TopicPolicies with clone and validation support - Implements REST APIs, admin client methods, and CLI commands for managing custom metric labels
- Integrates custom labels into Prometheus metrics generation in TopicStats
- Adds configuration options to enable/disable the feature and control allowed label keys and value lengths
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 8 comments.
Show a summary per file
| File | Description |
|---|---|
| pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/TopicPolicies.java | Adds customMetricLabels field with proper cloning and accessor methods |
| pulsar-common/src/main/java/org/apache/pulsar/common/policies/data/HierarchyTopicPolicies.java | Adds PolicyHierarchyValue for customMetricLabels to support policy hierarchy |
| pulsar-client-admin-api/src/main/java/org/apache/pulsar/client/admin/TopicPolicies.java | Adds API methods for setting, getting, and removing custom metric labels |
| pulsar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/TopicPoliciesImpl.java | Implements client-side REST API calls for custom metric labels operations |
| pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTopics.java | Adds CLI commands for managing custom metric labels in CmdTopics |
| pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTopicPolicies.java | Adds CLI commands for managing custom metric labels in CmdTopicPolicies |
| pulsar-client-tools/src/test/java/org/apache/pulsar/admin/cli/CmdTopicPoliciesTest.java | Adds comprehensive tests for CLI command functionality |
| pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/PersistentTopics.java | Adds REST endpoints for custom metric labels operations |
| pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java | Implements business logic for setting, getting, and removing custom metric labels with validation |
| pulsar-broker/src/main/java/org/apache/pulsar/broker/service/AbstractTopic.java | Updates topic policy loading to include customMetricLabels |
| pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/prometheus/TopicStats.java | Modifies metrics generation to include custom labels in Prometheus output |
| pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/prometheus/NamespaceStatsAggregator.java | Retrieves and passes custom metric labels to TopicStats for metrics generation |
| pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/TopicPoliciesTest.java | Adds integration tests for custom metric labels feature including validation scenarios |
| pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java | Adds three configuration properties for the custom metric labels feature |
| conf/broker.conf | Documents the new configuration properties with examples |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/v2/PersistentTopics.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
Outdated
Show resolved
Hide resolved
pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
Outdated
Show resolved
Hide resolved
...ar-client-admin/src/main/java/org/apache/pulsar/client/admin/internal/TopicPoliciesImpl.java
Outdated
Show resolved
Hide resolved
pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdTopics.java
Outdated
Show resolved
Hide resolved
…s and expose them in Prometheus
…tency in topic statistics
…with CLI commands
…configuration - Add namespace-level allowed_topic_properties_for_metrics field in Policies.java - Add admin API methods for set/get/remove allowedTopicPropertiesForMetrics - Add REST endpoints in Namespaces.java (v2) - Add CLI commands: set-allowed-topic-properties-for-metrics, get-allowed-topic-properties-for-metrics, remove-allowed-topic-properties-for-metrics - Integrate with Prometheus metrics generation to merge broker-level and namespace-level configurations - Add test for namespace-level allowedTopicPropertiesForMetrics
Since PIP-447 now uses topic properties instead of topic policies, this removes: - TopicPolicies.customMetricLabels field and methods - HierarchyTopicPolicies.customMetricLabels field - TopicPolicies admin API methods (set/get/removeCustomMetricLabels) - CLI commands in CmdTopicPolicies and CmdTopics - TopicPoliciesImpl implementation - REST API endpoints in PersistentTopics.java (v2) - Internal methods in PersistentTopicsBase.java - References in AbstractTopic.java - Unit tests in CmdTopicPoliciesTest.java - Renamed test method in PrometheusMetricsLabelsTest.java
…rectly and update usage in NamespacesBase
…x config validation order
…nore hidden directories
coderzc
force-pushed
the
feat_pip_447
branch
from
28db9d2 to
c431957
Compare
…metrics - Move logic for fetching and filtering custom metric labels to TopicStats - Simplify NamespaceStatsAggregator by delegating label handling to TopicStats - Ensure only allowed topic properties are exposed as custom metric labels
…tats to AbstractTopic
coderzc
force-pushed
the
feat_pip_447
branch
from
4b7fb7b to
804b972
Compare
…_METRIC_LABELS for consistency
coderzc
added
the
type/feature
coderzc
requested review from
BewareMyPower,
Technoboy-,
codelipenghui,
lhotari,
poorbarcode and
shibd
coderzc
force-pushed
the
feat_pip_447
branch
from
0c837f1 to
8c5d6f5
Compare
coderzc
force-pushed
the
feat_pip_447
branch
from
8c5d6f5 to
2e1cc8a
Compare
…xception with detailed error messages
shibd
approved these changes
Mar 6, 2026
…cPropertyKeysForMetrics for clarity Renamed the configuration, variable names, and API methods to use 'PropertyKeys' instead of 'Properties' to avoid ambiguity between property keys and values.
coderzc
force-pushed
the
feat_pip_447
branch
2 times, most recently
from
1852a3a to
d59fd5a
Compare
…cPropertyKeysForMetrics for clarity - part 2 Additional fixes: - PulsarService.java: fix config getter call - AbstractTopic.java: fix helper method and getter call - Namespaces.java: fix policies field reference (line 2648) - NamespacesTest.java: fix test method names and API calls - PrometheusMetricsLabelsTest.java: fix setter calls
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #24991 +/- ##
=============================================
+ Coverage 29.94% 72.61% +42.66%
- Complexity 107 34591 +34484
=============================================
Files 1899 1968 +69
Lines 151150 156120 +4970
Branches 17230 17783 +553
=============================================
+ Hits 45269 113369 +68100
+ Misses 98854 33705 -65149
- Partials 7027 9046 +2019
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PIP: https://github.com/apache/pulsar/blob/master/pip/pip-447.md
Motivation
Modifications
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: