SOLR-18165 Use dot separated metric names#4223
Conversation
There was a problem hiding this comment.
Pull request overview
This PR implements SOLR-18165 by aligning Solr’s OTLP-exported metric names with OpenTelemetry’s dot-separated naming conventions, while keeping Prometheus output stable and preserving Prometheus-style name filtering.
Changes:
- Renames OTLP metric instruments from underscore-separated to dot-separated names across core, node, and CrossDC metrics.
- Updates
FilterablePrometheusMetricReaderto filter using Prometheus base names (getMetadata().getPrometheusName()) rather than OTel internal names. - Adds release-facing documentation: Solr 10.1 upgrade note + an unreleased changelog YAML entry describing the breaking change.
Reviewed changes
Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc | Documents the OTLP metric-name breaking change and Prometheus impact/workaround. |
| solr/modules/cross-dc/src/java/org/apache/solr/crossdc/update/processor/ProducerMetrics.java | Updates CrossDC producer OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/update/UpdateShardHandler.java | Updates executor metric prefix to dot-separated format. |
| solr/core/src/java/org/apache/solr/update/UpdateLog.java | Updates update log OTLP metric names to dot-separated format (keeping selected underscores). |
| solr/core/src/java/org/apache/solr/update/SolrIndexWriter.java | Updates index writer OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/update/PeerSyncWithLeader.java | Updates peer-sync-with-leader OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/update/PeerSync.java | Updates peer-sync OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/update/DirectUpdateHandler2.java | Updates update handler OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/search/stats/StatsCache.java | Updates stats cache OTLP metric name to dot-separated format. |
| solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java | Updates index searcher OTLP metric names/prefixes to dot-separated format. |
| solr/core/src/java/org/apache/solr/search/SolrFieldCacheBean.java | Updates field cache OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/metrics/otel/instruments/AttributedInstrumentFactory.java | Updates core→node metric-name prefix substitution to dot-separated form. |
| solr/core/src/java/org/apache/solr/metrics/otel/FilterablePrometheusMetricReader.java | Switches name filtering to Prometheus base names to keep /admin/metrics?wt=prometheus&name=... working post-rename. |
| solr/core/src/java/org/apache/solr/handler/component/SuggestComponent.java | Updates suggester OTLP metric name to dot-separated format. |
| solr/core/src/java/org/apache/solr/handler/component/HttpShardHandlerFactory.java | Updates HTTP shard executor metric prefix to dot-separated format. |
| solr/core/src/java/org/apache/solr/handler/admin/CoreAdminHandler.java | Updates node executor metric prefix to dot-separated format. |
| solr/core/src/java/org/apache/solr/handler/RequestHandlerBase.java | Updates request handler OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/handler/ReplicationHandler.java | Updates replication OTLP metric names to dot-separated format (keeping selected underscores). |
| solr/core/src/java/org/apache/solr/core/SolrCore.java | Updates core/searcher OTLP metric names to dot-separated format. |
| solr/core/src/java/org/apache/solr/core/CoreContainer.java | Updates node cache/executor metric prefixes to dot-separated format. |
| changelog/unreleased/SOLR-18165-dot-separated-metric-names.yml | Adds changelog entry describing the change and its back-compat impact. |
Comments suppressed due to low confidence (1)
solr/core/src/java/org/apache/solr/metrics/otel/FilterablePrometheusMetricReader.java:80
- The suffix-stripping logic only handles
_total,_sum,_bucket,_created, and_info, but Prometheus histograms/summaries also expose a*_countseries. If a user filters withname=<metric>_count, it won't be sanitized to the base name and the metric won't match. Consider adding_countto the recognized suffix set (and keep the doc/examples in sync).
// Users may filter by Prometheus-format names (e.g. "solr_core_requests") or with a
// Prometheus type suffix (e.g. "solr_core_requests_total"). Strip any such suffix so we can
// compare against the Prometheus base name returned by getMetadata().getPrometheusName().
Set<String> sanitizedNames =
includedNames.stream()
.map(
(name) -> {
for (String suffix : PROM_SUFFIXES) {
if (name.endsWith(suffix)) {
return name.substring(0, name.lastIndexOf(suffix));
}
}
return name;
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
| @@ -84,7 +85,18 @@ public MetricSnapshots collect( | |||
| if (sanitizedNames.isEmpty()) { | |||
| snapshotsToFilter = super.collect(); | |||
| } else { | |||
| snapshotsToFilter = super.collect(sanitizedNames::contains); | |||
| // We collect all metrics and filter by Prometheus name rather than using | |||
| // super.collect(Predicate) which matches on OTel internal names. This avoids a mismatch | |||
| // when OTel names use dot-separators (e.g. "solr.core.requests") but users filter by the | |||
| // Prometheus underscore-format name they see in the output (e.g. "solr_core_requests"). | |||
| MetricSnapshots all = super.collect(); | |||
| MetricSnapshots.Builder nameFiltered = MetricSnapshots.builder(); | |||
| for (MetricSnapshot snapshot : all) { | |||
| if (sanitizedNames.contains(snapshot.getMetadata().getPrometheusName())) { | |||
| nameFiltered.metricSnapshot(snapshot); | |||
| } | |||
| } | |||
| snapshotsToFilter = nameFiltered.build(); | |||
Co-authored-by: Matthew Biscocho <biscocho.matthew@gmail.com>
solr/core/src/java/org/apache/solr/search/SolrIndexSearcher.java
Outdated
Show resolved
Hide resolved
Co-authored-by: Chan Chan <chan.dx.dev@gmail.com>
Deprecate solr.core.indexsearcher.open.warmup_time
solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc
Outdated
Show resolved
Hide resolved
solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc
Outdated
Show resolved
Hide resolved
|
|
||
| === OpenTelemetry | ||
|
|
||
| The new OTEL based metric system introduced in 10.0 is considered BETA, and some breaking changes are to be expected in this and coming minor releases. |
There was a problem hiding this comment.
I restructured major changes under a common "OpenTelemetry" heading, and re-iterate that it is BETA, in case someone using 10.0 wants to upgrade and did not catch the retroactive BETA decision. So when they read the 10.1 notes they will be notified.
Implements SOLR-18165: align Solr's OTLP metric names with the OpenTelemetry semantic conventions, which require dot-separated names.
What changed
OTLP metric names now use dots instead of underscores:
solr_core_requestssolr.core.requestssolr_core_update_committed_opssolr.core.update.committed.opssolr_node_executorsolr.node.executorPrometheus metric names are unchanged. The OTel Prometheus exporter automatically converts dots to underscores, so the
/admin/metrics?wt=prometheusoutput still producessolr_core_requests_total,solr_core_update_committed_opsetc.Metric name filtering (
/admin/metrics?wt=prometheus&name=solr_core_requests) continues to work using Prometheus-format names. TheFilterablePrometheusMetricReadernow filters bygetMetadata().getPrometheusName()(underscore format) rather than against the OTel internal name — which also fixes a pre-existing issue where metrics with unit suffixes likejvm_system_memory_bytescould not be filtered by name.Files changed
"solr_core_*"and"solr_node_*"occurrences)AttributedInstrumentFactory.java—toNodeMetricName()updated to use"solr.core"→"solr.node"substitutionFilterablePrometheusMetricReader.java— name-filter logic updated as described abovechangelog/unreleased/SOLR-18165-dot-separated-metric-names.ymlBreaking change
This is a back-compat break for any OTLP consumer relying on
solr_core_*/solr_node_*metric names from v10.0. Affected users can add ametricstransformprocessor in their OpenTelemetry Collector to rename metrics during transition.Review guidance
All feedback welcome, but one area especially worth a second look:
Metric names that retain underscores. The convention used here is:
solr→core→update→committed.ops)The following metric names intentionally retain underscores:
solr.core.is_leaderis_leaderis a single predicate)solr.core.ref_countsolr.core.disk_spacesolr.core.update.auto_commitssolr.core.update.docs.pending_commitsolr.core.update_log.*(7 metrics)update_logis the tlog component namesolr.core.replication.is_leader,is_follower,is_enabled,is_replicating,is_polling_disabledsolr.core.crossdc.producer.submitted.delete_by_idsolr.core.crossdc.producer.submitted.delete_by_querysolr.core.crossdc.producer.document_sizesolr.core.crossdc.producer.doc_too_large_errorssolr.core.indexsearcher.index.num_docssolr.core.indexsearcher.index.commit_sizesolr.core.indexsearcher.open.warmup_timesolr.core.replication.download_speedsolr.core.replication.downloaded_sizesolr.core.replication.time_elapsedsolr.core.searcher.warming_maxChanging any of these now is cheap; once released each rename becomes another breaking change.
https://issues.apache.org/jira/browse/SOLR-18165