Verify and document native histograms with custom buckets (NHCB) support (#1846)

Fixes #1838

## Summary

Addresses #1838 by verifying and documenting that `client_java` fully
supports native histograms with custom buckets (NHCB, schema -53).

  ## Background

According to the [Prometheus Native Histograms
specification](https://prometheus.io/docs/specs/native_histograms/),
NHCB is handled by:
  1. Clients expose classic histograms with custom bucket boundaries
2. Prometheus servers convert them to native histograms (schema -53)
when configured with `convert_classic_histograms_to_nhcb: true`
  3. Custom bucket boundaries are preserved in the conversion

  ## Changes

  ### 1. Comprehensive Test Suite ✅
- **File:**
`prometheus-metrics-core/src/test/java/io/prometheus/metrics/core/metrics/CustomBucketsHistogramTest.java`
  - **Tests:** 11 comprehensive tests (all passing)
  - **Coverage:**
    - Arbitrary custom bucket boundaries
    - Linear boundaries (equal-width buckets)
- Exponential boundaries (for data spanning multiple orders of
magnitude)
    - Classic-only and dual-mode histograms
    - Text and protobuf format serialization
    - Labeled histograms with custom buckets
    - Boundary edge cases and fine-grained precision

  ### 2. Documentation ✅
  - **File:** `docs/content/getting-started/metric-types.md`
  - **Added:**
- Custom Bucket Boundaries section with examples for arbitrary, linear,
and exponential boundaries
    - Native Histograms with Custom Buckets (NHCB) section explaining:
      - How NHCB works
      - Prometheus server configuration
      - When to use NHCB vs standard native histograms
      - Comparison table

  ### 3. Working Example ✅
  - **New module:** `examples/example-custom-buckets/`
  - **Demonstrates:** Three real-world use cases:
- API latency with arbitrary custom boundaries (0.01, 0.05, 0.1, 0.5,
1.0, 5.0, 10.0 seconds)
    - Queue size with linear boundaries (10, 20, 30, ..., 100)
- Response size with exponential boundaries (100, 1k, 10k, 100k, 1M, 10M
bytes)
  - **Includes:**
    - Docker Compose setup with Prometheus (NHCB enabled) and Grafana
    - Pre-configured Grafana dashboard showing all three histogram types
- Comprehensive README with build/run instructions and verification
steps

  ### 4. Verification Report ✅
  - **File:** `CUSTOM_BUCKETS_VERIFICATION.md`
- **Documents:** Detailed findings, test methodology, and verification
results

  ## Test Results

  [INFO] Tests run: 11, Failures: 0, Errors: 0, Skipped: 0
  [INFO] BUILD SUCCESS

All tests pass successfully, confirming that custom bucket support works
correctly in all scenarios.

  ## Key Findings

  ✅ **Custom bucket support is fully functional**
- The Java client correctly handles histograms with custom bucket
boundaries
- Dual-mode operation (classic + native) works seamlessly with custom
buckets
- Custom buckets are properly serialized in both text and protobuf
formats
- Prometheus servers can convert these to NHCB (schema -53) upon
ingestion

  ## How to Test

  ### Run the tests:
  ```bash
./mvnw test -Dtest=CustomBucketsHistogramTest -pl
prometheus-metrics-core
```

  Run the example:

  ```bash
  ./mvnw package
  cd examples/example-custom-buckets
  docker-compose up
```
  Then visit:
  - Application: http://localhost:9400/metrics
  - Prometheus: http://localhost:9090
  - Grafana: http://localhost:3000 (admin/admin)

  Documentation Links

  - https://prometheus.io/docs/specs/native_histograms/
  - CUSTOM_BUCKETS_VERIFICATION.md
  - examples/example-custom-buckets/README.md

---------

Signed-off-by: Gregor Zeitlinger <gregor.zeitlinger@grafana.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Author: zeitlinger

CLAUDE.md

@@ -69,19 +69,46 @@ Pre-built instrumentations: `prometheus-metrics-instrumentation-jvm`, `-caffeine
 ## Code Style
 
 - **Formatter**: Google Java Format (enforced via Spotless)
-- **Line length**: 100 characters
+- **Line length**: 100 characters (enforced for ALL files including Markdown, Java, YAML, etc.)
 - **Indentation**: 2 spaces
 - **Static analysis**: Error Prone with NullAway (`io.prometheus.metrics` package)
 - **Logger naming**: Logger fields must be named `logger` (not `log`, `LOG`, or `LOGGER`)
 - **Assertions in tests**: Use static imports from AssertJ (`import static org.assertj.core.api.Assertions.assertThat`)
 - **Empty catch blocks**: Use `ignored` as the exception variable name
+- **Markdown code blocks**: Always specify language (e.g., ` ```java`, ` ```bash`, ` ```text`)
 
 ## Linting and Validation
 
-- **IMPORTANT**: Always run `mise run build` after modifying Java files to ensure all lints, code formatting (Spotless), static analysis (Error Prone), and checkstyle checks pass
-- **IMPORTANT**: Always run `mise run lint:super-linter` after modifying non-Java files (YAML, Markdown, shell scripts, JSON, etc.)
-- Super-linter is configured to only show ERROR-level messages via `LOG_LEVEL=ERROR` in `.github/super-linter.env`
-- Local super-linter version is pinned to match CI (see `.mise/tasks/lint/super-linter.sh`)
+**CRITICAL**: These checks MUST be run before creating any commits. CI will fail if these checks fail.
+
+### Java Files
+
+- **ALWAYS** run `mise run build` after modifying Java files to ensure:
+  - Code formatting (Spotless with Google Java Format)
+  - Static analysis (Error Prone with NullAway)
+  - Checkstyle validation
+  - Build succeeds (tests are skipped; run `mise run test` or `mise run test-all` to execute tests)
+
+### Non-Java Files (Markdown, YAML, JSON, shell scripts, etc.)
+
+- **ALWAYS** run `mise run lint:super-linter` after modifying non-Java files
+- Super-linter will **auto-fix** many issues (formatting, trailing whitespace, etc.)
+- It only reports ERROR-level issues (configured via `LOG_LEVEL=ERROR` in `.github/super-linter.env`)
+- Common issues caught:
+  - Lines exceeding 100 characters in Markdown files
+  - Missing language tags in fenced code blocks
+  - Table formatting issues
+  - YAML/JSON syntax errors
+
+### Running Linters
+
+```bash
+# After modifying Java files (run BEFORE committing)
+mise run build
+
+# After modifying non-Java files (run BEFORE committing)
+mise run lint:super-linter
+```
 
 ## Testing
 

docs/content/getting-started/metric-types.md

@@ -121,6 +121,94 @@ for [Histogram.Builder](/client_java/api/io/prometheus/metrics/core/metrics/Hist
 for a complete list of options. Some options can be configured at runtime,
 see [config]({{< relref "../config/config.md" >}}).
 
+### Custom Bucket Boundaries
+
+The default bucket boundaries are designed for measuring request durations in seconds. For other
+use cases, you may want to define custom bucket boundaries. The histogram builder provides three
+methods for this:
+
+**1. Arbitrary Custom Boundaries**
+
+Use `classicUpperBounds(...)` to specify arbitrary bucket boundaries:
+
+```java
+Histogram responseSize = Histogram.builder()
+    .name("http_response_size_bytes")
+    .help("HTTP response size in bytes")
+    .classicUpperBounds(100, 1000, 10000, 100000, 1000000) // bytes
+    .register();
+```
+
+**2. Linear Boundaries**
+
+Use `classicLinearUpperBounds(start, width, count)` for equal-width buckets:
+
+```java
+Histogram queueSize = Histogram.builder()
+    .name("queue_size")
+    .help("Number of items in queue")
+    .classicLinearUpperBounds(10, 10, 10) // 10, 20, 30, ..., 100
+    .register();
+```
+
+**3. Exponential Boundaries**
+
+Use `classicExponentialUpperBounds(start, factor, count)` for exponential growth:
+
+```java
+Histogram dataSize = Histogram.builder()
+    .name("data_size_bytes")
+    .help("Data size in bytes")
+    .classicExponentialUpperBounds(100, 10, 5) // 100, 1k, 10k, 100k, 1M
+    .register();
+```
+
+### Native Histograms with Custom Buckets (NHCB)
+
+Prometheus supports a special mode called Native Histograms with Custom Buckets (NHCB) that uses
+schema -53. In this mode, custom bucket boundaries from classic histograms are preserved when
+converting to native histograms.
+
+The Java client library automatically supports NHCB:
+
+1. By default, histograms maintain both classic (with custom buckets) and native representations
+2. The classic representation with custom buckets is exposed to Prometheus
+3. Prometheus servers can convert these to NHCB upon ingestion when configured with the
+   `convert_classic_histograms_to_nhcb` scrape option
+
+Example:
+
+```java
+// This histogram will work seamlessly with NHCB
+Histogram apiLatency = Histogram.builder()
+    .name("api_request_duration_seconds")
+    .help("API request duration")
+    .classicUpperBounds(0.01, 0.05, 0.1, 0.5, 1.0, 5.0, 10.0) // custom boundaries
+    .register();
+```
+
+On the Prometheus side, configure the scrape job:
+
+```yaml
+scrape_configs:
+  - job_name: "my-app"
+    scrape_protocols: ["PrometheusProto"]
+    convert_classic_histograms_to_nhcb: true
+    static_configs:
+      - targets: ["localhost:9400"]
+```
+
+{{< hint type=note >}}
+NHCB is useful when:
+
+- You need precise bucket boundaries for your specific use case
+- You're migrating from classic histograms and want to preserve bucket boundaries
+- Exponential bucketing from standard native histograms isn't a good fit for your distribution
+  {{< /hint >}}
+
+See [examples/example-custom-buckets](https://github.com/prometheus/client_java/tree/main/examples/example-custom-buckets) <!-- editorconfig-checker-disable-line -->
+for a complete example with Prometheus and Grafana.
+
 Histograms and summaries are both used for observing distributions. Therefore, the both implement
 the `DistributionDataPoint` interface. Using the `DistributionDataPoint` interface directly gives
 you the option to switch between histograms and summaries later with minimal code changes.

examples/example-custom-buckets/README.md

@@ -0,0 +1,170 @@
+# Native Histograms with Custom Buckets (NHCB) Example
+
+This example demonstrates how to use native histograms with custom bucket boundaries (NHCB) in
+Prometheus Java client. It shows three different types of custom bucket configurations and how
+Prometheus converts them to native histograms with schema -53.
+
+## What are Native Histograms with Custom Buckets?
+
+Native Histograms with Custom Buckets (NHCB) is a Prometheus feature that combines the benefits of:
+
+- **Custom bucket boundaries**: Precisely defined buckets optimized for your specific use case
+- **Native histograms**: Efficient storage and querying capabilities of native histograms
+
+When you configure Prometheus with `convert_classic_histograms_to_nhcb: true`, it converts classic
+histograms with custom buckets into native histograms using schema -53, preserving the custom
+bucket boundaries.
+
+## Example Metrics
+
+This example application generates three different histogram metrics demonstrating different
+bucket configuration strategies:
+
+### 1. API Latency - Arbitrary Custom Boundaries
+
+```java
+Histogram apiLatency = Histogram.builder()
+    .name("api_request_duration_seconds")
+    .classicUpperBounds(0.01, 0.05, 0.1, 0.5, 1.0, 5.0, 10.0)
+    .register();
+```
+
+**Use case**: Optimized for typical API response times in seconds.
+
+### 2. Queue Size - Linear Boundaries
+
+```java
+Histogram queueSize = Histogram.builder()
+    .name("message_queue_size")
+    .classicLinearUpperBounds(10, 10, 10) // 10, 20, 30, ..., 100
+    .register();
+```
+
+**Use case**: Equal-width buckets for monitoring queue depth or other discrete values.
+
+### 3. Response Size - Exponential Boundaries
+
+```java
+Histogram responseSize = Histogram.builder()
+    .name("http_response_size_bytes")
+    .classicExponentialUpperBounds(100, 10, 6) // 100, 1k, 10k, 100k, 1M, 10M
+    .register();
+```
+
+**Use case**: Data spanning multiple orders of magnitude (bytes, milliseconds, etc).
+
+## Build
+
+This example is built as part of the `client_java` project:
+
+```shell
+./mvnw package
+```
+
+This creates `./examples/example-custom-buckets/target/example-custom-buckets.jar`.
+
+## Run
+
+With the JAR file present, run:
+
+```shell
+cd ./examples/example-custom-buckets/
+docker-compose up
+```
+
+This starts three Docker containers:
+
+- **[http://localhost:9400/metrics](http://localhost:9400/metrics)** - Example application
+- **[http://localhost:9090](http://localhost:9090)** - Prometheus server (with NHCB enabled)
+- **[http://localhost:3000](http://localhost:3000)** - Grafana (user: _admin_, password: _admin_)
+
+You might need to replace `localhost` with `host.docker.internal` on macOS or Windows.
+
+## Verify NHCB Conversion
+
+### 1. Check Prometheus Configuration
+
+The Prometheus configuration enables NHCB conversion:
+
+```yaml
+scrape_configs:
+  - job_name: "custom-buckets-demo"
+    scrape_protocols: ["PrometheusProto"]
+    convert_classic_histograms_to_nhcb: true
+    scrape_classic_histograms: true
+```
+
+### 2. Verify in Prometheus
+
+Visit [http://localhost:9090](http://localhost:9090) and run queries:
+
+```promql
+# View histogram metadata (should show schema -53 for NHCB)
+prometheus_tsdb_head_series
+
+# Calculate quantiles from custom buckets
+histogram_quantile(0.95, rate(api_request_duration_seconds[1m]))
+
+# View raw histogram structure
+api_request_duration_seconds
+```
+
+### 3. View in Grafana
+
+The Grafana dashboard at [http://localhost:3000](http://localhost:3000) shows:
+
+- p95 and p50 latencies for API endpoints (arbitrary custom buckets)
+- Queue size distribution (linear buckets)
+- Response size distribution (exponential buckets)
+
+## Key Observations
+
+1. **Custom Buckets Preserved**: The custom bucket boundaries you define are preserved when
+   converted to NHCB (schema -53).
+
+2. **Dual Representation**: By default, histograms maintain both classic and native
+   representations, allowing gradual migration.
+
+3. **Efficient Storage**: Native histograms provide more efficient storage than classic histograms
+   while preserving your custom bucket boundaries.
+
+4. **Flexible Bucket Strategies**: You can choose arbitrary, linear, or exponential buckets based
+   on your specific monitoring needs.
+
+## When to Use Custom Buckets
+
+Consider using custom buckets (and NHCB) when:
+
+- **Precise boundaries needed**: You know the expected distribution and want specific bucket edges
+- **Migrating from classic histograms**: You want to preserve existing bucket boundaries
+- **Specific use cases**: Default exponential bucketing doesn't fit your distribution well
+  - Temperature ranges (might include negative values)
+  - Queue depths (discrete values with linear growth)
+  - File sizes (exponential growth but with specific thresholds)
+  - API latencies (specific SLA boundaries)
+
+## Differences from Standard Native Histograms
+
+| Feature           | Standard Native Histograms      | NHCB (Schema -53)                 |
+| ----------------- | ------------------------------- | --------------------------------- |
+| Bucket boundaries | Exponential (base 2^(2^-scale)) | Custom boundaries                 |
+| Use case          | General-purpose                 | Specific distributions            |
+| Mergeability      | Can merge with same schema      | Cannot merge different boundaries |
+| Configuration     | Schema level (0-8)              | Explicit boundary list            |
+
+## Cleanup
+
+Stop the containers:
+
+```shell
+docker-compose down
+```
+
+## Further Reading
+
+<!-- editorconfig-checker-disable -->
+<!-- markdownlint-disable MD013 -->
+
+- [Prometheus Native Histograms Specification](https://prometheus.io/docs/specs/native_histograms/)
+- [Prometheus Java Client Documentation](https://prometheus.github.io/client_java/)
+- [OpenTelemetry Exponential Histograms](https://opentelemetry.io/docs/specs/otel/metrics/data-model/#exponentialhistogram)

examples/example-custom-buckets/docker-compose.yaml

@@ -0,0 +1,26 @@
+version: "3"
+services:
+  example-application:
+    image: eclipse-temurin:25.0.1_8-jre@sha256:9d1d3068b16f2c4127be238ca06439012ff14a8fdf38f8f62472160f9058464a
+    network_mode: host
+    volumes:
+      - ./target/example-custom-buckets.jar:/example-custom-buckets.jar
+    command:
+      - /opt/java/openjdk/bin/java
+      - -jar
+      - /example-custom-buckets.jar
+  prometheus:
+    image: prom/prometheus:v3.9.1@sha256:1f0f50f06acaceb0f5670d2c8a658a599affe7b0d8e78b898c1035653849a702
+    network_mode: host
+    volumes:
+      - ./docker-compose/prometheus.yml:/prometheus.yml
+    command:
+      - --enable-feature=native-histograms
+      - --config.file=/prometheus.yml
+  grafana:
+    image: grafana/grafana:12.3.2@sha256:ba93c9d192e58b23e064c7f501d453426ccf4a85065bf25b705ab1e98602bfb1
+    network_mode: host
+    volumes:
+      - ./docker-compose/grafana-datasources.yaml:/etc/grafana/provisioning/datasources/grafana-datasources.yaml
+      - ./docker-compose/grafana-dashboards.yaml:/etc/grafana/provisioning/dashboards/grafana-dashboards.yaml
+      - ./docker-compose/grafana-dashboard-custom-buckets.json:/etc/grafana/grafana-dashboard-custom-buckets.json