feat(node_monitor): monitor empty blocks and SR set

README.md

@@ -174,6 +174,7 @@ To start all available features, or you want more customized operations, navigat
    - **Gradle Docker**: Automate Docker image builds and testing. Check the [gradle docker](./tools/docker/README.md) documentation.
    - **Toolkit**: Perform a set of database related operations. Follow the [Toolkit guidance](./tools/toolkit/README.md).
    - **Stress Test**: Execute the stress test. Follow the [stress test guidance](./tools/stress_test/README.md).
+   - **Node Monitor**: Monitor Tron nodes for empty blocks and SR set changes. Follow the [node monitor guidance](./tools/node_monitor/README.md).
 
 ## Troubleshooting
 If you encounter any difficulties, please refer to the [Issue Work Flow](https://tronprotocol.github.io/documentation-en/developers/issue-workflow/#issue-work-flow), then raise an issue on [GitHub](https://github.com/tronprotocol/tron-docker/issues). For general questions, please use [Discord](https://discord.gg/cGKSsRVCGm) or [Telegram](https://t.me/TronOfficialDevelopersGroupEn).

tools/node_monitor/.gitignore

@@ -0,0 +1,32 @@
+# Binaries
+node_monitor
+*.exe
+*.exe~
+*.dll
+*.so
+*.dylib
+
+# Test binary, built with `go test -c`
+*.test
+
+# Output of the go coverage tool
+*.out
+
+# Dependency directories
+vendor/
+
+# Go workspace file
+go.work
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+*~
+
+# OS
+.DS_Store
+Thumbs.db
+
+node_monitor.yml

tools/node_monitor/README.md

@@ -0,0 +1,86 @@
+# Tron Node Monitor
+
+A Go tool for monitoring Tron nodes with the following capabilities:
+
+- **Empty Block Monitoring**: Monitor whether nodes produce empty blocks
+- **SR Set Change Monitoring**: Monitor changes in the Super Representative (SR) set
+- **Prometheus Metrics Export**: Export monitoring data to Prometheus with rich labels
+
+## Features
+
+- Periodically check the latest blocks from nodes to monitor empty block production
+- Monitor SR set changes (additions, removals, ordering changes, etc.)
+- Expose monitoring data through a single Prometheus metrics endpoint
+- Support graceful shutdown
+- Designed for monitoring **multiple nodes in one process** via the `node` label in metrics
+- **All node and runtime settings are managed via a YAML config file**
+
+## Build
+
+```bash
+cd tools/node_monitor
+go build -o node_monitor main.go
+```
+
+## Configuration (YAML)
+
+Example `node_monitor.yml`:
+
+```yaml
+metrics_addr: "0.0.0.0:9098"   # listen address for the metrics server
+interval: "10s"               # monitoring check interval
+nodes:
+  - label: "tron-node1"
+    url: "http://tron-node1:8090"
+  - label: "tron-node2"
+    url: "http://tron-node2:8090"
+  - label: "tron-node3"
+    url: "http://tron-node3:8090"
+```
+
+- `metrics_addr`: Address where the `/metrics` and `/health` endpoints are exposed.
+- `interval`: How often each node is checked (Go duration format, e.g. `5s`, `1m`).
+- `nodes`: List of Tron nodes to monitor.
+  - `label`: Logical node label used as the `node` label in Prometheus metrics.
+  - `url`: Tron node HTTP API URL (e.g. `http://host:8090`).
+
+## Run
+
+```bash
+./node_monitor -config node_monitor.yml
+```
+
+This will:
+- Start monitors for all nodes defined under `nodes:` in the config file
+- Use the `label` as the `node` label in Prometheus metrics
+- Expose all metrics at `http://<host>:9098/metrics` (or the `metrics_addr` you configure)
+
+## Command Line Arguments
+
+- `-config`: **Required**. Path to YAML config file. All nodes and runtime settings are read from this file.
+
+## Prometheus Metrics
+
+After the service starts, you can access the following endpoints:
+
+- `/metrics`: Prometheus metrics endpoint
+- `/health`: Health check endpoint
+
+Key labels exposed include:
+
+- `node`: logical node label (from config `label`)
+- `witness_address`, `block_number`, `block_hash` for empty-block metrics
+- `address`, `url`, `change_type` for SR-related metrics
+
+### Prometheus Configuration
+
+Example Prometheus scrape configuration for one monitor service:
+
+```yaml
+scrape_configs:
+  - job_name: 'tron-node-monitor'
+    static_configs:
+      - targets: ['monitor-node:9098']
+```
+
+Metrics for different Tron nodes are separated by the `node` label, so dashboards and alerts can filter or group by it.

tools/node_monitor/config.go

@@ -0,0 +1,67 @@
+package main
+
+import (
+	"fmt"
+	"os"
+	"time"
+
+	"gopkg.in/yaml.v3"
+)
+
+// fileConfig represents the YAML configuration file structure.
+//
+// Example:
+// metrics_addr: "0.0.0.0:9090"
+// interval: "10s"
+// nodes:
+//   - label: "tron-node1"
+//     url: "http://tron-node1:8090"
+//   - label: "tron-node2"
+//     url: "http://tron-node2:8090"
+type fileConfig struct {
+	MetricsAddr           string        `yaml:"metrics_addr"`
+	Interval              string        `yaml:"interval"`
+	SRBorderlineThreshold int64         `yaml:"sr_borderline_threshold"`
+	Nodes                 []fileCfgNode `yaml:"nodes"`
+}
+
+type fileCfgNode struct {
+	Label string `yaml:"label"`
+	URL   string `yaml:"url"`
+}
+
+// loadConfig reads and parses the YAML config file, returning node configurations,
+// optional metrics address override, optional interval override, and optional SR borderline threshold override.
+func loadConfig(path string) ([]nodeCfg, string, time.Duration, int64) {
+	data, err := os.ReadFile(path)
+	if err != nil {
+		panic(fmt.Errorf("failed to read config file %q: %w", path, err))
+	}
+
+	var cfg fileConfig
+	if err := yaml.Unmarshal(data, &cfg); err != nil {
+		panic(fmt.Errorf("failed to parse config file %q: %w", path, err))
+	}
+
+	var nodes []nodeCfg
+	for _, n := range cfg.Nodes {
+		if n.Label == "" || n.URL == "" {
+			continue
+		}
+		nodes = append(nodes, nodeCfg{
+			Label: n.Label,
+			URL:   n.URL,
+		})
+	}
+
+	var interval time.Duration
+	if cfg.Interval != "" {
+		d, err := time.ParseDuration(cfg.Interval)
+		if err != nil {
+			panic(fmt.Errorf("invalid interval %q in config file %q: %w", cfg.Interval, path, err))
+		}
+		interval = d
+	}
+
+	return nodes, cfg.MetricsAddr, interval, cfg.SRBorderlineThreshold
+}

tools/node_monitor/docker/Dockerfile

@@ -0,0 +1,32 @@
+# Multi-stage build for tron node monitor
+
+# Builder stage
+FROM golang:1.25.5-alpine AS builder
+
+WORKDIR /app
+
+# Enable Go modules and download dependencies first (better layer caching)
+COPY go.mod go.sum ./
+RUN go mod download
+
+# Copy the rest of the source code
+COPY . .
+
+# Build static binary
+RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o node_monitor .
+
+# Runtime stage
+FROM gcr.io/distroless/base-debian12
+
+WORKDIR /app
+
+# Copy binary from builder
+COPY --from=builder /app/node_monitor /app/node_monitor
+
+# Default config path (can be overridden by volume)
+COPY node_monitor.yml /app/node_monitor.yml
+
+EXPOSE 9098
+
+ENTRYPOINT ["/app/node_monitor"]
+CMD ["-config", "/app/node_monitor.yml"]

tools/node_monitor/docker/docker-compose.yml

@@ -0,0 +1,24 @@
+version: '3.8'
+
+services:
+  sr-node-monitor:
+    build:
+      context: ..
+      dockerfile: docker/Dockerfile
+    image: sr-node-monitor:latest
+    container_name: sr-node-monitor
+    restart: unless-stopped
+    # If you want this to share network with other stacks,
+    # you can change this to an external network.
+    networks:
+      - tron_monitor_network
+    ports:
+      - "9098:9098"  # expose Prometheus metrics endpoint
+    volumes:
+      # Mount config file so you can change nodes without rebuilding image
+      - ../node_monitor.yml:/app/node_monitor.yml:ro
+    # Logs will go to container stdout/stderr; manage via docker logs or logging driver
+
+networks:
+  tron_monitor_network:
+    driver: bridge

tools/node_monitor/go.mod

@@ -0,0 +1,21 @@
+module github.com/tronprotocol/tron-docker/tools/node_monitor
+
+go 1.25.5
+
+require (
+	github.com/prometheus/client_golang v1.23.2
+	gopkg.in/yaml.v3 v3.0.1
+)
+
+require (
+	github.com/beorn7/perks v1.0.1 // indirect
+	github.com/cespare/xxhash/v2 v2.3.0 // indirect
+	github.com/kr/text v0.2.0 // indirect
+	github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
+	github.com/prometheus/client_model v0.6.2 // indirect
+	github.com/prometheus/common v0.66.1 // indirect
+	github.com/prometheus/procfs v0.16.1 // indirect
+	go.yaml.in/yaml/v2 v2.4.2 // indirect
+	golang.org/x/sys v0.35.0 // indirect
+	google.golang.org/protobuf v1.36.8 // indirect
+)

tools/node_monitor/go.sum

@@ -0,0 +1,46 @@
+github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
+github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
+github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
+github.com/cespare/xxhash/v2 v2.3.0/go.mod h1:VGX0DQ3Q6kWi7AoAeZDth3/j3BFtOZR5XLFGgcrjCOs=
+github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
+github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
+github.com/google/go-cmp v0.7.0 h1:wk8382ETsv4JYUZwIsn6YpYiWiBsYLSJiTsyBybVuN8=
+github.com/google/go-cmp v0.7.0/go.mod h1:pXiqmnSA92OHEEa9HXL2W4E7lf9JzCmGVUdgjX3N/iU=
+github.com/klauspost/compress v1.18.0 h1:c/Cqfb0r+Yi+JtIEq73FWXVkRonBlf0CRNYc8Zttxdo=
+github.com/klauspost/compress v1.18.0/go.mod h1:2Pp+KzxcywXVXMr50+X0Q/Lsb43OQHYWRCY2AiWywWQ=
+github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
+github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
+github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
+github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
+github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
+github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
+github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
+github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
+github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/prometheus/client_golang v1.23.2 h1:Je96obch5RDVy3FDMndoUsjAhG5Edi49h0RJWRi/o0o=
+github.com/prometheus/client_golang v1.23.2/go.mod h1:Tb1a6LWHB3/SPIzCoaDXI4I8UHKeFTEQ1YCr+0Gyqmg=
+github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
+github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
+github.com/prometheus/common v0.66.1 h1:h5E0h5/Y8niHc5DlaLlWLArTQI7tMrsfQjHV+d9ZoGs=
+github.com/prometheus/common v0.66.1/go.mod h1:gcaUsgf3KfRSwHY4dIMXLPV0K/Wg1oZ8+SbZk/HH/dA=
+github.com/prometheus/procfs v0.16.1 h1:hZ15bTNuirocR6u0JZ6BAHHmwS1p8B4P6MRqxtzMyRg=
+github.com/prometheus/procfs v0.16.1/go.mod h1:teAbpZRB1iIAJYREa1LsoWUXykVXA1KlTmWl8x/U+Is=
+github.com/rogpeppe/go-internal v1.10.0 h1:TMyTOH3F/DB16zRVcYyreMH6GnZZrwQVAoYjRBZyWFQ=
+github.com/rogpeppe/go-internal v1.10.0/go.mod h1:UQnix2H7Ngw/k4C5ijL5+65zddjncjaFoBhdsK/akog=
+github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu7U=
+github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
+go.uber.org/goleak v1.3.0 h1:2K3zAYmnTNqV73imy9J1T3WC+gmCePx2hEGkimedGto=
+go.uber.org/goleak v1.3.0/go.mod h1:CoHD4mav9JJNrW/WLlf7HGZPjdw8EucARQHekz1X6bE=
+go.yaml.in/yaml/v2 v2.4.2 h1:DzmwEr2rDGHl7lsFgAHxmNz/1NlQ7xLIrlN2h5d1eGI=
+go.yaml.in/yaml/v2 v2.4.2/go.mod h1:081UH+NErpNdqlCXm3TtEran0rJZGxAYx9hb/ELlsPU=
+golang.org/x/sys v0.35.0 h1:vz1N37gP5bs89s7He8XuIYXpyY0+QlsKmzipCbUtyxI=
+golang.org/x/sys v0.35.0/go.mod h1:BJP2sWEmIv4KK5OTEluFJCKSidICx8ciO85XgH3Ak8k=
+google.golang.org/protobuf v1.36.8 h1:xHScyCOEuuwZEc6UtSOvPbAT4zRh0xcNRYekJwfqyMc=
+google.golang.org/protobuf v1.36.8/go.mod h1:fuxRtAxBytpl4zzqUh6/eyUujkJdNiuEkXntxiD/uRU=
+gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c h1:Hei/4ADfdWqJk1ZMxUNpqntNwaWcugrBjAiHlqqRiVk=
+gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c/go.mod h1:JHkPIbrfpd72SG/EVd6muEfDQjcINNoR0C8j2r3qZ4Q=
+gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
+gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=