Skip to content

Indexer metrics followups#655

Merged
AndresJulia merged 13 commits intomainfrom
CCIP-9200_indexer_metric_gaps
Feb 16, 2026
Merged

Indexer metrics followups#655
AndresJulia merged 13 commits intomainfrom
CCIP-9200_indexer_metric_gaps

Conversation

@AndresJulia
Copy link
Copy Markdown
Contributor

@AndresJulia AndresJulia requested review from a team and skudasov as code owners February 9, 2026 15:57
@AndresJulia AndresJulia requested a review from bukata-sa February 9, 2026 15:57
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Feb 9, 2026

👋 AndresJulia, thanks for creating this pull request!

To help reviewers, please consider creating future PRs as drafts first. This allows you to self-review and make any final changes before notifying the team.

Once you're ready, you can mark it as "Ready for review" to request feedback. Thanks!

Comment thread indexer/pkg/monitoring/metrics.go
monitoring.Metrics().IncrementActiveRequestsCounter(c.Request.Context())

// Increment HTTP request counter
monitoring.Metrics().IncrementHTTPRequestCounter(c.Request.Context())
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why removing this?

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

because we have counter in RecordHTTPRequestDuration

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to record HttpRequestDuration on all endpoints? I think the duration should be more precisely set to particular endpoints since it has increased cardinality, while we keep httprequestcounter as a more generic one, for all endpoints

Copy link
Copy Markdown
Collaborator

@bukata-sa bukata-sa Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it, yeah makes sense
@AndresJulia can you do this

  1. keep counter for all endpoints in middleware + record http status
  2. move http request duration to handler and instead of URI record path from predefined set of values: /verifierresults , /verifierresults/:messageID , /messages

Comment thread indexer/pkg/discovery/message_discovery.go Outdated
Comment thread indexer/pkg/discovery/message_discovery.go Outdated
Comment thread indexer/pkg/discovery/message_discovery.go
Comment thread indexer/pkg/monitoring/metrics.go Outdated
sdkmetric.Instrument{Name: "indexer_storage_query_duration_seconds"},
sdkmetric.Stream{Aggregation: sdkmetric.AggregationExplicitBucketHistogram{
Boundaries: []float64{0.001, 0.005, 0.01, 0.025, 0.05, 0.1, 0.25, 0.5, 1, 2.5, 5, 10},
Boundaries: []float64{0.001, 0.005, 0.01, 0.05, 0.1, 0.5, 1, 5, 10},
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i know it's kinda default, but is there any value in distinction between 1ms, 5ms, or 10ms calls?

Comment thread indexer/pkg/storage/in_memory.go Outdated
}

i.monitoring.Metrics().RecordStorageQueryDuration(ctx, time.Since(startQueryMetric))
i.monitoring.Metrics().RecordStorageQueryDuration(ctx, time.Since(startQueryMetric), opName)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to have full visibility, we should probably also include the query status (error=(true|false))

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is another metric for errors

Copy link
Copy Markdown
Collaborator

@bukata-sa bukata-sa Feb 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Marcin here, for latency we also might need a status here and in the storage query duration metric
I'm wondering if we can drop error metric in this case

Comment thread indexer/pkg/storage/postgres.go Outdated
Comment thread indexer/pkg/storage/postgres.go Outdated
Comment thread indexer/pkg/discovery/message_discovery.go Outdated
@mateusz-sekara
Copy link
Copy Markdown
Contributor

mateusz-sekara commented Feb 11, 2026

@AndresJulia, please make sure you merge main to your PR. I've moved some logic from indexer middleware to the shared packages, and now TokenVerifier relies on the same shared code (unified way of tracking API latencies)

Context: #676

Comment thread indexer/pkg/storage/postgres.go Outdated
@AndresJulia AndresJulia force-pushed the CCIP-9200_indexer_metric_gaps branch from 04bbc54 to a7969e1 Compare February 11, 2026 15:54
Comment thread indexer/pkg/api/middleware/active_requests.go Outdated
Comment thread indexer/pkg/storage/in_memory.go Fixed
Comment thread indexer/pkg/storage/postgres.go Outdated
Comment thread indexer/pkg/storage/postgres.go Outdated
Comment thread indexer/pkg/storage/postgres.go
@AndresJulia AndresJulia requested a review from emate February 13, 2026 14:01
emate
emate previously approved these changes Feb 16, 2026
@github-actions
Copy link
Copy Markdown

Code coverage report:

Package main CCIP-9200_indexer_metric_gaps diff
github.com/smartcontractkit/chainlink-ccv/aggregator 47.90% 47.90% +0.00%
github.com/smartcontractkit/chainlink-ccv/cmd 0.00% 0.00% +0.00%
github.com/smartcontractkit/chainlink-ccv/committee 100.00% 100.00% +0.00%
github.com/smartcontractkit/chainlink-ccv/common 52.13% 52.13% +0.00%
github.com/smartcontractkit/chainlink-ccv/executor 40.60% 40.54% -0.06%
github.com/smartcontractkit/chainlink-ccv/indexer 31.63% 31.41% -0.22%

WARNING: go tool cover failed for coverage_target.out
cover: open /home/runner/work/chainlink-ccv/chainlink-ccv/indexer/pkg/discovery/multi_source.go: no such file or directory
WARNING: go tool cover failed for coverage.out
cover: open /home/runner/work/chainlink-ccv/chainlink-ccv/indexer/pkg/discovery/multi_source.go: no such file or directory

@emate emate self-requested a review February 16, 2026 14:12
@AndresJulia AndresJulia requested review from emate and removed request for emate February 16, 2026 14:15
@emate emate requested review from emate and removed request for emate February 16, 2026 14:32
@AndresJulia AndresJulia added this pull request to the merge queue Feb 16, 2026
Merged via the queue into main with commit 51b182e Feb 16, 2026
24 checks passed
@AndresJulia AndresJulia deleted the CCIP-9200_indexer_metric_gaps branch February 16, 2026 15:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants