Skip to content

add monitoring.ignore-duplicates to skip duplicate time series#511

Open
hcavarsan wants to merge 1 commit into
prometheus-community:masterfrom
hcavarsan:fix-collected-before
Open

add monitoring.ignore-duplicates to skip duplicate time series#511
hcavarsan wants to merge 1 commit into
prometheus-community:masterfrom
hcavarsan:fix-collected-before

Conversation

@hcavarsan

Copy link
Copy Markdown

Problem

the Cloud Monitoring API sometimes returns the same time series more than once in a single scrape, and prometheus refuses the duplicate while gathering:

collected metric "stackdriver_..." { ... } was collected before with the same name and label values 

promhttp turns that into a 500/503, so one duplicate fails the whole scrape.

Solution

this adds monitoring.ignore-duplicates (off by default). when it's on, the gatherer drops the duplicate-series error and returns what was collected.

only exact duplicates get dropped, since prometheus decides what counts as the same series by name, labels, and timestamp. anything differing in a label or timestamp is kept, and with the flag off nothing changes.

Testing

collectors/gatherer_test.go has an end-to-end test where a duplicate gets suppressed and the series collapses. it also has a table test for the error filtering.

related: #57, #103, #166, #450, #470.

Signed-off-by: hcavarsan <hcavarsan@yahoo.com.br>

feat: add monitoring.ignore-duplicates to skip duplicate time series

Signed-off-by: hcavarsan <hcavarsan@yahoo.com.br>

feat: add monitoring.ignore-duplicates to skip duplicate time series

Signed-off-by: hcavarsan <hcavarsan@yahoo.com.br>

feat: add monitoring.ignore-duplicates to skip duplicate time series

Signed-off-by: hcavarsan <hcavarsan@yahoo.com.br>

feat: add monitoring.ignore-duplicates to skip duplicate time series

Signed-off-by: hcavarsan <hcavarsan@yahoo.com.br>

feat: add monitoring.ignore-duplicates to skip duplicate time series

Signed-off-by: hcavarsan <hcavarsan@yahoo.com.br>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant