Skip to content

Conversation

@dt
Copy link
Contributor

@dt dt commented Dec 12, 2025

Backport 1/1 commits from #155848.

/cc @cockroachdb/release


Automatic stats jobs are frequent -- sometimes almost continious -- background process in many clusters, constantly being created and running as part of its normal operation. In some clusters these completed jobs have been observed to make up more than 90% of the retained content of the jobs system, but in most user-visible surfaces, they are usually filtered out or excluded as they have typically been found to be of low relevance or utility to users looking for more notable, discrete cluster events tied to less continious jobs.

This change introduces automatic, immediate cleanup of automatic stats jobs so that they are eagerly removed rather than being retained for the full job retention period. This should dramatically reduce the number of these in the jobs system to just those are executing and could be relevant to the cluster's operation, and those which were manually run (and thus likely of concern to a user), as well as those which failed which could also be of interest.

Release note (ops change): successfully completed automatic SQL stats collecton jobs now automatically purged rather than being retained for the full default job retention period.
Epic: CRDB-55121.

Release justification: default-off so no behavior change unless opted-in as directed to mitigate certain production jobs system challenges.

Automatic stats jobs are frequent -- sometimes almost continious -- background process in
many clusters, constantly being created and running as part of its normal operation. In some
clusters these completed jobs have been observed to make up more than 90% of the
retained content of the jobs system, but in most user-visible surfaces, they are
usually filtered out or excluded as they have typically been found to be of low
relevance or utility to users looking for more notable, discrete cluster events
tied to less continious jobs.

This change introduces automatic, immediate cleanup of automatic stats jobs so that
they are eagerly removed rather than being retained for the full job retention period.
This should dramatically reduce the number of these in the jobs system to just those
are executing and could be relevant to the cluster's operation, and those which were
manually run (and thus likely of concern to a user), as well as those which failed
which could also be of interest.

Release note (ops change): successfully completed automatic SQL stats collecton jobs
now automatically purged rather than being retained for the full default job retention
period.
Epic: CRDB-55121.
@dt dt requested review from mgartner and michae2 December 12, 2025 19:25
@dt dt requested a review from a team as a code owner December 12, 2025 19:25
@blathers-crl
Copy link

blathers-crl bot commented Dec 12, 2025

Thanks for opening a backport.

Before merging, please confirm that it falls into one of the following categories (select one):

  • Non-production code changes. Includes test-only changes, build system changes, etc.
  • Fixes for serious issues. Defined in the policy as correctness, stability, or security issues, data corruption/loss, significant performance regressions, breaking working and widely used functionality, or an inability to detect and debug production issues.
  • Other approved changes. These changes must be gated behind a disabled-by-default feature flag unless there is a strong justification not to.

Add a brief release justification to the PR description explaining your selection.

Also, confirm that the change does not break backward compatibility and complies with all aspects of the backport policy.

All backports must be reviewed by the TL and EM for the owning area.

@blathers-crl blathers-crl bot added backport Label PR's that are backports to older release branches T-disaster-recovery labels Dec 12, 2025
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@dt dt added T-sql-queries SQL Queries Team and removed T-disaster-recovery labels Dec 12, 2025
Copy link
Collaborator

@michae2 michae2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport Label PR's that are backports to older release branches T-sql-queries SQL Queries Team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants