Skip to content

Post Processing: Refactor of result modules#347

Merged
harriscr merged 3 commits intoceph:mainfrom
harriscr:wip_ch_fio_cpu_report
Jan 15, 2026
Merged

Post Processing: Refactor of result modules#347
harriscr merged 3 commits intoceph:mainfrom
harriscr:wip_ch_fio_cpu_report

Conversation

@harriscr
Copy link
Copy Markdown
Contributor

@harriscr harriscr commented Dec 2, 2025

In order to be able to include resource usage statistics in the reports produced as part of the post processing some re-factoring of the post_processing/run_result modules was necessary.

This is mainly the creating of a run_result directory and creation of some base classes that can be extended for particular run types.

This is the first step in adding resource monitor result file parsing and adding to the common format output file (which we use to generate the reports). Future PRs will deal with:

  • parsing the various resource monitoring options that CBT supports, top, collectl, perf etc. into the common format.
  • creating resource monitoring plots for the report e.g. CPU and Memory utilisation over time
  • Additional report options to control which resource statistics are addre to a report

Testing

Teuthology perf-basic: https://pulpito.ceph.com/harriscr-2025-12-01_13:29:31-perf-basic-main-distro-default-smithi/
rados/perf: https://pulpito.ceph.com/harriscr-2025-12-02_08:34:46-rados:perf-main-distro-default-smithi/

All tests passed

Re-factor the run_result modules into a format that
will support the adding of resource usage statirstics
and plots to a report in the future

Signed-off-by: Chris Harris <harriscr@uk.ibm.com>
@harriscr harriscr self-assigned this Dec 2, 2025
@harriscr harriscr force-pushed the wip_ch_fio_cpu_report branch from a3773b1 to 797d387 Compare December 18, 2025 10:25
Re-factor the run_result modules into a format that
will support the adding of resource usage statirstics
and plots to a report in the future

Signed-off-by: Chris Harris <harriscr@uk.ibm.com>
@harriscr harriscr force-pushed the wip_ch_fio_cpu_report branch from 797d387 to c421fb3 Compare December 18, 2025 10:48
Add an extra line to the graphs in a simple report that shows the CPU usage as reported by FIO. The default is not to plot the CPU usage, but this can be changed by an option to the script or in the CBT yaml file in the report section

Signed-off-by: Chris Harris <harriscr@uk.ibm.com>
Copy link
Copy Markdown
Member

@lee-j-sanders lee-j-sanders left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've been using the comparison tool for the past couple of days without any issues. The change itself is quite large to review, but given I've been using this for a few days without problems, it should be good to merge.

@harriscr harriscr merged commit 5704b3c into ceph:main Jan 15, 2026
@harriscr harriscr deleted the wip_ch_fio_cpu_report branch April 28, 2026 13:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants