Add a progressr-based progress bar for sampling.#1138
Open
josswright wants to merge 11 commits intostan-dev:masterfrom
Open
Add a progressr-based progress bar for sampling.#1138josswright wants to merge 11 commits intostan-dev:masterfrom
progressr-based progress bar for sampling.#1138josswright wants to merge 11 commits intostan-dev:masterfrom
Conversation
This uses the [`progressr`](https://progressr.futureverse.org/index.html) framework to enable a progress bar for sampling operations. By default, this replaces standard iteration messages, but not other informative messages produced during sampling. From a user point of view, this adds two arguments to the `$sample()` method: - `show_progress_bar`: Default = FALSE If TRUE, registers a progress bar via `progressr` and signals an update for every output line that matches "Iteration:". The user is responsible for registering those progress updates with an appropriate handler. - `suppress_iteration_messages`: Defaults to the value of show_progress_bar, but can also be set directly. If TRUE, disables display of output lines matching "Iteration:", while still causing the progress bar to update. This keeps all of Stan's other informative output, but just removes the superfluous iteration messages when using a progress bar. I've tried to keep all additions to the code as non-intrusive as possible. One minor decision I made was to pass the value of `refresh` from the `$sample()` method through to the `CmdStanProcs` class so that the progress bar can report and update the number of steps in the bar to the same 'scale' as the number of iterations. (Calling `$sample()` with `iter_sampling=8000` and `refresh=100` will result in 80 'ticks' of the progress bar, each increasing the progress by 100.) By default, `progressr` doesn't register a handler to display the progress bar, as this is the responsibility of the user. Multiple packages can be used to display the progress bar. A default progress bar can be registered using the [`cli`](https://cli.r-lib.org/) library in this way: ```r library(progressr) library(cli) handlers( global=TRUE ) handlers("cli") options( cli.spinner = "moon", cli.progress_show_after = 0, cli.progress_clear = FALSE ) handlers( handler_cli( format = "{cli::pb_spin} Progress: |{cli::pb_bar}| {cli::pb_current}/{cli::pb_total} | {cli::pb_percent} | ETA: {cli::pb_eta}", clear = FALSE )) ``` A variety of alternative progress bar handlers are available, including audible and notification-based handlers, and ones that interact with RStudio's jobs pane: <https://progressr.futureverse.org/articles/progressr-11-handlers.html>
In addition to updating the progress bar on stdout lines that match "Iteration:", the CmdStanProcs object(s) now pass the current stdout line to the progress bar as a message for _every_ line, even if the number of completed iterations hasn't increased.
Added a `register_default_progress_handler()` function to `utils.R` that creates a default progress bar for sampling operations and registers it as global handler for progressr. Requires `cli` and `progressr`.
Was accidentally passing `refresh` to each informational progress bar update, rather than the calculated `progress_amount`. This caused each message line to update the progress bar, even if it didn't match an iteration message.
Moved code to close the progress bar to after the `check_finished()` function.
The nature of CmdStan's output makes pulling appropriate refresh and update values for the progress bar slightly tricky. In the original verison of the code, this led to the bar occasionally updating past the point where it was full, or terminating early due to an incorrectly calculated number of steps. This commit reworks the calculations of updates and refresh values for the progress bar, as well as adding some extra conditions to avoid the bar potentially crashing in odd scenarios.
New arguments for `iter_warmup` and `iter_sampling` were not being assigned a default value if not specified in sampling statement.
Open
Member
|
Looks like some tests are failing because the lower bound for iter_sampling should be 0 instead of 1. |
Author
Great. Thanks for this. (First pull request for me on a project like this, so I was a little unfamiliar with the process and running the unit tests.) I've fixed the initial errors, and a couple of other warnings. Now running the tests and documentation generation commands locally before I push an update. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Submission Checklist
Summary
As discussed in #18 , this pull request adds a progress bar to CmdStanR's
sample()function via theprogressrframework.This can be enabled via two new options to
model$sample(...):show_progress_bar- boolean - defaultFALSEEnables the progress bar.
suppress_iteration_messages- boolean - defaultFALSEifshow_progress_barisFALSE, andTRUEifshow_progress_barisTRUE.Causes any output lines reporting an iteration (such as
Chain 1 Iteration 1 / 2000 (Warmup)) to be suppressed.For the progress bar to be displayed, a progress bar handler must be created and registered by the end user. (See: https://progressr.futureverse.org/) For ease of use, a
register_default_progress_handler()function has been added. Callingregister_default_progress_handler()before amodel$sample(...)call will spawn a default progress bar via the cli library.Copyright and Licensing
Please list the copyright holder for the work you are submitting
(this will be you or your assignee, such as a university or company):
Joss Wright, University of Oxford
By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses: