Skip to content

add generate_quantities timing support#1168

Open
avehtari wants to merge 2 commits intomasterfrom
gq-timing
Open

add generate_quantities timing support#1168
avehtari wants to merge 2 commits intomasterfrom
gq-timing

Conversation

@avehtari
Copy link
Copy Markdown
Member

Closes #1070 after the next CmdStan release.

CmdStan develop version adds now timing information to standalone generated quantities csv. This PR adds CmdStanR support so that the timing reporting with $generate_quantities() matches $sample().

For CmdStan 2.38 and earlier the timing reported stays the same, that is, showing 0.0 seconds.

Example output with CmdStan 2.39+

Running standalone generated quantities after 4 MCMC chains, all chains in parallel ...

Chain 3 finished in 1.3 seconds.
Chain 4 finished in 1.4 seconds.
Chain 2 finished in 1.7 seconds.
Chain 1 finished in 1.9 seconds.

All 4 chains finished successfully.
Mean chain execution time: 1.6 seconds.
Total execution time: 2.0 seconds.

Copyright and Licensing

Please list the copyright holder for the work you are submitting
(this will be you or your assignee, such as a university or company):
INSERT COPYRIGHT HOLDER HERE

By submitting this pull request, the copyright holder is agreeing to
license the submitted work under the following licenses:

@avehtari avehtari requested a review from jgabry March 29, 2026 16:57
@jgabry
Copy link
Copy Markdown
Member

jgabry commented Mar 30, 2026

Thanks Aki. Looks like some tests are failing.

@avehtari
Copy link
Copy Markdown
Member Author

Some tests may fail if not using the develop version of CmdStan. I only thought that the actual code would work with old CmdStan, but did not think that it would be easier if the tests also work with old versions. Will fix tomorrow

@avehtari
Copy link
Copy Markdown
Member Author

Can we have conditional tests based on the CmdStan version? I tried to look at examples in tests, but didn't find a clear example

@jgabry
Copy link
Copy Markdown
Member

jgabry commented Mar 30, 2026

Can we have conditional tests based on the CmdStan version? I tried to look at examples in tests, but didn't find a clear example

We used to have a lot of tests like this for older versions of CmdStan but they were removed recently because for v1.0 we will require at least CmdStan v2.35.0. You can do things like this:

if (cmdstan_version() >= "2.39.0") {
  expect_true(...)
} 

But if the development version of CmdStan is not installed on any of the GitHub actions runs then the new functionality won't actually be tested. We could either setup one of the CI runs to use the development version so we can test it now or we could wait to merge this until the next CmdStan release.

@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Mar 30, 2026

Codecov Report

❌ Patch coverage is 92.85714% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 90.86%. Comparing base (5809552) to head (be63f37).
⚠️ Report is 2 commits behind head on master.

Files with missing lines Patch % Lines
R/run.R 75.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1168      +/-   ##
==========================================
+ Coverage   90.85%   90.86%   +0.01%     
==========================================
  Files          14       14              
  Lines        5924     5935      +11     
==========================================
+ Hits         5382     5393      +11     
  Misses        542      542              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

generate quantities method always reports completion of chains in 0s

3 participants