Case studies: EpiNow2 model performance is ~100x slower than equivalent EpiNow2

[seabbs]

The EpiAware implementation of the EpiNow2 model (case study 2) is currently running approximately 100 times slower than the equivalent model in the EpiNow2 R package.

Current status:

• EpiAware implementation uses ARIMA(2,1,1) with weekly broadcasting
• Uses NUTS sampling with pathfinder initialisation
• Includes delay convolutions and day-of-week effects

Potential performance bottlenecks:

• Delay convolution implementation efficiency
• Day-of-week effect structure
• ARIMA process generation on weekly timescale
• Pathfinder initialisation settings
• NUTS sampler configuration

Action items:

• Profile the model to identify computational bottlenecks
• Compare algorithmic choices with EpiNow2 implementation
• Investigate whether delay convolution can be optimised
• Review sampler settings (adapt iterations, target acceptance)
• Consider precompilation and type stability issues
• Test alternative inference approaches

[seabbs]

@SamuelBrand1 anything obvious you see here or shall we leave this as is and chalk it up to improvements in Turing closing this gap etc?

[SamuelBrand1]

@SamuelBrand1 anything obvious you see here or shall we leave this as is and chalk it up to improvements in Turing closing this gap etc?

Not obviously. It would have been good if we'd had established benchmarking earlier because my sense was as the structure of recursive @submodel macro calls got more dense then the performance declined. However, that is mainly vibes.

The only obvious thing to suggest that I don't think we can implement fast (i.e. don't reopen this issue) is that https://github.com/SpeedyWeather/SpeedyWeather.jl makes extensive use of function barriers with (I think) the goal of hitting compiler optimisations inside dispatched methods.

e.g. https://speedyweather.github.io/SpeedyWeatherDocumentation/dev/extensions/

[SamuelBrand1]

Also, actually, current EpiNow2 R pkg uses a lower dimensional hilbert representation of the latent Rt process which might be better sampling than weekly broadcast