GP Sampling

[Crown421]

Summary
This PR is a (at this stage) draft PR to allow sampling of GP, including sampling via Cholesky, via Random Fourier Features, and pathwise/ decoupled sampling.

Proposed changes
The main interface revolves around a new GPSampler struct, as a subtype of Random.Sampler, which takes a GP and a "method". One can then draw from this sampler using rand`.

However, there are still issues to this implementation

• GP must have a kernel (when using RFF)
• Not yet working with sparse approximations
• Pathwise functionality was written to (theoretically) enable using Cholesky samples for the prior, and RFF for the posterior, but in practice one would never want to do that.
  • It might not be worth to keep this use case around
• Multi-Output probably not working yet (and not tested)

What alternatives have you considered?
Making a separate package, and/or (due to the complexity) just keeping it good enough for my needs.

Breaking changes
Should not be breaking (at this point).

[codecov]

Codecov Report

❌ Patch coverage is 96.84211% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 96.33%. Comparing base (05deb6b) to head (c728eb6).
⚠️ Report is 15 commits behind head on main.

Files with missing lines	Patch %	Lines
src/sampling.jl	96.84%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #421      +/-   ##
==========================================
+ Coverage   96.21%   96.33%   +0.12%     
==========================================
  Files          12       13       +1     
  Lines         396      491      +95     
==========================================
+ Hits          381      473      +92     
- Misses         15       18       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[willtebbutt]

Very happy to give this a review -- when you're ready for one, please do tag me.

[Crown421]

@willtebbutt If you have some time, I would already appreciate a rough review, on the overall structure/ idea.
I worry that I got too "clever" with the type structure/ relationships/ interface.

[yebai]

Can you move this into a package extension?

[Crown421]

Apologies, it has been a minute.

The short answer is that I am not sure. On the one hand, it would make sense to keep everything clean, but

• KernelSpectralDensities is a tiny package
• The main reason it exists is to enable decoupled sampling with Fourier basis functions, which (AFAIK) still the state of the art for sampling GPs
• (the main uncertainty for me) I am not sure where to put some of the objects, since they only make sense when using the combination of both packages.