AI Test and Perf Improvers

[dsyme]

Hi @muehlhaus, @kMutagene and @ZimmerD, @bvenn! ❤️

I've been working on experiments of using near-automatic, repo-centric AI test and perf improvers in some F# repositories. The aim is to use AI for continuous software improvement.

I've had good results on this repo fsprojects/FSharp.Control.AsyncSeq#179 for example, increasing coverage and merging some very positive perf improvements, including some long-standing technical debt.

I'm wondering if you'd have time and interest for applying this technique in this repo. I know you're in the middle of some lift-and-shift like #351, so some thought would be needed about when's the right time to try this.

Thanks
Don

[dsyme]

Practically speaking this would mean I would:

1. Install the Improver Actions to be active for at most 48h along with an API key
2. Assess the Test Coverage Improver Plan and Perf Improver Plan issues that get created
3. Add the extra configuration build steps inferred for each (these can be removed later)
4. Let each produce about 10 PRs
5. Assess quality of PRs. The Test Coverage Improver ones are usually easy to accept as long as they are CI-green. The Perf Improver ones are much more mixed and may often be thrown away.
6. Disable and delete the Improvers

Then you would:

1. Assess and merge the PRs at your own cadence

No PRs get merged without human approval

Let me know what you think :)

[dsyme]

Examples of iterative Test Coverage Improvement here: fsprojects/FSharp.Data#1574

Examples of iterative Perf Improvement here https://github.com/dsyme/librapid/pulls. This is harder and needs more input/configuration about what matters w.r.t. performance goals, and there are tradeoffs between clean code and perf. The root planning issue is here: dsyme/librapid#29

[bvenn]

Hi @dsyme,
I think this would be a great opportunity to increase coverage and add some performance improvements. Personally, I think #352 should be merged first because this PR will introduce deep structural changes. If that's okay with you, we'll get back to you here in the issue when we're ready.

[kMutagene]

Hey Don, sounds great and I agree that we'd want to merge #352 first. Should not be too long until that's done though.

[dsyme]

@kMutagene @bvenn Yup, I agree. Please reach out here or email me at GitHub (my alias @ github.com) when you're ready.