Working set fix

[mm-andritz]

Fixes #721

[Tinche]

LGTM but please add a docstring to the unit test explaining the case being tested (doesn't need to be long).

[codspeed-hq]

Merging this PR will not alter performance

✅ 54 untouched benchmarks

---

_{Comparing mm-andritz:working_set_fix (9e21a10) with main (fd887b7)}