Place large subhalos on their own chunk

[robjmcgibbon]

The low-z snapshots of the largest COLIBRE runs are crashing due to memory. I think this is because single objects are so massive that they can't be processed in parallel.

This PR adds a threshold value, above which a subhalo is placed onto its own chunk.

I think placing each large halo on its own rank is too inefficient, I've gone for a "hybrid" approach instead. The user can now specify multiple threshold values. They can also specify the number of halos that can be processed in parallel for each threshold value.

Ideally we would do this automatically rather than having the user specify parameters. We would need to estimate the memory required by each halo (which would depend on the number of properties switched on), and then use that (combined with the system memory) to set the threshold values. I don't see myself getting round to doing that anytime soon though.

TODO

• Test
• Pick suitable threshold values for COLIBRE

[VictorForouhar]

Will have a go at L200m6 with morphological properties only. If that does not work, then I will run this branch.

[robjmcgibbon]

After some experimentation on the low-z snapshots of the L200m6 I have settled on some threshold values. They may be slightly conservative, but we're only taking ~20k cpu hours per snapshot, so I think it's ok