May I ask how long it will take to train 8 units of 4090?

[zfinn1]

hello. May I ask how long it will take to train 8 units of 4090?

[suniique]

Hi, thanks for the question!

The training process depends on the dataset. In our setup, when training on all datasets jointly from scratch with a total batch size of 320 on 8×A100 GPUs, it takes about 1300 epochs to reach reasonable performance and 2000 epochs to fully converge. Meanwhile, finetuning from a trained checkpoint on smaller datasets is significantly faster; we've tested this on an internal dataset of ~1k meshes using a single 4090, completing within around a day.

Could you share more context—such as which dataset you're using, how large is it, and whether you're finetuning or training from scratch?