Have a look at this great post by @rwightman.
Based on his experiments he recommends buying NVME drives:
- Quick fix: buy an NVME SSD drive, or two.
- SATA SSD is not necessarily going to cut it, you can saturate them with small to medium image files + default loader/dataset setups feeding multiple GPUs.