Multi-gpu model parallel training in PyTorch

Hi, can someone recommend me a book or youtube tutorial channel or blogs to learn multi-gpu distributed model training or model parallel training using pytorch.


Have you looked at Distributed and Parallel Training Tutorials — PyTorch Tutorials 2.0.1+cu117 documentation ?
You’ll find pretty much anything over there…

thanks for this. I noticed this as well. I thought there would be some books to learn about it. Thanks though :slight_smile: