Multi-gpu model parallel training in PyTorch

distributed
1

Hi, can someone recommend me a book or youtube tutorial channel or blogs to learn multi-gpu distributed model training or model parallel training using pytorch.
@ptrblck

2

Hi,

Have you looked at Distributed and Parallel Training Tutorials — PyTorch Tutorials 2.0.1+cu117 documentation ?
You’ll find pretty much anything over there…

3

thanks for this. I noticed this as well. I thought there would be some books to learn about it. Thanks though :slight_smile: