How to install torchrun?

distributed
1

This doc encourages to use torchrun. But doesn’t tell how to install it :frowning:
How to install and get started with torchrun?

3 Likes
2

torchrun is part of PyTorch v1.10. If you are running an older version, python -m torch.distributed.run command serves the same purpose.

3 Likes
3

thanks @cbalioglu ! nice to meet you here, long time no see :wink:

4

@cbalioglu shall I run python -m torch.distributed.run once for the whole cluster? or once per node? or once per GPU? also asked here How to map processes to GPU in DDP and how to launch the DDP cluster?

5

It should be once per node IIUC