Distributed Data Parallel over the Internet

No, you don’t need to ssh from node-1 to other nodes to launch the script. The ping/ssh I mentioned is only to check what IP would work. If you confirm that the IP of one node is accessible for all other nodes, you can set that node as master. This is only for rendezvous, and all nodes will use the rendezvous process to discover each other automatically.

1 Like