Can I use Linux-computers connected to the internet to use DistributedDataParallel?

distributed
1

Good Day

As example:
I have two computers which both work on Linux and connect both with a ethernet cable to the router. I checked and proved with a simple python script with socket that the computers are reachable. (Code: Send/receive data with python socket - Stack Overflow )
Can I use this Linux-computers connected to the internet to use DistributedDataParallel (DistributedDataParallel — PyTorch master documentation )?
I searched the web for an answer but there where no clear answer to this question and so I am not sure if I have to create a special network architecture and install some programms first to use DistributedDataParallel.

I am grateful for every answer and suggestion where I can find this answer. Thank you very much for your time.

2

I would assume you could use DDP as long as both nodes can reach each other.
However, if the traffic is indeed using an internet connection (and not a local network) I would assume to see a massive slowdown in your training as this would most likely be the bottleneck of your application.

3

Thank you very much for your answer.
So it would become faster if I use a ethernet switch?
I tried this out with two windows computer. They connected but as soon as they did I got an “RuntimeError: Stop_waiting response is expected” generated from “init_process_group”. Do you know why thats happening?