Hi, I am wondering if the init_method to initialise the process group indeed causes any diff in performance? The first init_method is to use tcp://<host>:<port>
, the second is to use the default env.
No, there aren’t performance differences between the two init methods. Default env initialization creates a TCPStore instance (same as tcp://
init method) but uses environment variables MASTER_ADDR and MASTER_PORT as the <host>:<port>
see pytorch/rendezvous.py at master · pytorch/pytorch · GitHub
1 Like