Hello,
PyTorch has a built-in MPI backend, which is only available if PyTorch is compiled from source on a system which has an MPI implementation already installed link1, link2. Why can’t PyTorch ship with a default MPI implementation and enable the MPI backend without requiring compiling from source?
link1 states “Unfortunately, PyTorch’s binaries cannot include an MPI implementation”, but does not expand as to why.