Gloo backend with CPU

distributed
1

I’m using gloo along with CPU. it gives me the following warning when I submit a job and it wouldn’t continue after the warning:
UserWarning: CUDA initialization: Found no NVIDIA driver on your system. Please check that you have an NVIDIA GPU and installed a driver from http://www.nvidia.com/Download/index.aspx (Triggered internally at /opt/conda/conda-bld/pytorch_1607370172916/work/c10/cuda/CUDAFunctions.cpp:100.)
return torch._C._cuda_getDeviceCount() > 0

However when I run in an “interact” mode without submitting jobs it runs fine. Any clue how to get the jobs running?

2

Hi,

Would it be possible to get a minimal script that reproduces the issue so we can investigate? Also, information about your environment (PyTorch version, etc) would be great as well. Also, this seems like it may be an unexpected bug, so the issues Issues · pytorch/pytorch · GitHub might be a better place for this.

3

Sure, in the meantime, is there a way to suppress the use of GPU for gloo? like there is --mca mpi_cuda_support 0 for mpi?