Install Pytorch with MAGMA without build from source?

XT_Li · October 9, 2017, 11:01pm

I’m trying to build pytorch from source. From the instructions, it seems we have to use anaconda. But anaconda gives me this error:

anaconda2/compiler_compat/ld: cannot find -lcuda
anaconda2/compiler_compat/ld: cannot find -lc

If I use sudo python setup.py install , there’s no such problem, but when importing it gives me this error:

ImportError: libmkl_gf_lp64.so: cannot open shared object file: No such file or directory

I assume there’s some issues relating sudo, is there way to install MAGMA without building pytorch from source? Thanks.

QuantScientist · October 10, 2017, 6:06am

Make sure you have CUDA available to sudo:

sudo ldconfig
to update all the linker paths. Then, you can do
sudo ldconfig -p | grep -i cuda
to see that your system knows where all the cuda libraries are.

and finaly try this:

github.com

QuantScientist/Deep-Learning-Boot-Camp/blob/master/day02-PyTORCH-and-PyCUDA/PyTorch/build_torch.sh

# PyTorch GPU and CPU
# If you dont have CUDA installed, run this first:
# https://github.com/QuantScientist/Deep-Learning-Boot-Camp/blob/master/docker/deps_nvidia_docker.sh

#GPU version
export PATH=/usr/local/cuda-8.0/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH
export PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
export CUDA_BIN_PATH=/usr/local/cuda
export CUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-8.0


# Build PyTorch from source
git clone https://github.com/pytorch/pytorch.git
cd pytorch
git submodule update --init
#git checkout 4eb448a051a1421de1dda9bd2ddfb34396eb7287 

export TORCH_CUDA_ARCH_LIST="3.5 5.2 6.0 6.1+PTX"

This file has been truncated. show original

XT_Li · October 10, 2017, 6:10am

Thanks for the reply. In my case, I actually just want to build pytorch with MAGMA. Does that mean I have to build pytorch using anaconda?

tom · October 10, 2017, 6:15am

It seems like you linked to mkl (Intel math kernel library). You could either see that pytorch finds it or not have the build link to it. Note that sudo sanitizes your environment variables unless you pass -E, so if you rely on these to find Magma, this cab be a problem.

I usually build with setup.py bdist_wheel (no sudo) and then uses pip to install the whl in dist. That has turned out to have less potential for surprises for me.

Best regards

Thomas