Problem with pytorch installation from source


(Mohammad Mehdi Derakhshani) #1

Hey guys,
when I was installing the pytorch based on its source, I’ve encountered with this error:

CMake Error at /home/mlcmdeep/anaconda3/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find CUDA (missing: CUDA_CUDART_LIBRARY) (found suitable version
"7.5", minimum required is “5.5”)
Call Stack (most recent call first):
/home/mlcmdeep/anaconda3/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
/home/mlcmdeep/pytorch/cmake/FindCUDA/FindCUDA.cmake:1013 (find_package_handle_standard_args)
CMakeLists.txt:28 (FIND_PACKAGE)

– Configuring incomplete, errors occurred!
See also “/home/mlcmdeep/pytorch/torch/lib/build/THC/CMakeFiles/CMakeOutput.log”

The installation commands which I’ve used were:

$ git clone https://github.com/pytorch/pytorch
$ export CMAKE_PREFIX_PATH=/home/yunjey/anaconda3 # your anaconda path
$ conda install numpy mkl setuptools cmake gcc cffi
$ conda install -c soumith magma-cuda80
$ export MACOSX_DEPLOYMENT_TARGET=10.9 # if OSX
$ pip install -r requirements.txt
$ python setup.py install (error has occurred when running this command)

So Could you make me aware about the source of the problem? Actually, how can I solve this problem?
Thanks!


(Yunjey) #2

If you are using cuda 7.5 not cuda 8.0, you must change the command conda install -c soumith magma-cuda80 to conda install -c soumith magma-cuda75.


(Mohammad Mehdi Derakhshani) #3

Hi @yunjey , Thanks for your response! But above error has occurred again. Any idea?
My installation commands were:

$ git clone https://github.com/pytorch/pytorch
$ export CMAKE_PREFIX_PATH=/home/yunjey/anaconda3 # your anaconda path
$ conda install numpy mkl setuptools cmake gcc cffi
$ conda install -c soumith magma-cuda75
$ pip install -r requirements.txt
$ python setup.py install


(Yunjey) #4

Did you change this command export CMAKE_PREFIX_PATH=/home/yunjey/anaconda3? Actually, you should not write /home/yunjey/anaconda3 but your anaconda installation path.


(Mohammad Mehdi Derakhshani) #5

@yunjey. yes, I’ve change it to export CMAKE_PREFIX_PATH=~/anaconda3/


(Yunjey) #6

Did you do source anaconda3/bin/activate ~/anaconda3 before run the commands?


(Mohammad Mehdi Derakhshani) #7

@yunjey yes, I’ve done it. :frowning: no effects. no solving.
@apaszke, Sorry man, Are there any opinions with you about my problem?


(Adam Paszke) #8

It’s not a problem with anaconda but with the CUDA driver. It seems that some of the libraries are missing. Can you reinstall the driver?


(Alexis David Jacq) #9

I also have a compilation error when trying to install from source:

Install the project...
-- Install configuration: "Release"
-- Installing: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTHNN.so.1
-- Installing: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTHNN.so
-- Up-to-date: /home/alexis/src/pytorch/torch/lib/tmp_install/include/THNN/THNN.h
-- Up-to-date: /home/alexis/src/pytorch/torch/lib/tmp_install/include/THNN/generic/THNN.h
-- Removing -DNDEBUG from compile flags
-- TH_LIBRARIES: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTH.so.1
-- Compiling with MAGMA support
-- MAGMA INCLUDE DIRECTORIES: /home/alexis/anaconda2/include
-- MAGMA LIBRARIES: /home/alexis/anaconda2/lib/libmagma.a
-- MAGMA V2 check: 1
-- Autodetected CUDA architecture(s): 5.0
-- got cuda version 8.0
-- Found CUDA with FP16 support, compiling with torch.CudaHalfTensor
-- CUDA_NVCC_FLAGS:  -DTH_INDEX_BASE=0 -I/home/alexis/src/pytorch/torch/lib/tmp_install/include   -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/TH -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/THC   -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/THPP;-gencode;arch=compute_50,code=sm_50;-DCUDA_HAS_FP16=1
-- THC_SO_VERSION: 1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/alexis/src/pytorch/torch/lib/build/THC
[  1%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorMath.cu.o
[  2%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o
[  3%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorRandom.cu.o
[  5%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorScatterGather.cu.o
In file included from /home/alexis/src/pytorch/torch/lib/THC/THCTensorMathMagma.cu:4:0:
/home/alexis/src/pytorch/torch/lib/THC/THCTensorMathMagma.cuh:5:19: fatal error: magma.h: No such file or directory
 #include <magma.h>
                   ^
compilation terminated.
CMake Error at THC_generated_THCTensorMathMagma.cu.o.cmake:207 (message):
  Error generating
  /home/alexis/src/pytorch/torch/lib/build/THC/CMakeFiles/THC.dir//./THC_generated_THCTensorMathMagma.cu.o


CMakeFiles/THC.dir/build.make:3869: recipe for target 'CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o' failed
make[2]: *** [CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....

before I run:

$ git clone https://github.com/pytorch/pytorch
$ export CMAKE_PREFIX_PATH=/home/alexis/anaconda2
$ conda install numpy mkl setuptools cmake gcc cffi
$ conda install -c soumith magma-cuda80
$ pip install -r requirements.txt
$ python setup.py install

It seems that magma.h is not included, while I have the cmake message -- MAGMA INCLUDE DIRECTORIES: /home/alexis/anaconda2/include that looks ok…

Edit: Problem solved after updating my version of cuda to the release one.


(Abhijat) #12

@mderakhshani Did you fix this by reinstalling CUDA?


(Mohammad Mehdi Derakhshani) #13

Hey, negative response! I have resigned to reinstall it! sorry man! :frowning:


(Marcin Elantkowski) #14

I’ve experienced the very same problem.

Grabbing the latest cuda-8.0 and reinstalling it solved the issue.


(Ywu36) #15

I faced the same problem when installing Pytorch in a new computer today. I solve it by edit the ‘.bashrc’, please make sure the environment variables of CUDA is set accurately as:

export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64/:$LD_LIBRARY_PATH"
export CUDNN_PATH="/home/yuhang/dependencies/cudnn-5/cuda/lib64/libcudnn.so.5"

The actual path will change based on your setup.
Hope it will helps.


#16

I came into the same problem. For some reason in my environmental variables there is a CPLUS_INCLUDE_PATH=/usr/local/cuda/include. I solved this with export CPLUS_INCLUDE_PATH=

See ref here: https://code.google.com/archive/p/thrust/issues/359


(Wei Fu) #17

export CUDNN_PATH="/home/yuhang/dependencies/cudnn-5/cuda/lib64/libcudnn.so.5"

this saves my day!!!
thanks
:smile:


#18

error running this command:

python setup.py install
Could not find /home/koeldc23/pytorch/torch/lib/gloo/CMakeLists.txt
Did you run ‘git submodule update --init’?


#19

Have you pulled Pytorch with the --recursive flag?
I think I had the same issue when I forgot it.


(Simon Wang) #20

So run git submodule update --init? The error is pretty clear itself.


(Simon Wang) #22

try installing bzip2-devel


#23

is there a way to install bzip2-devel without having root access? i’ve trying different things but no luck so far.