Problem with pytorch installation from source

mderakhshani · March 4, 2017, 10:36am

Hey guys,
when I was installing the pytorch based on its source, I’ve encountered with this error:

CMake Error at /home/mlcmdeep/anaconda3/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:148 (message):
Could NOT find CUDA (missing: CUDA_CUDART_LIBRARY) (found suitable version
“7.5”, minimum required is “5.5”)
Call Stack (most recent call first):
/home/mlcmdeep/anaconda3/share/cmake-3.6/Modules/FindPackageHandleStandardArgs.cmake:388 (_FPHSA_FAILURE_MESSAGE)
/home/mlcmdeep/pytorch/cmake/FindCUDA/FindCUDA.cmake:1013 (find_package_handle_standard_args)
CMakeLists.txt:28 (FIND_PACKAGE)

– Configuring incomplete, errors occurred!
See also “/home/mlcmdeep/pytorch/torch/lib/build/THC/CMakeFiles/CMakeOutput.log”

The installation commands which I’ve used were:

$ git clone GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration
$ export CMAKE_PREFIX_PATH=/home/yunjey/anaconda3 # your anaconda path
$ conda install numpy mkl setuptools cmake gcc cffi
$ conda install -c soumith magma-cuda80
$ export MACOSX_DEPLOYMENT_TARGET=10.9 # if OSX
$ pip install -r requirements.txt
$ python setup.py install (error has occurred when running this command)

So Could you make me aware about the source of the problem? Actually, how can I solve this problem?
Thanks!

yunjey · March 4, 2017, 10:40am

If you are using cuda 7.5 not cuda 8.0, you must change the command conda install -c soumith magma-cuda80 to conda install -c soumith magma-cuda75.

mderakhshani · March 4, 2017, 11:28am

Hi @yunjey , Thanks for your response! But above error has occurred again. Any idea?
My installation commands were:

$ git clone GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration
$ export CMAKE_PREFIX_PATH=/home/yunjey/anaconda3 # your anaconda path
$ conda install numpy mkl setuptools cmake gcc cffi
$ conda install -c soumith magma-cuda75
$ pip install -r requirements.txt
$ python setup.py install

yunjey · March 4, 2017, 11:51am

Did you change this command export CMAKE_PREFIX_PATH=/home/yunjey/anaconda3? Actually, you should not write /home/yunjey/anaconda3 but your anaconda installation path.

mderakhshani · March 4, 2017, 11:58am

@yunjey. yes, I’ve change it to export CMAKE_PREFIX_PATH=~/anaconda3/

yunjey · March 4, 2017, 12:23pm

Did you do source anaconda3/bin/activate ~/anaconda3 before run the commands?

mderakhshani · March 4, 2017, 12:37pm

@yunjey yes, I’ve done it. no effects. no solving.
@apaszke, Sorry man, Are there any opinions with you about my problem?

apaszke · March 4, 2017, 11:21pm

It’s not a problem with anaconda but with the CUDA driver. It seems that some of the libraries are missing. Can you reinstall the driver?

alexis-jacq · March 5, 2017, 12:59pm

I also have a compilation error when trying to install from source:

Install the project...
-- Install configuration: "Release"
-- Installing: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTHNN.so.1
-- Installing: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTHNN.so
-- Up-to-date: /home/alexis/src/pytorch/torch/lib/tmp_install/include/THNN/THNN.h
-- Up-to-date: /home/alexis/src/pytorch/torch/lib/tmp_install/include/THNN/generic/THNN.h
-- Removing -DNDEBUG from compile flags
-- TH_LIBRARIES: /home/alexis/src/pytorch/torch/lib/tmp_install/lib/libTH.so.1
-- Compiling with MAGMA support
-- MAGMA INCLUDE DIRECTORIES: /home/alexis/anaconda2/include
-- MAGMA LIBRARIES: /home/alexis/anaconda2/lib/libmagma.a
-- MAGMA V2 check: 1
-- Autodetected CUDA architecture(s): 5.0
-- got cuda version 8.0
-- Found CUDA with FP16 support, compiling with torch.CudaHalfTensor
-- CUDA_NVCC_FLAGS:  -DTH_INDEX_BASE=0 -I/home/alexis/src/pytorch/torch/lib/tmp_install/include   -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/TH -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/THC   -I/home/alexis/src/pytorch/torch/lib/tmp_install/include/THPP;-gencode;arch=compute_50,code=sm_50;-DCUDA_HAS_FP16=1
-- THC_SO_VERSION: 1
-- Configuring done
-- Generating done
-- Build files have been written to: /home/alexis/src/pytorch/torch/lib/build/THC
[  1%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorMath.cu.o
[  2%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o
[  3%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorRandom.cu.o
[  5%] Building NVCC (Device) object CMakeFiles/THC.dir/THC_generated_THCTensorScatterGather.cu.o
In file included from /home/alexis/src/pytorch/torch/lib/THC/THCTensorMathMagma.cu:4:0:
/home/alexis/src/pytorch/torch/lib/THC/THCTensorMathMagma.cuh:5:19: fatal error: magma.h: No such file or directory
 #include <magma.h>
                   ^
compilation terminated.
CMake Error at THC_generated_THCTensorMathMagma.cu.o.cmake:207 (message):
  Error generating
  /home/alexis/src/pytorch/torch/lib/build/THC/CMakeFiles/THC.dir//./THC_generated_THCTensorMathMagma.cu.o


CMakeFiles/THC.dir/build.make:3869: recipe for target 'CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o' failed
make[2]: *** [CMakeFiles/THC.dir/THC_generated_THCTensorMathMagma.cu.o] Error 1
make[2]: *** Waiting for unfinished jobs....

before I run:

$ git clone https://github.com/pytorch/pytorch
$ export CMAKE_PREFIX_PATH=/home/alexis/anaconda2
$ conda install numpy mkl setuptools cmake gcc cffi
$ conda install -c soumith magma-cuda80
$ pip install -r requirements.txt
$ python setup.py install

It seems that magma.h is not included, while I have the cmake message -- MAGMA INCLUDE DIRECTORIES: /home/alexis/anaconda2/include that looks ok…

Edit: Problem solved after updating my version of cuda to the release one.

ajdroid · March 15, 2017, 9:00am

@mderakhshani Did you fix this by reinstalling CUDA?

mderakhshani · March 15, 2017, 9:43am

Hey, negative response! I have resigned to reinstall it! sorry man!

elanmart · March 20, 2017, 1:42pm

I’ve experienced the very same problem.

Grabbing the latest cuda-8.0 and reinstalling it solved the issue.

ywu36 · March 28, 2017, 10:30pm

I faced the same problem when installing Pytorch in a new computer today. I solve it by edit the ‘.bashrc’, please make sure the environment variables of CUDA is set accurately as:

export PATH="/usr/local/cuda/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda/lib64/:$LD_LIBRARY_PATH"
export CUDNN_PATH="/home/yuhang/dependencies/cudnn-5/cuda/lib64/libcudnn.so.5"

The actual path will change based on your setup.
Hope it will helps.

yunpengli · April 20, 2017, 9:18pm

I came into the same problem. For some reason in my environmental variables there is a CPLUS_INCLUDE_PATH=/usr/local/cuda/include. I solved this with export CPLUS_INCLUDE_PATH=

See ref here: https://code.google.com/archive/p/thrust/issues/359

WeiFoo · May 15, 2017, 3:11pm

export CUDNN_PATH=“/home/yuhang/dependencies/cudnn-5/cuda/lib64/libcudnn.so.5”

this saves my day!!!
thanks

K23 · January 11, 2018, 4:31pm

error running this command:

python setup.py install
Could not find /home/koeldc23/pytorch/torch/lib/gloo/CMakeLists.txt
Did you run ‘git submodule update --init’?

ptrblck · January 11, 2018, 5:39pm

Have you pulled Pytorch with the --recursive flag?
I think I had the same issue when I forgot it.

SimonW · January 11, 2018, 5:40pm

So run git submodule update --init? The error is pretty clear itself.

SimonW · January 11, 2018, 5:47pm

try installing bzip2-devel

K23 · January 11, 2018, 7:03pm

is there a way to install bzip2-devel without having root access? i’ve trying different things but no luck so far.