Cpp_extension looks for the wrong version of nvcc


(Joong Kun Lee) #1

I have multiple versions of cuda toolkit and driver on my server ( 8.0, 9.2, and 10.0 ).
My g++ and gcc version is: 7.3.1

when I run python build.py install, it throws an error:

running install
running bdist_egg
running egg_info
writing test_make_condInput_ext.egg-info/PKG-INFO
writing dependency_links to test_make_condInput_ext.egg-info/dependency_links.txt
writing top-level names to test_make_condInput_ext.egg-info/top_level.txt
reading manifest file 'test_make_condInput_ext.egg-info/SOURCES.txt'
writing manifest file 'test_make_condInput_ext.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
building 'test_make_condInput_ext' extension
gcc -pthread -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -std=c99 -fPIC -I/home/msl/isaac/parallel-wavenet/brain-tts/pysrc/maum/brain/tts/wavenet/run/cuda/make_condInput -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include/torch/csrc/api/include -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include/TH -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/usr/local/include/python3.5m -c test_make_condInput_wrapper.cpp -o build/temp.linux-x86_64-3.5/test_make_condInput_wrapper.o -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=test_make_condInput_ext -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++
cc1plus: warning: command line option ‘-std=c99’ is valid for C/ObjC but not for C++
In file included from test_make_condInput_wrapper.cpp:2:0:
/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include/torch/csrc/api/include/torch/torch.h:7:2: warning: #warning "Including torch/torch.h for C++ extensions is deprecated. Please include torch/extension.h" [-Wcpp]
 #warning \
  ^~~~~~~
test_make_condInput_wrapper.cpp: In function ‘int runTest(int, int, at::Tensor, at::Tensor, at::Tensor, at::Tensor)’:
test_make_condInput_wrapper.cpp:26:55: warning: format ‘%i’ expects a matching ‘int’ argument [-Wformat=]
             printf("    \n%i th weight is not equal\n");
                                                       ^
/usr/local/cuda/bin/nvcc -I/home/msl/isaac/parallel-wavenet/brain-tts/pysrc/maum/brain/tts/wavenet/run/cuda/make_condInput -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include/torch/csrc/api/include -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include/TH -I/home/msl/.virtualenvs/venv_wavenet/lib/python3.5/site-packages/torch/lib/include/THC -I/usr/local/cuda/include -I/usr/local/include/python3.5m -c test_make_condInput.cu -o build/temp.linux-x86_64-3.5/test_make_condInput.o -D__CUDA_NO_HALF_OPERATORS__ -D__CUDA_NO_HALF_CONVERSIONS__ -D__CUDA_NO_HALF2_OPERATORS__ --compiler-options '-fPIC' -DTORCH_API_INCLUDE_EXTENSION_H -DTORCH_EXTENSION_NAME=test_make_condInput_ext -D_GLIBCXX_USE_CXX11_ABI=0 -std=c++11
nvcc warning : The 'compute_20', 'sm_20', and 'sm_21' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets to suppress warning).
In file included from /usr/local/cuda/include/cuda_runtime.h:78:0,
                 from <command-line>:0:
/usr/local/cuda/include/host_config.h:119:2: error: #error -- unsupported GNU version! gcc versions later than 5 are not supported!
 #error -- unsupported GNU version! gcc versions later than 5 are not supported!
  ^~~~~
error: command '/usr/local/cuda/bin/nvcc' failed with exit status 1

This line /usr/local/cuda/bin/nvcc in the message above is strange because
I have set the environmnent variable to go into '/usr/local/cuda10.0` first.

$ echo $PATH
/home/msl/.virtualenvs/venv_wavenet/bin:/usr/local/cuda-10.0/bin:/home/msl/maum/bin:/home/msl/.virtualenvs/venv_wavenet/bin:/opt/rh/devtoolset-7/root/usr/bin:/usr/local/cuda-10.0/bin

and when I run nvcc --version, I get this:

$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2018 NVIDIA Corporation
Built on Sat_Aug_25_21:08:01_CDT_2018
Cuda compilation tools, release 10.0, V10.0.130

which is what I would expect, since I have cuda 10.0 in my PATH first. So why is cpp_extension going to /usr/local/cuda/bin/nvcc?