I have a conda environment with all require dependencies to build the C++ LibTorch Libraries on my Ubunutu 18.04 Machine. The cmake build process with Cuda 11.1 works fine.
SOURCE_DIR=pytorch
BUILD_DIR=pytorch-cuda-build
INSTALL_DIR=pytorch-cuda-install
echo "Setup exports for Cuda 11.1..."
export CUDA_NVCC_EXECUTABLE="/usr/local/cuda-11.1/bin/nvcc"
export CUDA_HOME="/usr/local/cuda-11.1"
export CUDA_BIN_PATH="/usr/local/cuda-11.1"
export CUDA_TOOLKIT_ROOT_DIR="/usr/local/cuda-11.1"
export CUDNN_INCLUDE_PATH="/usr/local/cuda-11.1/include/"
export CUDNN_LIBRARY_PATH="/usr/local/cuda-11.1/lib64/"
export LIBRARY_PATH="/usr/local/cuda-11.1/lib64"
mkdir $BUILD_DIR
cd $BUILD_DIR
echo "Build libtorch with Cuda 11.1 support..."
export USE_CUDA=1 USE_CUDNN=1 USE_MKLDNN=1
cmake -j 8 -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DPYTHON_EXECUTABLE:PATH=`which python3` -DCMAKE_INSTALL_PREFIX:PATH=../$INSTALL_DIR ../$SOURCE_DIR
Now I need compile the LibTorch Libraries without the CUDA envrionment. Therefore I changed the script.
echo "Build libtorch without Cuda support..."
export USE_CUDA=0 USE_CUDNN=0
cmake -j 8 -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DPYTHON_EXECUTABLE:PATH=`which python3` -DCMAKE_INSTALL_PREFIX:PATH=../$INSTALL_DIR ../$SOURCE_DIR
The compilation fails with the following error message. It looks like the scripts still search for any CUDA libraries. Any Ideas?
Build libtorch without Cuda...
-- The CXX compiler identification is GNU 8.4.0
-- The C compiler identification is GNU 8.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Not forcing any particular BLAS to be found
-- Performing Test COMPILER_WORKS
-- Performing Test COMPILER_WORKS - Success
-- Performing Test SUPPORT_GLIBCXX_USE_C99
-- Performing Test SUPPORT_GLIBCXX_USE_C99 - Success
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED - Success
-- std::exception_ptr is supported.
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING - Success
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS - Success
-- Current compiler supports avx2 extension. Will build perfkernels.
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512_EXTENSIONS - Success
-- Current compiler supports avx512f extension. Will build fbgemm.
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC - Success
-- Building using own protobuf under third_party per request.
-- Use custom protobuf build.
--
-- 3.11.4.0
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Check if compiler accepts -pthread
-- Check if compiler accepts -pthread - yes
-- Found Threads: TRUE
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS
-- Performing Test protobuf_HAVE_BUILTIN_ATOMICS - Success
-- Caffe2 protobuf include directory: $<BUILD_INTERFACE:/home/dd/workspace/AI_EnvPerception/frameworks/pytorch/third_party/protobuf/src>$<INSTALL_INTERFACE:include>
-- Trying to find preferred BLAS backend of choice: MKL
-- MKL_THREADING = OMP
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to `find_package_handle_standard_args` (OpenMP_C)
does not match the name of the calling package (OpenMP). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
cmake/Modules/FindMKL.cmake:213 (FIND_PACKAGE)
cmake/Modules/FindMKL.cmake:307 (CHECK_ALL_LIBRARIES)
cmake/Dependencies.cmake:144 (find_package)
CMakeLists.txt:621 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)
does not match the name of the calling package (OpenMP). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
cmake/Modules/FindMKL.cmake:213 (FIND_PACKAGE)
cmake/Modules/FindMKL.cmake:307 (CHECK_ALL_LIBRARIES)
cmake/Dependencies.cmake:144 (find_package)
CMakeLists.txt:621 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Looking for cblas_sgemm
-- Looking for cblas_sgemm - found
-- MKL libraries: /opt/intel/mkl/lib/intel64/libmkl_intel_lp64.so;/opt/intel/mkl/lib/intel64/libmkl_gnu_thread.so;/opt/intel/mkl/lib/intel64/libmkl_core.so;-fopenmp;/usr/lib/x86_64-linux-gnu/libpthread.so;/usr/lib/x86_64-linux-gnu/libm.so;/usr/lib/x86_64-linux-gnu/libdl.so
-- MKL include directory: /opt/intel/mkl/include
-- MKL OpenMP type: GNU
-- MKL OpenMP library: -fopenmp
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Brace yourself, we are building NNPACK
-- Performing Test NNPACK_ARCH_IS_X86_32
-- Performing Test NNPACK_ARCH_IS_X86_32 - Failed
-- Found PythonInterp: /home/dd/miniconda3/envs/pytorch-build/bin/python3 (found version "3.6.13")
-- NNPACK backend is x86-64
-- Performing Test COMPILER_SUPPORTS_AVX512
-- Performing Test COMPILER_SUPPORTS_AVX512 - Success
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to `find_package_handle_standard_args` (OpenMP_C)
does not match the name of the calling package (OpenMP). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
third_party/fbgemm/CMakeLists.txt:60 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found OpenMP_C: -fopenmp (found version "4.5")
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)
does not match the name of the calling package (OpenMP). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
third_party/fbgemm/CMakeLists.txt:60 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
CMake Warning at third_party/fbgemm/CMakeLists.txt:62 (message):
OpenMP found! OpenMP_C_INCLUDE_DIRS =
CMake Warning at third_party/fbgemm/CMakeLists.txt:142 (message):
==========
CMake Warning at third_party/fbgemm/CMakeLists.txt:143 (message):
CMAKE_BUILD_TYPE = Release
CMake Warning at third_party/fbgemm/CMakeLists.txt:144 (message):
CMAKE_CXX_FLAGS_DEBUG is -g
CMake Warning at third_party/fbgemm/CMakeLists.txt:145 (message):
CMAKE_CXX_FLAGS_RELEASE is -O3 -DNDEBUG
CMake Warning at third_party/fbgemm/CMakeLists.txt:146 (message):
==========
-- Performing Test __CxxFlag__fno_threadsafe_statics
-- Performing Test __CxxFlag__fno_threadsafe_statics - Success
-- Performing Test __CxxFlag__fno_semantic_interposition
-- Performing Test __CxxFlag__fno_semantic_interposition - Success
-- Performing Test __CxxFlag__fmerge_all_constants
-- Performing Test __CxxFlag__fmerge_all_constants - Success
-- Performing Test __CxxFlag__fno_enforce_eh_specs
-- Performing Test __CxxFlag__fno_enforce_eh_specs - Success
** AsmJit Summary **
ASMJIT_DIR=/home/dd/workspace/AI_EnvPerception/frameworks/pytorch/third_party/fbgemm/third_party/asmjit
ASMJIT_TEST=FALSE
ASMJIT_TARGET_TYPE=STATIC
ASMJIT_DEPS=pthread;rt
ASMJIT_LIBS=asmjit;pthread;rt
ASMJIT_CFLAGS=-DASMJIT_STATIC
ASMJIT_PRIVATE_CFLAGS=-Wall;-Wextra;-Wconversion;-fno-math-errno;-fno-threadsafe-statics;-fno-semantic-interposition;-DASMJIT_STATIC
ASMJIT_PRIVATE_CFLAGS_DBG=
ASMJIT_PRIVATE_CFLAGS_REL=-O2;-fmerge-all-constants;-fno-enforce-eh-specs
-- Found Numa: /usr/include
-- Found Numa (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libnuma.so)
-- Using third party subdirectory Eigen.
Python 3.6.13 :: Anaconda, Inc.
-- Setting Python's include dir to /home/dd/miniconda3/envs/pytorch-build/include/python3.6m from sysconfig
-- Setting Python's library to /home/dd/miniconda3/envs/pytorch-build/lib/python3.6
-- Found PythonInterp: /home/dd/miniconda3/envs/pytorch-build/bin/python3 (found suitable version "3.6.13", minimum required is "3.0")
-- Found PythonLibs: /home/dd/miniconda3/envs/pytorch-build/lib/python3.6 (found suitable version "3.6.13", minimum required is "3.0")
-- Found NumPy: /home/dd/.local/lib/python3.6/site-packages/numpy/core/include (found version "1.19.5")
-- NumPy ver. 1.19.5 found (include: /home/dd/.local/lib/python3.6/site-packages/numpy/core/include)
-- Could NOT find pybind11 (missing: pybind11_DIR)
-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR)
-- Using third_party/pybind11.
-- pybind11 include dirs: /home/dd/workspace/AI_EnvPerception/frameworks/pytorch/cmake/../third_party/pybind11/include
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- MPI support found
-- MPI compile flags: -pthread
-- MPI include path: /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include/usr/lib/x86_64-linux-gnu/openmpi/include
-- MPI LINK flags path: -L/usr/lib -pthread
-- MPI libraries: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so/usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so
CMake Warning at cmake/Dependencies.cmake:1045 (message):
OpenMPI found, but it is not built with CUDA support.
Call Stack (most recent call first):
CMakeLists.txt:621 (include)
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to `find_package_handle_standard_args` (OpenMP_C)
does not match the name of the calling package (OpenMP). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
cmake/Dependencies.cmake:1105 (find_package)
CMakeLists.txt:621 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
CMake Warning (dev) at /home/dd/miniconda3/envs/pytorch-build/share/cmake-3.19/Modules/FindPackageHandleStandardArgs.cmake:426 (message):
The package name passed to `find_package_handle_standard_args` (OpenMP_CXX)
does not match the name of the calling package (OpenMP). This can lead to
problems in calling code that expects `find_package` result variables
(e.g., `_FOUND`) to follow a certain pattern.
Call Stack (most recent call first):
cmake/Modules/FindOpenMP.cmake:576 (find_package_handle_standard_args)
cmake/Dependencies.cmake:1105 (find_package)
CMakeLists.txt:621 (include)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Adding OpenMP CXX_FLAGS: -fopenmp
-- Will link against OpenMP libraries: /usr/lib/gcc/x86_64-linux-gnu/8/libgomp.so;/usr/lib/x86_64-linux-gnu/libpthread.so
-- Found CUDA: /usr (found version "9.1")
-- Caffe2: CUDA detected: 9.1
-- Caffe2: CUDA nvcc is: /usr/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr
-- Caffe2: Header version is: 9.1
-- Found CUDNN: /usr/lib/x86_64-linux-gnu/libcudnn.so
-- Found cuDNN: v8.2.4 (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- /usr/lib/x86_64-linux-gnu/libnvrtc.so shorthash is 9862b2bf
CMake Error at cmake/public/cuda.cmake:440 (message):
CUDA 9.1 is not compatible with std::tuple from GCC version >= 6. Please
upgrade to CUDA 9.2 or set the following environment variable to use
another version (for example):
export CUDAHOSTCXX='/usr/bin/gcc-5'
Call Stack (most recent call first):
cmake/Dependencies.cmake:1155 (include)
CMakeLists.txt:621 (include)
-- Configuring incomplete, errors occurred!
See also "/home/dd/workspace/AI_EnvPerception/frameworks/pytorch-cpu-build/CMakeFiles/CMakeOutput.log".
See also "/home/dd/workspace/AI_EnvPerception/frameworks/pytorch-cpu-build/CMakeFiles/CMakeError.log".
make: *** No rule to make target 'install'. Stop.
Build done! Check pytorch-cpu-build folder for libraries.