Failed to compile pytorch from source. Could NOT find Gloo (missing: Gloo_LIBRARY)

Hi,
I try to compile pytorch from source code by python setup.py install, but encounter error:

+ popd
~/liujiaxiang/pytorch/third_party ~/liujiaxiang/pytorch
+ local lib_prefix=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libgloo
++ uname
+ [[ Linux == \D\a\r\w\i\n ]]
+ popd
~/liujiaxiang/pytorch
+ for arg in '"$@"'
+ [[ THD == \n\c\c\l ]]
+ [[ THD == \g\l\o\o ]]
+ [[ THD == \c\a\f\f\e\2 ]]
+ [[ THD == \T\H\D ]]
+ pushd /home/liyukun01/liujiaxiang/pytorch/torch/lib
~/liujiaxiang/pytorch/torch/lib ~/liujiaxiang/pytorch
+ build THD
+ mkdir -p build/THD
+ pushd build/THD
~/liujiaxiang/pytorch/torch/lib/build/THD ~/liujiaxiang/pytorch/torch/lib ~/liujiaxiang/pytorch
+ BUILD_C_FLAGS=
+ case $1 in
+ BUILD_C_FLAGS=' -DTH_INDEX_BASE=0 -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/TH" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THC"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THS" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCS"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THNN" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1 -fexceptions'
+ cmake ../../THD -DCMAKE_MODULE_PATH=/home/liyukun01/liujiaxiang/pytorch/cmake/Modules_CUDA_fix -DTorch_FOUND=1 -DCMAKE_INSTALL_PREFIX=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install '-DCMAKE_C_FLAGS= -DTH_INDEX_BASE=0 -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/TH" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THC"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THS" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCS"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THNN" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1 -fexceptions ' '-DCMAKE_CXX_FLAGS= -DTH_INDEX_BASE=0 -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/TH" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THC"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THS" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCS"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THNN" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1 -fexceptions  -std=c++11  ' '-DCMAKE_EXE_LINKER_FLAGS=-L"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib"  -Wl,-rpath,$ORIGIN ' '-DCMAKE_SHARED_LINKER_FLAGS=-L"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib"  -Wl,-rpath,$ORIGIN ' -DCMAKE_INSTALL_LIBDIR=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib '-DCUDA_NVCC_FLAGS= -DTH_INDEX_BASE=0 -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/TH" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THC"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THS" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCS"   -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THNN" -I"/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include/THCUNN" -DOMPI_SKIP_MPICXX=1' -DCUDA_DEVICE_DEBUG=0 -DCMAKE_PREFIX_PATH=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install '-Dcwrap_files=/home/liyukun01/liujiaxiang/pytorch/torch/lib/ATen/Declarations.cwrap;/home/liyukun01/liujiaxiang/pytorch/torch/lib/THNN/generic/THNN.h;/home/liyukun01/liujiaxiang/pytorch/torch/lib/THCUNN/generic/THCUNN.h;/home/liyukun01/liujiaxiang/pytorch/torch/lib/ATen/nn.yaml' -DTH_INCLUDE_PATH=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/include -DTH_LIB_PATH=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib -DTH_LIBRARIES=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libTH.so -DCAFFE2_LIBRARIES=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libcaffe2.so -DTHNN_LIBRARIES=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libTHNN.so -DTHCUNN_LIBRARIES=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libTHCUNN.so -DTHS_LIBRARIES=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libTHS.so -DTHC_LIBRARIES=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libTHC.so -DTHCS_LIBRARIES=/home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/lib/libTHCS.so -DTH_SO_VERSION=1 -DTHC_SO_VERSION=1 -DTHNN_SO_VERSION=1 -DTHCUNN_SO_VERSION=1 -DTHD_SO_VERSION=1 -DUSE_CUDA=1 -DNO_NNPACK=0 -DNCCL_EXTERNAL=1 -Dnanopb_BUILD_GENERATOR=0 -DCMAKE_DEBUG_POSTFIX= -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=1
CMake Warning (dev) at /home/liyukun01/liujiaxiang/pytorch/cmake-3.12.0-rc1-Linux-x86_64/share/cmake-3.12/Modules/CMakeGenericSystem.cmake:4 (include):
  File
  /home/liyukun01/liujiaxiang/pytorch/cmake-3.12.0-rc1-Linux-x86_64/share/cmake-3.12/Modules/CMakeGenericSystem.cmake
  includes
  /home/liyukun01/liujiaxiang/pytorch/cmake/Modules_CUDA_fix/CMakeInitializeConfigs.cmake
  (found via CMAKE_MODULE_PATH) which shadows
  /home/liyukun01/liujiaxiang/pytorch/cmake-3.12.0-rc1-Linux-x86_64/share/cmake-3.12/Modules/CMakeInitializeConfigs.cmake.
  This may cause errors later on .

  Policy CMP0017 is not set: Prefer files from the CMake module directory
  when including from there.  Run "cmake --help-policy CMP0017" for policy
  details.  Use the cmake_policy command to set the policy and suppress this
  warning.
Call Stack (most recent call first):
  /home/liyukun01/liujiaxiang/pytorch/cmake-3.12.0-rc1-Linux-x86_64/share/cmake-3.12/Modules/CMakeSystemSpecificInformation.cmake:21 (include)
This warning is for project developers.  Use -Wno-dev to suppress it.

-- Could NOT find Gloo (missing: Gloo_LIBRARY)
-- Caffe2: Found protobuf with new-style protobuf targets.
-- Caffe2: Protobuf version 3.5.0
-- Could NOT find CUDA (missing: CUDA_NVCC_EXECUTABLE) (Required is at least version "7.0")
CMake Warning at /home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/share/cmake/Caffe2/public/cuda.cmake:11 (message):
  Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
  or a Caffe2 dependent library, the next warning / error will give you more
  info.
Call Stack (most recent call first):
  /home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/share/cmake/Caffe2/Caffe2Config.cmake:79 (include)
  CMakeLists.txt:48 (FIND_PACKAGE)


CMake Error at /home/liyukun01/liujiaxiang/pytorch/torch/lib/tmp_install/share/cmake/Caffe2/Caffe2Config.cmake:81 (message):
  Your installed Caffe2 version uses CUDA but I cannot find the CUDA
  libraries.  Please set the proper CUDA prefixes and / or install CUDA.
Call Stack (most recent call first):
  CMakeLists.txt:48 (FIND_PACKAGE)


-- Configuring incomplete, errors occurred!
See also "/home/liyukun01/liujiaxiang/pytorch/torch/lib/build/THD/CMakeFiles/CMakeOutput.log".
See also "/home/liyukun01/liujiaxiang/pytorch/torch/lib/build/THD/CMakeFiles/CMakeError.log".
Failed to run 'bash tools/build_pytorch_libs.sh --use-cuda --use-nnpack nccl caffe2 nanopb libshm gloo THD c10d'

This is the env:
system = CentOS release 6.3 (Final) Kernel \r on an \m
gcc = gcc (GCC) 4.8.2

What can I do to fix this compile problem?

Thanks in advance.

What’s the commit number you used? We merged a couple patches recently on build systems. Could you pulling from master and build again? If you still see some error, it’d be great if you can give us the full log too. Thanks!

The commit number is fed44cb1b37fc112357b35e1e178a6ecc824cfdb
I will try to pull the master and compile.

1 Like