I am trying to build pytorch from source to have GPU enabled on Mac OSX Mojave. I have CUDA 10.0 and CuDNN v7.3.0 installed, and have tried with Xcode command line tool version 8.3.2 and also 10.0. I have tried to run the build multiple times, but each time it hangs after it comes to this step:
-- Found CUDA: /usr/local/cuda (found suitable version "10.0", minimum required is "7.0")
-- Caffe2: CUDA detected: 10.0
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 10.0
-- Found CUDNN: /usr/local/cuda/include
-- Found cuDNN: v7.3.0 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib/libcudnn.7.dylib)
I am following the build steps from pytorch site and also from this link: https://zhaoyu.li/post/install-pytorch-on-mac-with-nvidia-gpu/
Appreciate any suggestions/help in resolving this issue.
Thanks!
This is the complete output.
L-iMac:pytorch Shonu$ MACOSX_DEPLOYMENT_TARGET=10.14 CC=clang CXX=clang++ CXXFLAGS=-stdlib=libc++ python setup.py install
Building wheel torch-1.1.0a0+7bb41e3
running install
setup.py::run()
running build_deps
setup.py::build_deps::run()
+ SYNC_COMMAND=cp
++ command -v rsync
+ '[' -x /usr/bin/rsync ']'
+ SYNC_COMMAND='rsync -lptgoD'
+ CMAKE_COMMAND=cmake
++ command -v cmake3
+ [[ -x '' ]]
+ USE_CUDA=0
+ USE_FBGEMM=0
+ USE_ROCM=0
+ USE_NNPACK=0
+ USE_MKLDNN=0
+ USE_QNNPACK=0
+ USE_GLOO_IBVERBS=0
+ CAFFE2_STATIC_LINK_CUDA=0
+ RERUN_CMAKE=1
+ [[ 6 -gt 0 ]]
+ case "$1" in
+ USE_CUDA=1
+ shift
+ [[ 5 -gt 0 ]]
+ case "$1" in
+ USE_FBGEMM=1
+ shift
+ [[ 4 -gt 0 ]]
+ case "$1" in
+ USE_NNPACK=1
+ shift
+ [[ 3 -gt 0 ]]
+ case "$1" in
+ USE_MKLDNN=1
+ shift
+ [[ 2 -gt 0 ]]
+ case "$1" in
+ USE_QNNPACK=1
+ shift
+ [[ 1 -gt 0 ]]
+ case "$1" in
+ break
+ CMAKE_INSTALL='make install'
+ BUILD_SHARED_LIBS=ON
+ USER_CFLAGS=
+ USER_LDFLAGS=
+ [[ -n '' ]]
+ [[ -n '' ]]
+ [[ -n '' ]]
++ uname
+ '[' Darwin == Darwin ']'
+ '[' -d /usr/local/opt/ccache/libexec ']'
+++ dirname ../tools/build_pytorch_libs.sh
++ cd ../tools/..
+++ pwd
++ printf '%q\n' /Users/Shonu/pytorch
+ BASE_DIR=/Users/Shonu/pytorch
+ TORCH_LIB_DIR=/Users/Shonu/pytorch/torch/lib
+ INSTALL_DIR=/Users/Shonu/pytorch/torch/lib/tmp_install
+ THIRD_PARTY_DIR=/Users/Shonu/pytorch/third_party
+ C_FLAGS=
+ C_FLAGS=' -DOMPI_SKIP_MPICXX=1'
+ LDFLAGS=
+ LD_POSTFIX=.so
++ uname
+ [[ Darwin == \D\a\r\w\i\n ]]
+ LDFLAGS=' -Wl,-rpath,@loader_path'
+ LD_POSTFIX=.dylib
+ CPP_FLAGS=' -std=c++11 '
+ THD_FLAGS=
+ [[ 0 -eq 1 ]]
+ CUDA_NVCC_FLAGS=' -DOMPI_SKIP_MPICXX=1'
+ [[ -z '' ]]
+ CUDA_DEVICE_DEBUG=0
+ '[' -z '' ']'
++ getconf _NPROCESSORS_ONLN
+ MAX_JOBS=8
+ BUILD_TYPE=Release
+ [[ -n '' ]]
+ [[ -n '' ]]
+ report 'Building in Release mode'
+ echo 'Building in Release mode'
Building in Release mode
+ mkdir -p /Users/Shonu/pytorch/torch/lib/tmp_install
+ for arg in '"$@"'
+ [[ caffe2 == \c\a\f\f\e\2 ]]
+ build_caffe2
+ [[ -z '' ]]
+ EXTRA_CAFFE2_CMAKE_FLAGS=()
+ [[ -n '' ]]
+ [[ -n /usr/local/miniconda3/lib/python3.7/site-packages ]]
+ EXTRA_CAFFE2_CMAKE_FLAGS+=("-DCMAKE_PREFIX_PATH=$CMAKE_PREFIX_PATH")
+ [[ 1 -eq 1 ]]
+ cmake /Users/Shonu/pytorch -DPYTHON_EXECUTABLE=/usr/local/miniconda3/bin/python -DPYTHON_LIBRARY=/usr/local/miniconda3/lib/libpython3.7m.dylib -DPYTHON_INCLUDE_DIR=/usr/local/miniconda3/include/python3.7m -DBUILDING_WITH_TORCH_LIBS=ON -DTORCH_BUILD_VERSION=1.1.0a0+7bb41e3 -DCMAKE_BUILD_TYPE=Release -DBUILD_TORCH=ON -DBUILD_PYTHON=ON -DBUILD_SHARED_LIBS=ON -DBUILD_BINARY=OFF -DBUILD_TEST=ON -DINSTALL_TEST=ON -DBUILD_CAFFE2_OPS=ON -DONNX_NAMESPACE=onnx_torch -DUSE_CUDA=1 -DUSE_DISTRIBUTED=ON -DUSE_FBGEMM=1 -DUSE_NUMPY= -DNUMPY_INCLUDE_DIR=/usr/local/miniconda3/lib/python3.7/site-packages/numpy/core/include -DUSE_SYSTEM_NCCL=OFF -DNCCL_INCLUDE_DIR= -DNCCL_ROOT_DIR= -DNCCL_SYSTEM_LIB= -DCAFFE2_STATIC_LINK_CUDA=0 -DUSE_ROCM=0 -DUSE_NNPACK=1 -DUSE_LEVELDB=OFF -DUSE_LMDB=OFF -DUSE_OPENCV=OFF -DUSE_QNNPACK=1 -DUSE_TENSORRT=OFF -DUSE_FFMPEG=OFF -DUSE_GLOG=OFF -DUSE_GFLAGS=OFF -DUSE_SYSTEM_EIGEN_INSTALL=OFF -DCUDNN_INCLUDE_DIR=/usr/local/cuda/include -DCUDNN_LIB_DIR=/usr/local/cuda/lib -DCUDNN_LIBRARY=/usr/local/cuda/lib/libcudnn.7.dylib -DUSE_MKLDNN=1 -DNCCL_EXTERNAL=1 -DCMAKE_INSTALL_PREFIX=/Users/Shonu/pytorch/torch/lib/tmp_install -DCMAKE_C_FLAGS= -DCMAKE_CXX_FLAGS= '-DCMAKE_EXE_LINKER_FLAGS= -Wl,-rpath,@loader_path ' '-DCMAKE_SHARED_LINKER_FLAGS= -Wl,-rpath,@loader_path ' -DTHD_SO_VERSION=1 -DCMAKE_PREFIX_PATH=/usr/local/miniconda3/lib/python3.7/site-packages
-- The CXX compiler identification is AppleClang 10.0.0.10001044
-- The C compiler identification is AppleClang 10.0.0.10001044
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/clang++
-- Check for working CXX compiler: /Library/Developer/CommandLineTools/usr/bin/clang++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/clang
-- Check for working C compiler: /Library/Developer/CommandLineTools/usr/bin/clang -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Not forcing any particular BLAS to be found
-- CLANG_VERSION_STRING: 10.0
-- Performing Test COMPILER_WORKS
-- Performing Test COMPILER_WORKS - Success
-- Performing Test SUPPORT_GLIBCXX_USE_C99
-- Performing Test SUPPORT_GLIBCXX_USE_C99 - Success
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED
-- Performing Test CAFFE2_EXCEPTION_PTR_SUPPORTED - Success
-- std::exception_ptr is supported.
-- Performing Test CAFFE2_IS_NUMA_AVAILABLE
-- Performing Test CAFFE2_IS_NUMA_AVAILABLE - Failed
-- NUMA is not available
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING
-- Performing Test CAFFE2_NEED_TO_TURN_OFF_DEPRECATION_WARNING - Failed
-- Turning off deprecation warning due to glog.
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX2_EXTENSIONS - Success
-- Current compiler supports avx2 extension. Will build perfkernels.
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512F_EXTENSIONS
-- Performing Test CAFFE2_COMPILER_SUPPORTS_AVX512F_EXTENSIONS - Success
-- Current compiler supports avx512f extension. Will build fbgemm.
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY
-- Performing Test COMPILER_SUPPORTS_HIDDEN_INLINE_VISIBILITY - Success
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC
-- Performing Test COMPILER_SUPPORTS_RDYNAMIC - Success
-- Building using own protobuf under third_party per request.
-- Use custom protobuf build.
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for pthread_create
-- Looking for pthread_create - found
-- Found Threads: TRUE
-- Caffe2 protobuf include directory: $<BUILD_INTERFACE:/Users/Shonu/pytorch/third_party/protobuf/src>$<INSTALL_INTERFACE:include>
-- The BLAS backend of choice:MKL
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stddef.h
-- Looking for stddef.h - found
-- Check size of void*
-- Check size of void* - done
-- Checking for [mkl_intel_lp64 - mkl_intel_thread - mkl_core - iomp5 - pthread - m]
-- Library mkl_intel_lp64: /usr/local/miniconda3/lib/libmkl_intel_lp64.dylib
-- Library mkl_intel_thread: /usr/local/miniconda3/lib/libmkl_intel_thread.dylib
-- Library mkl_core: /usr/local/miniconda3/lib/libmkl_core.dylib
-- Library iomp5: /usr/local/miniconda3/lib/libiomp5.dylib
-- Library pthread: /usr/lib/libpthread.dylib
-- Library m: /usr/lib/libm.dylib
-- Looking for cblas_sgemm
-- Looking for cblas_sgemm - found
-- The ASM compiler identification is AppleClang
-- Found assembler: /Library/Developer/CommandLineTools/usr/bin/clang
-- Brace yourself, we are building NNPACK
-- Found PythonInterp: /usr/local/miniconda3/bin/python (found version "3.7.1")
-- Failed to find LLVM FileCheck
-- Found Git: /usr/bin/git (found version "2.17.1 (Apple Git-112)")
-- git Version: v1.4.0-505be96a
-- Version: 1.4.0
-- Performing Test HAVE_CXX_FLAG_STD_CXX11
-- Performing Test HAVE_CXX_FLAG_STD_CXX11 - Success
-- Performing Test HAVE_CXX_FLAG_WALL
-- Performing Test HAVE_CXX_FLAG_WALL - Success
-- Performing Test HAVE_CXX_FLAG_WEXTRA
-- Performing Test HAVE_CXX_FLAG_WEXTRA - Success
-- Performing Test HAVE_CXX_FLAG_WSHADOW
-- Performing Test HAVE_CXX_FLAG_WSHADOW - Success
-- Performing Test HAVE_CXX_FLAG_WERROR
-- Performing Test HAVE_CXX_FLAG_WERROR - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC
-- Performing Test HAVE_CXX_FLAG_PEDANTIC - Success
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS
-- Performing Test HAVE_CXX_FLAG_PEDANTIC_ERRORS - Success
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32
-- Performing Test HAVE_CXX_FLAG_WSHORTEN_64_TO_32 - Success
-- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL
-- Performing Test HAVE_CXX_FLAG_WFLOAT_EQUAL - Success
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_FSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS
-- Performing Test HAVE_CXX_FLAG_WNO_DEPRECATED_DECLARATIONS - Success
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING
-- Performing Test HAVE_CXX_FLAG_WSTRICT_ALIASING - Success
-- Performing Test HAVE_CXX_FLAG_WD654
-- Performing Test HAVE_CXX_FLAG_WD654 - Failed
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY
-- Performing Test HAVE_CXX_FLAG_WTHREAD_SAFETY - Success
-- Performing Test HAVE_THREAD_SAFETY_ATTRIBUTES
-- Performing Test HAVE_THREAD_SAFETY_ATTRIBUTES
-- Performing Test HAVE_THREAD_SAFETY_ATTRIBUTES -- failed to compile
-- Performing Test HAVE_CXX_FLAG_COVERAGE
-- Performing Test HAVE_CXX_FLAG_COVERAGE - Success
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX
-- Performing Test HAVE_STD_REGEX -- success
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX
-- Performing Test HAVE_GNU_POSIX_REGEX -- failed to compile
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX
-- Performing Test HAVE_POSIX_REGEX -- success
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK
-- Performing Test HAVE_STEADY_CLOCK -- success
-- Performing Test COMPILER_SUPPORTS_AVX512
-- Performing Test COMPILER_SUPPORTS_AVX512 - Success
-- Could NOT find OpenMP_C (missing: OpenMP_C_FLAGS OpenMP_C_LIB_NAMES)
-- Could NOT find OpenMP_CXX (missing: OpenMP_CXX_FLAGS OpenMP_CXX_LIB_NAMES)
-- Could NOT find OpenMP (missing: OpenMP_C_FOUND OpenMP_CXX_FOUND)
CMake Warning at third_party/fbgemm/CMakeLists.txt:61 (message):
OpenMP is not supported by the compiler
-- Downloading asmjit to /Users/Shonu/pytorch/build/third_party/fbgemm/third_party/asmjit
(define ASMJIT_SRC_DIR to avoid it)
-- Configuring done
-- Generating done
-- Build files have been written to: /Users/Shonu/pytorch/build/third_party/fbgemm/asmjit-download
[ 11%] Performing update step for 'asmjit'
[ 22%] No configure step for 'asmjit'
[ 33%] No build step for 'asmjit'
[ 44%] No install step for 'asmjit'
[ 55%] No test step for 'asmjit'
[ 66%] Completed 'asmjit'
[100%] Built target asmjit
-- Performing Test __CxxFlag__msse
-- Performing Test __CxxFlag__msse - Success
-- Performing Test __CxxFlag__msse2
-- Performing Test __CxxFlag__msse2 - Success
-- Performing Test __CxxFlag__msse3
-- Performing Test __CxxFlag__msse3 - Success
-- Performing Test __CxxFlag__mssse3
-- Performing Test __CxxFlag__mssse3 - Success
-- Performing Test __CxxFlag__msse4_1
-- Performing Test __CxxFlag__msse4_1 - Success
-- Performing Test __CxxFlag__msse4_2
-- Performing Test __CxxFlag__msse4_2 - Success
-- Performing Test __CxxFlag__mavx
-- Performing Test __CxxFlag__mavx - Success
-- Performing Test __CxxFlag__mavx2
-- Performing Test __CxxFlag__mavx2 - Success
-- Performing Test __CxxFlag__std_c__17
-- Performing Test __CxxFlag__std_c__17 - Success
-- Performing Test __CxxFlag__std_c__14
-- Performing Test __CxxFlag__std_c__14 - Success
-- Performing Test __CxxFlag__std_c__11
-- Performing Test __CxxFlag__std_c__11 - Success
-- Performing Test __CxxFlag__std_c__0x
-- Performing Test __CxxFlag__std_c__0x - Success
-- [asmjit]
BuildMode=Static
BuildTest=Off
ASMJIT_DIR=/Users/Shonu/pytorch/build/third_party/fbgemm/third_party/asmjit
ASMJIT_DEPS=pthread
ASMJIT_LIBS=asmjit;pthread
ASMJIT_CFLAGS=-DASMJIT_STATIC
ASMJIT_SOURCE_DIR=/Users/Shonu/pytorch/build/third_party/fbgemm/third_party/asmjit/src
ASMJIT_INCLUDE_DIR=/Users/Shonu/pytorch/build/third_party/fbgemm/third_party/asmjit/src
ASMJIT_PRIVATE_CFLAGS=
-DASMJIT_STATIC
-std=c++17
CMake Warning at cmake/Dependencies.cmake:398 (message):
NUMA is currently only supported under Linux.
Call Stack (most recent call first):
CMakeLists.txt:199 (include)
-- Using third party subdirectory Eigen.
Python 3.7.1
-- Found PythonInterp: /usr/local/miniconda3/bin/python (found suitable version "3.7.1", minimum required is "2.7")
-- Found PythonLibs: /usr/local/miniconda3/lib/libpython3.7m.dylib (found suitable version "3.7.1", minimum required is "2.7")
-- Could NOT find pybind11 (missing: pybind11_DIR)
-- Could NOT find pybind11 (missing: pybind11_INCLUDE_DIR)
-- Using third_party/pybind11.
-- Could NOT find MPI_C (missing: MPI_C_LIB_NAMES MPI_C_HEADER_DIR MPI_C_WORKS)
-- Could NOT find MPI_CXX (missing: MPI_CXX_LIB_NAMES MPI_CXX_HEADER_DIR MPI_CXX_WORKS)
-- Could NOT find MPI (missing: MPI_C_FOUND MPI_CXX_FOUND)
CMake Warning at cmake/Dependencies.cmake:633 (message):
Not compiling with MPI. Suppress this warning with -DUSE_MPI=OFF
Call Stack (most recent call first):
CMakeLists.txt:199 (include)
-- Found CUDA: /usr/local/cuda (found suitable version "10.0", minimum required is "7.0")
-- Caffe2: CUDA detected: 10.0
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 10.0
-- Found CUDNN: /usr/local/cuda/include
-- Found cuDNN: v7.3.0 (include: /usr/local/cuda/include, library: /usr/local/cuda/lib/libcudnn.7.dylib)