Hey guys, I’m new to pytorch/caffe2 and am having a really hard time getting it compiled from source with the dependencies I need, which are in this case FFMPEG and OPENCV. I need them compiled together to try to reproduce the environment of a paper that I’m trying to work with.
Well… I’ve tried everything but can’t get it working. I can get it compiled till the end but when I try to run from caffe2.python import core
I get the message saying it doesn’t have GPU support. I’m compiling it with the cuda flag and have a titan x at my disposal.
I’m running it on docker with centOS. Using python 2.7, cmake 3.11.4, gcc/g++ 5.4.0, opencv 3.4.4, conda, etc (You can see the rest in the summary below)
That’s the full output I get when compiling it. Did anyone have a similar problem and solved it?
-- ******** Summary ********
-- CMake version : 3.11.4
-- CMake command : /usr/local/bin/cmake
-- System : Linux
-- C++ compiler : /usr/local/bin/c++
-- C++ compiler version : 5.4.0
-- CXX flags : -Wno-deprecated -fvisibility-inlines-hidden -Wnon-virtual-dtor
-- Build type : Release
-- Compile definitions : TH_BLAS_MKL
-- CMAKE_PREFIX_PATH : /root/minoconda/envs/r21d
-- CMAKE_INSTALL_PREFIX : /root/miniconda/envs/r21d
-- CMAKE_MODULE_PATH : /usr/local/pytorch/cmake/Modules;/usr/local/pytorch/cmake/public/../Modules_CUDA_fix
--
-- ONNX version : 1.3.0
-- ONNX NAMESPACE : onnx_c2
-- ONNX_BUILD_TESTS : OFF
-- ONNX_BUILD_BENCHMARKS : OFF
-- ONNX_USE_LITE_PROTO : OFF
-- ONNXIFI_DUMMY_BACKEND : OFF
--
-- Protobuf compiler :
-- Protobuf includes :
-- Protobuf libraries :
-- BUILD_ONNX_PYTHON : OFF
-- Found CUDA with FP16 support, compiling with torch.cuda.HalfTensor
-- Removing -DNDEBUG from compile flags
-- Checking prototype magma_get_sgeqrf_nb for MAGMA_V2 - True
-- Compiling with MAGMA support
-- MAGMA INCLUDE DIRECTORIES: /root/miniconda/envs/r21d/include
-- MAGMA LIBRARIES: /root/miniconda/envs/r21d/lib/libmagma.a
-- MAGMA V2 check: 1
-- Could not find hardware support for NEON on this machine.
-- No OMAP3 processor on this machine.
-- No OMAP4 processor on this machine.
-- Looking for cpuid.h
-- Looking for cpuid.h - found
-- Performing Test HAVE_GCC_GET_CPUID
-- Performing Test HAVE_GCC_GET_CPUID - Success
-- Performing Test NO_GCC_EBX_FPIC_BUG
-- Performing Test NO_GCC_EBX_FPIC_BUG - Success
-- Performing Test C_HAS_AVX_1
-- Performing Test C_HAS_AVX_1 - Failed
-- Performing Test C_HAS_AVX_2
-- Performing Test C_HAS_AVX_2 - Success
-- Performing Test C_HAS_AVX2_1
-- Performing Test C_HAS_AVX2_1 - Failed
-- Performing Test C_HAS_AVX2_2
-- Performing Test C_HAS_AVX2_2 - Success
-- Performing Test CXX_HAS_AVX_1
-- Performing Test CXX_HAS_AVX_1 - Failed
-- Performing Test CXX_HAS_AVX_2
-- Performing Test CXX_HAS_AVX_2 - Success
-- Performing Test CXX_HAS_AVX2_1
-- Performing Test CXX_HAS_AVX2_1 - Failed
-- Performing Test CXX_HAS_AVX2_2
-- Performing Test CXX_HAS_AVX2_2 - Success
-- AVX compiler support found
-- AVX2 compiler support found
-- Performing Test HAS_C11_ATOMICS
-- Performing Test HAS_C11_ATOMICS - Success
-- Atomics: using C11 intrinsics
-- Performing Test BLAS_F2C_DOUBLE_WORKS
-- Performing Test BLAS_F2C_DOUBLE_WORKS - Failed
-- Performing Test BLAS_F2C_FLOAT_WORKS
-- Performing Test BLAS_F2C_FLOAT_WORKS - Success
-- Performing Test BLAS_USE_CBLAS_DOT
-- Performing Test BLAS_USE_CBLAS_DOT - Success
-- Found a library with BLAS API (mkl).
-- Found a library with LAPACK API. (mkl)
-- Found CUDA: /usr/local/cuda (found suitable version "8.0", minimum required is "5.5")
e[91mdisabling ROCM because NOT USE_ROCM is set
e[0m-- MIOpen not found. Compiling without MIOpen support
-- Found MKLDNN: /root/miniconda/envs/r21d/lib/libmkl_intel_lp64.so;/root/miniconda/envs/r21d/lib/libmkl_gnu_thread.so;/root/miniconda/envs/r21d/lib/libmkl_core.so;-fopenmp;/usr/lib64/libpthread.so;/usr/lib64/libm.so;/usr/lib64/libdl.so
-- Detecting Intel(R) MKL: trying mklml_intel
-- Detecting Intel(R) MKL: trying mklml_gnu
-- Detecting Intel(R) MKL: trying mklml
-- Detecting Intel(R) MKL: trying mkl_rt
-- Intel(R) MKL: include /root/miniconda/envs/r21d/include
-- Intel(R) MKL: lib /root/miniconda/envs/r21d/lib/libmkl_rt.so
-- Found OpenMP_C: -fopenmp (found version "4.0")
-- Found OpenMP_CXX: -fopenmp (found version "4.0")
-- OpenMP lib: -fopenmp
-- Found Doxygen: /usr/bin/doxygen (found version "1.8.5") found components: doxygen dot
-- VTune profiling environment is unset
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for mmap
-- Looking for mmap - found
-- Looking for shm_open
-- Looking for shm_open - found
-- Looking for shm_unlink
-- Looking for shm_unlink - found
-- Looking for malloc_usable_size
-- Looking for malloc_usable_size - found
-- Performing Test C_HAS_THREAD
-- Performing Test C_HAS_THREAD - Success
-- GCC 5.4.0: Adding gcc and gcc_s libs to link line
-- Check size of long double
-- Check size of long double - done
-- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE
-- Performing Test COMPILER_SUPPORTS_LONG_DOUBLE - Success
-- Performing Test COMPILER_SUPPORTS_FLOAT128
-- Performing Test COMPILER_SUPPORTS_FLOAT128 - Success
-- Performing Test COMPILER_SUPPORTS_SSE2
-- Performing Test COMPILER_SUPPORTS_SSE2 - Success
-- Performing Test COMPILER_SUPPORTS_SSE4
-- Performing Test COMPILER_SUPPORTS_SSE4 - Success
-- Performing Test COMPILER_SUPPORTS_AVX
-- Performing Test COMPILER_SUPPORTS_AVX - Success
-- Performing Test COMPILER_SUPPORTS_FMA4
-- Performing Test COMPILER_SUPPORTS_FMA4 - Success
-- Performing Test COMPILER_SUPPORTS_AVX2
-- Performing Test COMPILER_SUPPORTS_AVX2 - Success
-- Performing Test COMPILER_SUPPORTS_SVE
-- Performing Test COMPILER_SUPPORTS_SVE - Failed
-- Performing Test COMPILER_SUPPORTS_AVX512F
-- Performing Test COMPILER_SUPPORTS_AVX512F - Failed
-- Performing Test COMPILER_SUPPORTS_OPENMP
-- Performing Test COMPILER_SUPPORTS_OPENMP - Success
-- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES
-- Performing Test COMPILER_SUPPORTS_WEAK_ALIASES - Success
-- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH
-- Performing Test COMPILER_SUPPORTS_BUILTIN_MATH - Success
-- Configuring build for SLEEF-v3.2
e[91m Target system: Linux-4.4.0-139-generic
Target processor: x86_64
Host system: Linux-4.4.0-139-generic
Host processor: x86_64
Detected C compiler: GNU @ /usr/local/bin/gcc
e[0m-- Using option `-Wall -Wno-unused -Wno-attributes -Wno-unused-result -Wno-psabi -ffp-contract=off -fno-math-errno -fno-trapping-math` to compile libsleef
-- Building shared libs : OFF
-- MPFR : /usr/lib64/libmpfr.so
-- MPFR header file in /usr/include
-- GMP : /usr/lib64/libgmp.so
-- RUNNING_ON_TRAVIS : 0
-- COMPILER_SUPPORTS_OPENMP : 1
-- NCCL operators skipped due to no CUDA support
-- Including IDEEP operators
-- Including image processing operators
-- Including video processing operators
-- MPI operators skipped due to no MPI support
-- Include Observer library
-- Using lib/python2.7/site-packages as python relative installation path
-- Automatically generating missing __init__.py files.
e[91mCMake Warning at CMakeLists.txt:389 (message):
Generated cmake files are only fully tested if one builds with system glog,
gflags, and protobuf. Other settings may generate files that are not well
tested.
e[0m--
-- ******** Summary ********
-- General:
-- CMake version : 3.11.4
-- CMake command : /usr/local/bin/cmake
-- System : Linux
-- C++ compiler : /usr/local/bin/c++
-- C++ compiler version : 5.4.0
-- BLAS : MKL
-- CXX flags : -Wno-deprecated -fvisibility-inlines-hidden -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -Wno-unused-but-set-variable -Wno-maybe-uninitialized
-- Build type : Release
-- Compile definitions : TH_BLAS_MKL;ONNX_NAMESPACE=onnx_c2;MAGMA_V2;USE_C11_ATOMICS=1;HAVE_MMAP=1;_FILE_OFFSET_BITS=64;HAVE_SHM_OPEN=1;HAVE_SHM_UNLINK=1;HAVE_MALLOC_USABLE_SIZE=1
-- CMAKE_PREFIX_PATH : /root/minoconda/envs/r21d
-- CMAKE_INSTALL_PREFIX : /root/miniconda/envs/r21d
--
-- TORCH_VERSION : 1.0.0
-- CAFFE2_VERSION : 1.0.0
-- BUILD_ATEN_MOBILE : OFF
-- BUILD_ATEN_ONLY : OFF
-- BUILD_BINARY : OFF
-- BUILD_CUSTOM_PROTOBUF : ON
-- Link local protobuf : ON
-- BUILD_DOCS : OFF
-- BUILD_PYTHON : ON
-- Python version : 2.7.15
-- Python executable : /root/miniconda/envs/r21d/bin/python
-- Pythonlibs version : 2.7.15
-- Python library : /root/miniconda/envs/r21d/lib/python2.7
-- Python includes : /root/miniconda/envs/r21d/include/python2.7
-- Python site-packages: lib/python2.7/site-packages
-- BUILD_CAFFE2_OPS : ON
-- BUILD_SHARED_LIBS : ON
-- BUILD_TEST : OFF
-- USE_ASAN : OFF
-- USE_CUDA : ON
-- CUDA static link : OFF
-- USE_CUDNN : ON
-- CUDA version : 8.0
-- cuDNN version : 6.0.21
-- CUDA root directory : /usr/local/cuda
-- CUDA library : /usr/local/cuda/lib64/stubs/libcuda.so
-- cudart library : /usr/local/cuda/lib64/libcudart_static.a;-pthread;dl;/usr/lib64/librt.so
-- cublas library : /usr/local/cuda/lib64/libcublas.so;/usr/local/cuda/lib64/libcublas_device.a
-- cufft library : /usr/local/cuda/lib64/libcufft.so
-- curand library : /usr/local/cuda/lib64/libcurand.so
-- cuDNN library : /usr/local/cuda/lib64/libcudnn.so
-- nvrtc : /usr/local/cuda/lib64/libnvrtc.so
-- CUDA include path : /usr/local/cuda/include
-- NVCC executable : /usr/local/cuda/bin/nvcc
-- CUDA host compiler : /usr/local/bin/gcc
-- USE_TENSORRT : OFF
-- USE_ROCM : OFF
-- USE_EIGEN_FOR_BLAS :
-- USE_FBGEMM : OFF
-- USE_FFMPEG : ON
-- USE_GFLAGS : ON
-- USE_GLOG : ON
-- USE_LEVELDB : ON
-- LevelDB version : 1.12
-- Snappy version : 1.1.0
-- USE_LITE_PROTO : OFF
-- USE_LMDB : ON
-- LMDB version : 0.9.22
-- USE_METAL : OFF
-- USE_MKL : ON
-- USE_MKLDNN : ON
-- USE_MOBILE_OPENGL : OFF
-- USE_NCCL : OFF
-- USE_NNPACK : OFF
-- USE_NUMPY : ON
-- USE_OBSERVERS : ON
-- USE_OPENCL : OFF
-- USE_OPENCV : ON
-- OpenCV version : 3.4.4
-- USE_OPENMP : OFF
-- USE_PROF : OFF
-- USE_QNNPACK : ON
-- USE_REDIS : OFF
-- USE_ROCKSDB : OFF
-- USE_ZMQ : OFF
-- USE_DISTRIBUTED : OFF
-- Public Dependencies : Threads::Threads;caffe2::mkl;glog::glog;caffe2::mkldnn
-- Private Dependencies : qnnpack;cpuinfo;/usr/lib64/liblmdb.so;/usr/lib64/libleveldb.so;/usr/lib64/libsnappy.so;/usr/lib64/libnuma.so;opencv_core;opencv_highgui;opencv_imgproc;opencv_imgcodecs;opencv_videoio;opencv_video;/usr/local/lib/libavcodec.so;/usr/local/lib/libavformat.so;/usr/local/lib/libavutil.so;/usr/local/lib/libswscale.so;fp16;aten_op_header_gen;onnxifi_loader;rt;gcc_s;gcc;dl
-- Configuring done
-- Generating done
The full output is in here: https://gist.github.com/pdrglv/b9e255ec4029f87ebe4e585439fa6b5e
Thank you!