Current pytorch.org ONLY provides the installation with Cuda9.2 …
Where can I find the pytorch installation with Cuda10.0?
Cheers
2 Likes
Hi,
You can compile from source to be able to use CUDA 10 right now.
It will be added to the pre-built packages soon (@smth should be able to give you an exact timeline).
1 Like
Hi, albanD:
I actually have built PyTorch with Cuda 10.0, however, it’s still quite complicated.
My commands are:
TORCH_CUDA_ARCH_LIST="5.2" NO_TEST=1 USE_MKLDNN=1 FULL_CAFFE2=1 python setup.py build
TORCH_CUDA_ARCH_LIST="5.2" NO_TEST=1 USE_MKLDNN=1 FULL_CAFFE2=1 python setup.py install
However, after the installation, I failed to test caffe2 with GPU support…
from caffe2.python import core
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
CRITICAL:root:Cannot load caffe2.python. Error: ~/.local/lib/python3.6/site-packages/caffe2/python/caffe2_pybind11_state.cpython-36m-x86_64-linux-gnu.so: undefined symbol: mkldnn_primitive_destroy
Where can I set MKL directories in setup.py?
I mean, if I want to build pytorch with the command python setup.py build, how can I specify the directories for MKLDNN_ROOT, etc.?
Some more pre-configuration is:
$ TORCH_CUDA_ARCH_LIST=“5.2” USE_MKLDNN=1 FULL_CAFFE2=1 python setup.py build
Building wheel torch-1.0.0a0+a78c6c4
running build
running build_deps
setup.py::build_deps::run()
- SYNC_COMMAND=cp
++ command -v rsync- ‘[’ -x /usr/bin/rsync ‘]’
- SYNC_COMMAND=‘rsync -lptgoD’
- USE_CUDA=1
- USE_ROCM=0
- USE_NNPACK=0
- USE_MKLDNN=1
- USE_GLOO_IBVERBS=0
- CAFFE2_STATIC_LINK_CUDA=0
- RERUN_CMAKE=1
- [[ 8 -gt 0 ]]
- case “$1” in
- USE_CUDA=1
- shift
- [[ 7 -gt 0 ]]
- case “$1” in
- USE_NNPACK=1
- shift
- [[ 6 -gt 0 ]]
- case “$1” in
- break
- CMAKE_INSTALL=‘make install’
- BUILD_SHARED_LIBS=ON
- USER_CFLAGS=
- USER_LDFLAGS=
- [[ -n ‘’ ]]
- [[ -n ‘’ ]]
- [[ -n ‘’ ]]
++ uname- ‘[’ Linux == Darwin ‘]’
+++ dirname …/tools/build_pytorch_libs.sh
++ cd …/tools/…
+++ pwd
Hi,
If you want pytorch, you don’t need to compile full caffe2, is there a reason you use the FULL_CAFFE2 flag? I’m not even sure it exists any more.
You can find the the setup.py here all the env variables used to find MKLDNN.
The cuda architecture list should be detected automatically, you don’t need to specify it by hand.
Which Eigen version is needed?
[ 33%] Building CXX object caffe2/CMakeFiles/caffe2.dir/operators/cross_entropy_op.cc.o
cd ....../pytorch/build/caffe2 && /usr/bin/c++ -DCPUINFO_SUPPORTED_PLATFORM=1 -DHAVE_MALLOC_USABLE_SIZE=1 -DHAVE_MMAP=1 -DHAVE_SHM_OPEN=1 -DHAVE_SHM_UNLINK=1 -DNNP_CONVOLUTION_ONLY=0 -DNNP_INFERENCE_ONLY=0 -DONNX_NAMESPACE=onnx_torch -DTH_BLAS_MKL -DUSE_C11_ATOMICS=1 -D_FILE_OFFSET_BITS=64 -Dcaffe2_EXPORTS -I....../pytorch/aten/src -I....../pytorch/build -I....../pytorch -I....../pytorch/third_party/protobuf/src -I....../pytorch/build/caffe2/contrib/aten -I....../pytorch/third_party/onnx -I....../pytorch/build/third_party/onnx -I....../pytorch/build/caffe2/aten/src/TH -I....../pytorch/aten/src/TH -I....../pytorch/build/caffe2/aten/src -I....../pytorch/build/aten/src -I....../pytorch/aten/src/THNN -I....../pytorch/aten/src/THCUNN -I....../pytorch/aten/../third_party/catch/single_include -I....../pytorch/build/caffe2/aten/src/ATen -I....../pytorch/aten/src/ATen/.. -I....../pytorch/caffe2/core/nomnigraph/include -I/usr/local/cuda/include -I....../pytorch/third_party/NNPACK/include -I....../pytorch/third_party/pthreadpool/include -I....../pytorch/third_party/cpuinfo/include -isystem /opt/intel/mkl/include -isystem /usr/local/include -isystem ....../pytorch/cmake/../third_party/eigen -isystem /usr/include/python3.6m -isystem ~/.local/lib/python3.6/site-packages/numpy/core/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include -isystem ....../pytorch/cmake/../third_party/cub -isystem ....../pytorch/torch/lib/tmp_install/include -isystem ....../pytorch/build/include -msse3 -msse4.1 -msse4.2 --std=c++11 -fvisibility-inlines-hidden -fopenmp -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -fPIC -march=native -DCUDA_HAS_FP16=1 -DHAVE_GCC_GET_CPUID -DUSE_SSE2 -DUSE_SSE4_1 -DUSE_SSE4_2 -DUSE_SSE3 -DUSE_AVX -DUSE_AVX2 -DTH_HAVE_THREAD -fvisibility=hidden -DCAFFE2_BUILD_MAIN_LIB -O2 -pthread -std=gnu++11 -o CMakeFiles/caffe2.dir/operators/cross_entropy_op.cc.o -c ....../pytorch/caffe2/operators/cross_entropy_op.cc
In file included from /usr/local/include/unsupported/Eigen/CXX11/Tensor:104:0,
from ....../pytorch/caffe2/operators/conv_op_eigen.cc:10:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorIndexList.h:220:10: error: ‘array_size’ is not a class template
struct array_size<IndexTuple<T, O...> > {
^~~~~~~~~~
In file included from /usr/local/include/unsupported/Eigen/CXX11/Tensor:114:0,
from ....../pytorch/caffe2/operators/conv_op_eigen.cc:10:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:203:60: error: ‘scalar_logistic_op’ is not a member of ‘Eigen::internal’
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_logistic_op<Scalar>, const Derived>
^~~~~~~~~~~~~~~~~~
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:203:60: note: suggested alternative: ‘scalar_log10_op’
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_logistic_op<Scalar>, const Derived>
^~~~~~~~~~~~~~~~~~
scalar_log10_op
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:203:60: error: ‘scalar_logistic_op’ is not a member of ‘Eigen::internal’
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:203:60: note: suggested alternative: ‘scalar_log10_op’
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_logistic_op<Scalar>, const Derived>
^~~~~~~~~~~~~~~~~~
scalar_log10_op
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:203:85: error: wrong number of template arguments (1, should be 2)
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_logistic_op<Scalar>, const Derived>
^
In file included from /usr/local/include/unsupported/Eigen/CXX11/Tensor:96:0,
from ....../pytorch/caffe2/operators/conv_op_eigen.cc:10:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorForwardDeclarations.h:69:52: note: provided for ‘template<class UnaryOp, class XprType> class Eigen::TensorCwiseUnaryOp’
template<typename UnaryOp, typename XprType> class TensorCwiseUnaryOp;
^~~~~~~~~~~~~~~~~~
In file included from /usr/local/include/unsupported/Eigen/CXX11/Tensor:114:0,
from ....../pytorch/caffe2/operators/conv_op_eigen.cc:10:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:203:86: error: expected unqualified-id before ‘,’ token
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_logistic_op<Scalar>, const Derived>
^
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:215:60: error: ‘scalar_expm1_op’ is not a member of ‘Eigen::internal’
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_expm1_op<Scalar>, const Derived>
^~~~~~~~~~~~~~~
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:215:60: note: suggested alternative: ‘scalar_exp_op’
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_expm1_op<Scalar>, const Derived>
^~~~~~~~~~~~~~~
scalar_exp_op
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:215:60: error: ‘scalar_expm1_op’ is not a member of ‘Eigen::internal’
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:215:60: note: suggested alternative: ‘scalar_exp_op’
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_expm1_op<Scalar>, const Derived>
^~~~~~~~~~~~~~~
scalar_exp_op
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:215:82: error: wrong number of template arguments (1, should be 2)
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_expm1_op<Scalar>, const Derived>
^
In file included from /usr/local/include/unsupported/Eigen/CXX11/Tensor:96:0,
from ....../pytorch/caffe2/operators/conv_op_eigen.cc:10:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorForwardDeclarations.h:69:52: note: provided for ‘template<class UnaryOp, class XprType> class Eigen::TensorCwiseUnaryOp’
template<typename UnaryOp, typename XprType> class TensorCwiseUnaryOp;
^~~~~~~~~~~~~~~~~~
In file included from /usr/local/include/unsupported/Eigen/CXX11/Tensor:114:0,
from ....../pytorch/caffe2/operators/conv_op_eigen.cc:10:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorBase.h:215:83: error: expected unqualified-id before ‘,’ token
EIGEN_STRONG_INLINE const TensorCwiseUnaryOp<internal::scalar_expm1_op<Scalar>, const Derived>
^
In file included from /usr/include/c++/7/cassert:44:0,
from ....../pytorch/aten/src/ATen/core/SmallVector.h:27,
from ....../pytorch/aten/src/ATen/core/ArrayRef.h:20,
from ....../pytorch/caffe2/core/context.h:17,
from ....../pytorch/caffe2/operators/conv_op_eigen.cc:6:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorMap.h: In member function ‘const Scalar& Eigen::TensorMap<PlainObjectType, Options_, MakePointer_>::operator()(Eigen::TensorMap<PlainObjectType, Options_, MakePointer_>::Index, Eigen::TensorMap<PlainObjectType, Options_, MakePointer_>::Index, IndexTypes ...) const’:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorMap.h:153:7: error: ‘all’ is not a member of ‘Eigen::internal’
eigen_assert(internal::all((Eigen::NumTraits<Index>::highest() >= otherIndices)...));
^
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorMap.h: In member function ‘Eigen::TensorMap<PlainObjectType, Options_, MakePointer_>::Scalar& Eigen::TensorMap<PlainObjectType, Options_, MakePointer_>::operator()(Eigen::TensorMap<PlainObjectType, Options_, MakePointer_>::Index, Eigen::TensorMap<PlainObjectType, Options_, MakePointer_>::Index, IndexTypes ...)’:
/usr/local/include/unsupported/Eigen/CXX11/src/Tensor/TensorMap.h:241:8: error: ‘all’ is not a member of ‘Eigen::internal’
eigen_assert(internal::all((Eigen::NumTraits<Index>::highest() >= otherIndices)...));
^
caffe2/CMakeFiles/caffe2.dir/build.make:4195: recipe for target 'caffe2/CMakeFiles/caffe2.dir/operators/conv_op_eigen.cc.o' failed
make[2]: *** [caffe2/CMakeFiles/caffe2.dir/operators/conv_op_eigen.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
make[2]: Leaving directory '....../pytorch/build'
CMakeFiles/Makefile2:1727: recipe for target 'caffe2/CMakeFiles/caffe2.dir/all' failed
make[1]: *** [caffe2/CMakeFiles/caffe2.dir/all] Error 2
make[1]: Leaving directory '....../pytorch/build'
Makefile:143: recipe for target 'all' failed
make: *** [all] Error 2
Failed to run 'bash ../tools/build_pytorch_libs.sh --use-cuda --use-nnpack --use-mkldnn caffe2 libshm gloo c10d THD'
Some more ERRORs:
[ 42%] Building CXX object caffe2/CMakeFiles/caffe2.dir//aten/src/TH/THStorageFunctions.cpp.o
cd …/pytorch/build/caffe2 && /usr/bin/c++ -DCPUINFO_SUPPORTED_PLATFORM=1 -DGFLAGS_IS_A_DLL=0 -DGOOGLE_GLOG_DLL_DECL=“” -DGOOGLE_GLOG_DLL_DECL_FOR_UNITTESTS=“” -DNNP_CONVOLUTION_ONLY=0 -DNNP_INFERENCE_ONLY=0 -DONNX_NAMESPACE=onnx_c2 -DOPENCV_TRAITS_ENABLE_DEPRECATED -DTH_BLAS_MKL -DUSE_C11_ATOMICS=1 -D_FILE_OFFSET_BITS=64 -Dcaffe2_EXPORTS -I…/pytorch/aten/src -I…/pytorch/build -I…/pytorch -isystem …/pytorch/build/third_party/gloo -isystem …/pytorch/cmake/…/third_party/gloo -I…/pytorch/third_party/protobuf/src -isystem /opt/intel/mkl/include -isystem /usr/local/include -isystem /usr/local/include/opencv4 -isystem /usr/include/eigen3 -isystem /usr/include/python3.6m -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent -isystem /usr/lib/x86_64-linux-gnu/openmpi/include/openmpi/opal/mca/event/libevent2022/libevent/include -isystem /usr/lib/x86_64-linux-gnu/openmpi/include -isystem …/pytorch/cmake/…/third_party/cub -I…/pytorch/build/caffe2/contrib/aten -I…/pytorch/third_party/onnx -I…/pytorch/build/third_party/onnx -I…/pytorch/build/caffe2/aten/src/TH -I…/pytorch/aten/src/TH -I…/pytorch/build/caffe2/aten/src -I…/pytorch/build/aten/src -I…/pytorch/aten/src/THNN -I…/pytorch/aten/src/THCUNN -I…/pytorch/aten/…/third_party/catch/single_include -I…/pytorch/build/caffe2/aten/src/ATen -I…/pytorch/aten/src/ATen/… -I…/pytorch/caffe2/core/nomnigraph/include -isystem …/pytorch/build/include -I/usr/local/cuda/include -I…/pytorch/third_party/NNPACK/include -I…/pytorch/third_party/pthreadpool/include -I…/pytorch/third_party/cpuinfo/include -msse3 -msse4.1 -msse4.2 --std=c++11 -I~/.local/lib/python3.6/site-packages/numpy/core/include -fvisibility-inlines-hidden -fopenmp -O2 -fPIC -Wno-narrowing -Wall -Wextra -Wno-missing-field-initializers -Wno-type-limits -Wno-array-bounds -Wno-unknown-pragmas -Wno-sign-compare -Wno-unused-parameter -Wno-unused-variable -Wno-unused-function -Wno-unused-result -Wno-strict-overflow -Wno-strict-aliasing -Wno-error=deprecated-declarations -Wno-stringop-overflow -Wno-error=pedantic -Wno-error=redundant-decls -Wno-error=old-style-cast -faligned-new -Wno-unused-but-set-variable -Wno-maybe-uninitialized -Wno-stringop-overflow -DHAVE_AVX_CPU_DEFINITION -DHAVE_AVX2_CPU_DEFINITION -O3 -fPIC -DCUDA_HAS_FP16=1 -DHAVE_GCC_GET_CPUID -DUSE_SSE2 -DUSE_SSE4_1 -DUSE_SSE4_2 -DUSE_SSE3 -DUSE_AVX -DUSE_AVX2 -fvisibility=hidden -DCAFFE2_BUILD_MAIN_LIB -O2 -pthread -std=gnu++11 -o CMakeFiles/caffe2.dir//aten/src/TH/THStorageFunctions.cpp.o -c …/pytorch/aten/src/TH/THStorageFunctions.cpp
…/pytorch/aten/src/TH/THAllocator.cpp:397:1: error: prototype for ‘THMapAllocator::THMapAllocator(WithFd, const char*, int, int)’ does not match any in class ‘THMapAllocator’
THMapAllocator::THMapAllocator(WithFd, const char filename, int fd, int flags) {
^~~~~~~~~~~~~~
In file included from …/pytorch/aten/src/TH/THAllocator.cpp:1:0:
…/pytorch/aten/src/TH/THAllocator.h:41:3: error: candidates are: THMapAllocator::THMapAllocator(THMapAllocator&&)
THMapAllocator(THMapAllocator&&) = delete;
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.h:39:3: error: THMapAllocator::THMapAllocator(const THMapAllocator&)
THMapAllocator(const THMapAllocator&) = delete;
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.h:38:3: error: THMapAllocator::THMapAllocator(WithFd, const char, int, int, size_t)
THMapAllocator(WithFd, const char filename, int fd, int flags, size_t size);
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:393:1: error: THMapAllocator::THMapAllocator(const char, int, size_t)
THMapAllocator::THMapAllocator(const char filename, int flags, size_t size) {
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:401:52: error: destructors may not have parameters
THMapAllocator::~THMapAllocator(THMapAllocator ctx) {}
^
…/pytorch/aten/src/TH/THAllocator.cpp:401:1: error: redefinition of ‘THMapAllocator::~THMapAllocator()’
THMapAllocator::~THMapAllocator(THMapAllocator* ctx) {}
^~~~~~~~~~~~~~
In file included from …/pytorch/aten/src/TH/THAllocator.cpp:1:0:
…/pytorch/aten/src/TH/THAllocator.h:67:11: note: ‘virtual THMapAllocator::~THMapAllocator()’ previously defined here
virtual ~THMapAllocator() { close(); }
^
…/pytorch/aten/src/TH/THAllocator.cpp: In constructor ‘THRefcountedMapAllocator::THRefcountedMapAllocator(const char*, int, size_t)’:
…/pytorch/aten/src/TH/THAllocator.cpp:508:96: error: no matching function for call to ‘THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck()’
THRefcountedMapAllocator::THRefcountedMapAllocator(const char filename, int flags, size_t size) {
^
…/pytorch/aten/src/TH/THAllocator.cpp:506:1: note: candidate: THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(int)
THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(int flags) {}
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:506:1: note: candidate expects 1 argument, 0 provided
In file included from …/pytorch/aten/src/TH/THAllocator.cpp:1:0:
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate: constexpr THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(const THRefcountedMapAllocatorArgCheck&)
struct AT_API THRefcountedMapAllocatorArgCheck {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate expects 1 argument, 0 provided
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate: constexpr THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(THRefcountedMapAllocatorArgCheck&&)
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate expects 1 argument, 0 provided
…/pytorch/aten/src/TH/THAllocator.cpp:508:96: error: no matching function for call to ‘THMapAllocator::THMapAllocator()’
THRefcountedMapAllocator::THRefcountedMapAllocator(const char filename, int flags, size_t size) {
^
In file included from …/pytorch/aten/src/TH/THAllocator.cpp:1:0:
…/pytorch/aten/src/TH/THAllocator.h:38:3: note: candidate: THMapAllocator::THMapAllocator(WithFd, const char, int, int, size_t)
THMapAllocator(WithFd, const char filename, int fd, int flags, size_t size);
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.h:38:3: note: candidate expects 5 arguments, 0 provided
…/pytorch/aten/src/TH/THAllocator.cpp:393:1: note: candidate: THMapAllocator::THMapAllocator(const char, int, size_t)
THMapAllocator::THMapAllocator(const char filename, int flags, size_t size) {
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:393:1: note: candidate expects 3 arguments, 0 provided
…/pytorch/aten/src/TH/THAllocator.cpp: In constructor ‘THRefcountedMapAllocator::THRefcountedMapAllocator(WithFd, const char, int, int, size_t)’:
…/pytorch/aten/src/TH/THAllocator.cpp:512:112: error: no matching function for call to ‘THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck()’
THRefcountedMapAllocator::THRefcountedMapAllocator(WithFd, const char filename, int fd, int flags, size_t size) {
^
…/pytorch/aten/src/TH/THAllocator.cpp:506:1: note: candidate: THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(int)
THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(int flags) {}
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:506:1: note: candidate expects 1 argument, 0 provided
In file included from …/pytorch/aten/src/TH/THAllocator.cpp:1:0:
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate: constexpr THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(const THRefcountedMapAllocatorArgCheck&)
struct AT_API THRefcountedMapAllocatorArgCheck {
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate expects 1 argument, 0 provided
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate: constexpr THRefcountedMapAllocatorArgCheck::THRefcountedMapAllocatorArgCheck(THRefcountedMapAllocatorArgCheck&&)
…/pytorch/aten/src/TH/THAllocator.h:85:15: note: candidate expects 1 argument, 0 provided
…/pytorch/aten/src/TH/THAllocator.cpp:512:112: error: no matching function for call to ‘THMapAllocator::THMapAllocator()’
THRefcountedMapAllocator::THRefcountedMapAllocator(WithFd, const char filename, int fd, int flags, size_t size) {
^
In file included from …/pytorch/aten/src/TH/THAllocator.cpp:1:0:
…/pytorch/aten/src/TH/THAllocator.h:38:3: note: candidate: THMapAllocator::THMapAllocator(WithFd, const char, int, int, size_t)
THMapAllocator(WithFd, const char filename, int fd, int flags, size_t size);
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.h:38:3: note: candidate expects 5 arguments, 0 provided
…/pytorch/aten/src/TH/THAllocator.cpp:393:1: note: candidate: THMapAllocator::THMapAllocator(const char, int, size_t)
THMapAllocator::THMapAllocator(const char filename, int flags, size_t size) {
^~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:393:1: note: candidate expects 3 arguments, 0 provided
…/pytorch/aten/src/TH/THAllocator.cpp: At global scope:
…/pytorch/aten/src/TH/THAllocator.cpp:517:1: error: redefinition of ‘THRefcountedMapAllocator::~THRefcountedMapAllocator()’
THRefcountedMapAllocator::~THRefcountedMapAllocator() {}
^~~~~~~~~~~~~~~~~~~~~~~~
In file included from …/pytorch/aten/src/TH/THAllocator.cpp:1:0:
…/pytorch/aten/src/TH/THAllocator.h:104:11: note: ‘virtual THRefcountedMapAllocator::~THRefcountedMapAllocator()’ previously defined here
virtual ~THRefcountedMapAllocator() { close(); }
^
…/pytorch/aten/src/TH/THAllocator.cpp: In static member function ‘static at::DataPtr THRefcountedMapAllocator::makeDataPtr(const char, int, size_t, size_t)’:
…/pytorch/aten/src/TH/THAllocator.cpp:551:61: error: ‘TH_ALLOC_ALIGNMENT’ was not declared in this scope
if (actual_size_out) actual_size_out = context->size() - TH_ALLOC_ALIGNMENT;
^~~~~~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:551:61: note: suggested alternative: ‘__LOCK_ALIGNMENT’
if (actual_size_out) actual_size_out = context->size() - TH_ALLOC_ALIGNMENT;
^~~~~~~~~~~~~~~~~~
__LOCK_ALIGNMENT
…/pytorch/aten/src/TH/THAllocator.cpp: In static member function ‘static at::DataPtr THRefcountedMapAllocator::makeDataPtr(WithFd, const char, int, int, size_t, size_t)’:
…/pytorch/aten/src/TH/THAllocator.cpp:557:61: error: ‘TH_ALLOC_ALIGNMENT’ was not declared in this scope
if (actual_size_out) actual_size_out = context->size() - TH_ALLOC_ALIGNMENT;
^~~~~~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:557:61: note: suggested alternative: ‘__LOCK_ALIGNMENT’
if (actual_size_out) actual_size_out = context->size() - TH_ALLOC_ALIGNMENT;
^~~~~~~~~~~~~~~~~~
__LOCK_ALIGNMENT
…/pytorch/aten/src/TH/THAllocator.cpp: In member function ‘virtual void THRefcountedMapAllocator::data() const’:
…/pytorch/aten/src/TH/THAllocator.cpp:562:61: error: ‘TH_ALLOC_ALIGNMENT’ was not declared in this scope
return static_cast<void>(static_cast<char>(base_ptr_) + TH_ALLOC_ALIGNMENT);
^~~~~~~~~~~~~~~~~~
…/pytorch/aten/src/TH/THAllocator.cpp:562:61: note: suggested alternative: ‘LOCK_ALIGNMENT’
return static_cast<void*>(static_cast<char*>(base_ptr) + TH_ALLOC_ALIGNMENT);
^~~~~~~~~~~~~~~~~~
LOCK_ALIGNMENT
caffe2/CMakeFiles/caffe2.dir/build.make:2961: recipe for target 'caffe2/CMakeFiles/caffe2.dir//aten/src/TH/THAllocator.cpp.o’ failed
make[2]: *** [caffe2/CMakeFiles/caffe2.dir/_/aten/src/TH/THAllocator.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs…
make[2]: Leaving directory ‘…/pytorch/build’
CMakeFiles/Makefile2:1795: recipe for target ‘caffe2/CMakeFiles/caffe2.dir/all’ failed
make[1]: *** [caffe2/CMakeFiles/caffe2.dir/all] Error 2
make[1]: Leaving directory ‘…/pytorch/build’
Makefile:143: recipe for target ‘all’ failed
make: *** [all] Error 2
Actually, I’ve already installed and I prefer installing some of those 3rd party libraries manually before building/installing PyTorch. However, after installing those 3rd party libraries, I failed to install PyTorch and it’s always complaining “some library has already been installed” …
Successfully installed pytorch, but ALWAYS failed to import torch…
>>> import torch
Segmentation fault (core dumped)
hmmmmmm .what can I say?
Hi,
Can you use gdb to check where this segfault comes from?
Have NO idea how to use gdb to debug Python code at all…
import caffe2.python.core
WARNING:root:This caffe2 python run does not have GPU support. Will run in CPU only mode.
Segmentation fault (core dumped)
You can run gdb python. Then enter r when gdb is started. You then get into your python shell. From there you can call import torch. When it segfaults and control returns to gdb, enter bt to get the stacktrace.
Also make sure to uninstall all pytorch install (pip uninstall torch multiple times) before installing. Having partial installations lingering can cause problems when an old library is used instead of the new one.
Dear Sir,
Should I install magma-cuda with CUDA 10 when my environment is miniconda, and build from source ?
Thank you for your help.
$ gdb python
GNU gdb (Ubuntu 8.1-0ubuntu3) 8.1.0.20180409-git
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python...(no debugging symbols found)...done.
(gdb) r                                                                                                                                                                                                        
Starting program: /usr/bin/python                                                                                                                                                                              
[Thread debugging using libthread_db enabled]                                                                                                                                                                  
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".                                                                                                                                     
Python 3.6.6 (default, Sep 12 2018, 18:26:19)                                                                                                                                                                  
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux                                                                                                                                           
Type "help", "copyright", "credits" or "license" for more information.                                                                                                                                         
>>> import torch                                                                                                                                                                                               
[New Thread 0x7ffff2b03700 (LWP 23763)]                                                                                                                                                                        
[New Thread 0x7ffff2302700 (LWP 23764)]                                                                                                                                                                        
[New Thread 0x7fffefb01700 (LWP 23765)]                                                                                                                                                                        
[New Thread 0x7fffeb300700 (LWP 23766)]                                                                                                                                                                        
[New Thread 0x7fffe8aff700 (LWP 23767)]                                                                                                                                                                        
[New Thread 0x7fffe62fe700 (LWP 23768)]                                                                                                                                                                        
[New Thread 0x7fffe3afd700 (LWP 23769)]
[Thread 0x7fffe3afd700 (LWP 23769) exited]
[Thread 0x7fffe62fe700 (LWP 23768) exited]
[Thread 0x7fffe8aff700 (LWP 23767) exited]
[Thread 0x7fffeb300700 (LWP 23766) exited]
[Thread 0x7fffefb01700 (LWP 23765) exited]
[Thread 0x7ffff2302700 (LWP 23764) exited]
[Thread 0x7ffff2b03700 (LWP 23763) exited]
Thread 1 "python" received signal SIGSEGV, Segmentation fault.
0x00007fffdb9f2710 in ?? ()
(gdb) bt
#0  0x00007fffdb9f2710 in ?? ()
#1  0x00007fffdb7b7bce in ?? () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
#2  0x00007fffdb7b8c17 in pkgInitSystem(Configuration&, pkgSystem*&) () from /usr/lib/x86_64-linux-gnu/libapt-pkg.so.5.0
#3  0x00007fffdba522af in ?? () from /usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so
#4  0x00000000005117df in ?? ()
#5  0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#6  0x00000000004f3338 in ?? ()
#7  0x00000000004ff8d8 in ?? ()
#8  0x0000000000556c12 in PyCFunction_Call ()
#9  0x00000000004fa8e2 in _PyEval_EvalFrameDefault ()
#10 0x00000000004f3338 in ?? ()
#11 0x0000000000510fb0 in ?? ()
#12 0x00000000005119bd in ?? ()
#13 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#14 0x0000000000510c78 in ?? ()
#15 0x00000000005119bd in ?? ()
#16 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#17 0x0000000000510c78 in ?? ()
#18 0x00000000005119bd in ?? ()
#19 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#20 0x0000000000510c78 in ?? ()
#21 0x00000000005119bd in ?? ()
#22 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#23 0x0000000000510375 in _PyFunction_FastCallDict ()
#24 0x000000000059d591 in _PyObject_FastCallDict ()
#25 0x000000000059d8fe in _PyObject_CallMethodIdObjArgs ()
#26 0x00000000004dab6d in PyImport_ImportModuleLevelObject ()
#27 0x00000000004f70ba in _PyEval_EvalFrameDefault ()
#28 0x00000000004f3338 in ?? ()
#29 0x00000000004ff8d8 in ?? ()
#30 0x0000000000556c12 in PyCFunction_Call ()
#31 0x00000000004fa8e2 in _PyEval_EvalFrameDefault ()
#32 0x00000000004f3338 in ?? ()
#33 0x0000000000510fb0 in ?? ()
#34 0x00000000005119bd in ?? ()
#35 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#36 0x0000000000510c78 in ?? ()
#37 0x00000000005119bd in ?? ()
#38 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#39 0x0000000000510c78 in ?? ()
#40 0x00000000005119bd in ?? ()
#41 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#42 0x0000000000510c78 in ?? ()
#43 0x00000000005119bd in ?? ()
#44 0x00000000004f5277 in _PyEval_EvalFrameDefault ()
#45 0x0000000000510375 in _PyFunction_FastCallDict ()
#46 0x000000000059d591 in _PyObject_FastCallDict ()
#47 0x000000000059d8fe in _PyObject_CallMethodIdObjArgs ()
#48 0x00000000004dab6d in PyImport_ImportModuleLevelObject ()
#49 0x00000000004f70ba in _PyEval_EvalFrameDefault ()
#50 0x00000000004f3338 in ?? ()
#51 0x00000000004ff8d8 in ?? ()
#52 0x0000000000556c12 in PyCFunction_Call ()
---Type <return> to continue, or q <return> to quit---
Weird, it segfault in the apt_pkg.
What does import apt and import apt_pkg do?
1 Like
$ python
Python 3.6.6 (default, Sep 12 2018, 18:26:19)  
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import apt
>>> import apt_pkg
>>> apt.__file__
'/usr/lib/python3/dist-packages/apt/__init__.py'
>>> apt.__version__
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
AttributeError: module 'apt' has no attribute '__version__'
>>> apt_pkg.__file__
'/usr/lib/python3/dist-packages/apt_pkg.cpython-36m-x86_64-linux-gnu.so'
>>> apt_pkg.__version__
Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
AttributeError: module 'apt_pkg' has no attribute '__version__'
>>>
Does it have something to do with GCC version?
The GCC on my laptop is 
$ gcc --version
gcc (Ubuntu 7.3.0-27ubuntu1~18.04) 7.3.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
However, it seems the current Python is built from GCC 8.0.1 20180414 ??
Maybe? I have to admit I’m not sure here. Do you have multiple gcc on your laptop? Or different environments with different versions of these libraries?
I build PyTorch from source and I never installed Magma-cuda
I have cuda 10 installed but installed pytorch through cuda 9.2 using pip. It seems to be using gpu.
Will this cause any hardware or software issues to my system in the long run?
It should not, but you might not get the best performances as some code is not fully compiled and tuned for your cuda version.