Running simple codes in the USING THE PYTORCH C++ FRONTEND tutorial end with linker error

Hello,

I am trying to run the simple code segments given in the tutorial USING THE PYTORCH C++ FRONTEND . I installed Pytorch with CUDA in a conda environment. Then as given in the installation guide I used

-DCMAKE_PREFIX_PATH=`python -c 'import torch;print(torch.utils.cmake_prefix_path)'`

as a cmake option in Clion.

I get a cmake warning as follows

CMake Warning at xxx/xxx/lib/python3.8/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
static library kineto_LIBRARY-NOTFOUND not found.

When I run the first example which print a tensor in Clion, at the linking stage it fails with the following output

/bin/ld: warning: libmkl_intel_lp64.so, needed by /xxx/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so, not found (try using -rpath or -rpath-link)
/bin/ld: warning: libmkl_gnu_thread.so, needed by /xxx/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so, not found (try using -rpath or -rpath-link)
/bin/ld: warning: libmkl_core.so, needed by /xxx/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so, not found (try using -rpath or -rpath-link)
/xxx/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so: undefined reference to `vslDeleteStream'
/xxx/xxx/lib/python3.8/site-packages/torch/lib/libtorch_cpu.so: undefined reference to `vdLn'

Then I installed MKL and the above problems were solved and I can compile and run the first example which simply prints a tensor.

When I try the second example where struct Net is created and the params are printed, I get the following error.

[ 50%] Building CXX object CMakeFiles/untitled.dir/main.cpp.o
[100%] Linking CXX executable untitled
CMakeFiles/untitled.dir/main.cpp.o: In function `Net::Net(long, long)':
main.cpp:(.text._ZN3NetC2Ell[_ZN3NetC2Ell]+0x8b): undefined reference to `torch::nn::Module::register_parameter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, at::Tensor, bool)'
main.cpp:(.text._ZN3NetC2Ell[_ZN3NetC2Ell]+0x1a0): undefined reference to `torch::nn::Module::register_parameter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, at::Tensor, bool)'
CMakeFiles/untitled.dir/main.cpp.o: In function `torch::nn::Module::_forward_num_required_args()':
main.cpp:(.text._ZN5torch2nn6Module26_forward_num_required_argsEv[_ZN5torch2nn6Module26_forward_num_required_argsEv]+0x4f): undefined reference to `c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
CMakeFiles/untitled.dir/main.cpp.o: In function `torch::nn::Module::_forward_populate_default_args(std::vector<torch::nn::AnyValue, std::allocator<torch::nn::AnyValue> >&&)':
main.cpp:(.text._ZN5torch2nn6Module30_forward_populate_default_argsEOSt6vectorINS0_8AnyValueESaIS3_EE[_ZN5torch2nn6Module30_forward_populate_default_argsEOSt6vectorINS0_8AnyValueESaIS3_EE]+0x4f): undefined reference to `c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
clang-12: error: linker command failed with exit code 1 (use -v to see invocation)
gmake[3]: *** [untitled] Error 1
gmake[2]: *** [CMakeFiles/untitled.dir/all] Error 2
gmake[1]: *** [CMakeFiles/untitled.dir/rule] Error 2
gmake: *** [untitled] Error 2

I am stuck at this without any clue how to proceed. I am using clang 12.0 and cmake 3.17.3.

With regard to @tom reply on “Python packages are built with a ancient-compatibility string ABI”, I thought to add some more content.

I have tried both cxx11 and pre cxx11 libtorch prebuilt packages and the results were as follows

using clang with pre cxx11

Scanning dependencies of target untitled
[ 50%] Building CXX object CMakeFiles/untitled.dir/main.cpp.o
[100%] Linking CXX executable untitled
CMakeFiles/untitled.dir/main.cpp.o: In function `main':
/xxx/xxx/CLionProjects/untitled/main.cpp:33: undefined reference to `torch::data::datasets::MNIST::MNIST(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, torch::data::datasets::MNIST::Mode)'
CMakeFiles/untitled.dir/main.cpp.o: In function `Net':
/xxx/xxx/CLionProjects/untitled/main.cpp:16: undefined reference to `torch::nn::Module::register_parameter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, at::Tensor, bool)'
/xxx/xxx/CLionProjects/untitled/main.cpp:17: undefined reference to `torch::nn::Module::register_parameter(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, at::Tensor, bool)'
CMakeFiles/untitled.dir/main.cpp.o: In function `torch::nn::Module::_forward_num_required_args()':
/xxx/xxx/libtorch/include/torch/csrc/api/include/torch/nn/module.h:526: undefined reference to `c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
CMakeFiles/untitled.dir/main.cpp.o: In function `torch::nn::Module::_forward_populate_default_args(std::vector<torch::nn::AnyValue, std::allocator<torch::nn::AnyValue> >&&)':
/xxx/xxx/libtorch/include/torch/csrc/api/include/torch/nn/module.h:534: undefined reference to `c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
CMakeFiles/untitled.dir/main.cpp.o: In function `c10::Device::validate()':
/xxx/xxx/libtorch/include/c10/core/Device.h:110: undefined reference to `c10::detail::torchInternalAssertFail(char const*, char const*, unsigned int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
/xxx/xxx/libtorch/include/c10/core/Device.h:114: undefined reference to `c10::detail::torchInternalAssertFail(char const*, char const*, unsigned int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'
CMakeFiles/untitled.dir/main.cpp.o: In function `Error':
/xxx/xxx/libtorch/include/c10/util/Exception.h:175: undefined reference to `c10::Error::Error(c10::SourceLocation, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >)'
CMakeFiles/untitled.dir/main.cpp.o: In function `torch::detail::TensorDataContainer::fill_tensor(at::Tensor&) const':
/xxx/xxx/libtorch/include/torch/csrc/api/include/torch/detail/TensorDataContainer.h:277: undefined reference to `c10::detail::torchInternalAssertFail(char const*, char const*, unsigned int, char const*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)'

Here if I use cxx11 build with clang I get the following error

/applics/jetbrains/clion/2020_2_1/bin/cmake/linux/bin/cmake --build /xxx/xxx/CLionProjects/untitled/cmake-build-release-system --target untitled -- -j 28
[ 50%] Linking CXX executable untitled
/xxx/xxx/libtorch11/lib/libtorch_cpu.so: undefined reference to `lgammaf@GLIBC_2.23'
/xxx/xxx/libtorch11/lib/libtorch_cpu.so: undefined reference to `lgamma@GLIBC_2.23'
collect2: error: ld returned 1 exit status
gmake[3]: *** [untitled] Error 1
gmake[2]: *** [CMakeFiles/untitled.dir/all] Error 2
gmake[1]: *** [CMakeFiles/untitled.dir/rule] Error 2
gmake: *** [untitled] Error 2

I don’t know if there are any clang vs. gcc compatibility issues here. One thing to know is that the PyTorch Python packages are built with a ancient-compatibility string ABI.
(If torch.compiled_with_cxx11_abi returns False.) I would not know how to select that mode in clang.

Best regards

Thomas

Hello,

Thank you very much for your reply.

I have tried both pre cxx11 and cxx11 libtorch packages without success as in the updated OP. I think you are suggesting something similar to setting D_GLIBCXX_USE_CXX11_ABI=0 in gcc. I still looking for how to do that in clang.

But you pointed the prefix path to your libftorch installations rather than the Python one, right?