How does one use LibTorch (C++) with the Arch64 build of PyTorch for the Nvidia Jetson?

I’ve successfully installed PyTorch v1.10.0 on my Nvidia Jetson but I would like to use the C++ / Libtorch portion of the install.
The “wheels” specifically for the Nvidia Jetsons are here

On my Jetson (AGX Xavier) I believe Libtorch is installed in this path
~/.local/lib/python3.6/site-packages/torch/lib

But I don’t know how to make use of anything in this path. I would like to use it to run a Libtorch script in C++. This is a list of all the files in that path.
libbackend_with_compiler.so, libcaffe2_detectron_ops_gpu.so, libcaffe2_observers.so, libtorchbind_test.so, libtorch_global_deps.so, libc10_cuda.so, libcaffe2_module_test_dynamic.so, libjitbackend_test.so, libtorch_cpu.so libtorch_python.so, libc10.so, libcaffe2_nvrtc.so, libshm.so, libtorch_cuda.so, libtorch.so

Could I specifically use the libtorch.so to compile my LibTorch script somehow?

Below: I’m simply trying to compile the example script for Libtorch from the official site
I am following the instructions on the page exactly and for /absolute/path/to/libtorch I put the path I mentioned above, ~/.local/lib/python3.6/site-packages/torch/lib.

I need to build the Libtorch script with cmake and I’m getting the error…


$ cmake -DCMAKE_PREFIX_PATH=/home/jet/.local/lib/python3.6/site-packages/torch/lib ..
-- The C compiler identification is GNU 7.5.0
-- The CXX compiler identification is GNU 7.5.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:4 (find_package):
  By not providing "FindTorch.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Torch", but
  CMake did not find one.


  Could not find a package configuration file provided by "Torch" with any of
  the following names:

    TorchConfig.cmake
    torch-config.cmake

  Add the installation prefix of "Torch" to CMAKE_PREFIX_PATH or set
  "Torch_DIR" to a directory containing one of the above files.  If "Torch"
  provides a separate development package or SDK, be sure it has been
  installed.


-- Configuring incomplete, errors occurred!

This is my CMakeLists.txt


cmake_minimum_required(VERSION 3.0 FATAL_ERROR)
project(example-app)

find_package(Torch REQUIRED)
set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${TORCH_CXX_FLAGS}")

add_executable(example-app example-app.cpp)
target_link_libraries(example-app "${TORCH_LIBRARIES}")
set_property(TARGET example-app PROPERTY CXX_STANDARD 14)

# The following code block is suggested to be used on Windows.
# According to https://github.com/pytorch/pytorch/issues/25457,
# the DLLs need to be copied to avoid memory errors.
if (MSVC)
  file(GLOB TORCH_DLLS "${TORCH_INSTALL_PREFIX}/lib/*.dll")
  add_custom_command(TARGET example-app
                     POST_BUILD
                     COMMAND ${CMAKE_COMMAND} -E copy_if_different
                     ${TORCH_DLLS}
                     $<TARGET_FILE_DIR:example-app>)
endif (MSVC)

I haven’t checked it, but maybe try to use the built-in cmake_prefix_path instead of providing it manually via:

-DCMAKE_PREFIX_PATH=`python -c 'import torch;print(torch.utils.cmake_prefix_path)'`

I tried to build that way. I get errors and no build. I am trying to build the official LibTorch example.

I wrote ‘python3’ instead of just python because I am able to import torch when I open python3.

The cmake command

cmake -DCMAKE_PREFIX_PATH=`python3 -c 'import torch;print(torch.utils.cmake_prefix_path)'` ..
-- The C compiler identification is Clang 6.0.0
-- The CXX compiler identification is Clang 6.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/lib/ccache/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/lib/ccache/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found CUDA: /usr/local/cuda-10.2 (found version "10.2")
-- Caffe2: CUDA detected: 10.2
-- Caffe2: CUDA nvcc is: /usr/local/cuda-10.2/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-10.2
-- Caffe2: Header version is: 10.2
-- Found CUDNN: /usr/lib/aarch64-linux-gnu/libcudnn.so
-- Found cuDNN: v8.2.1  (include: /usr/include, library: /usr/lib/aarch64-linux-gnu/libcudnn.so)
-- /usr/local/cuda-10.2/lib64/libnvrtc.so shorthash is 7d272a04
CMake Warning at /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Caffe2/public/utils.cmake:365 (message):
  In the future we will require one to explicitly pass TORCH_CUDA_ARCH_LIST
  to cmake instead of implicitly setting it as an env variable.  This will
  become a FATAL_ERROR in future version of pytorch.
Call Stack (most recent call first):
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Caffe2/public/cuda.cmake:437 (torch_cuda_get_nvcc_gencode_flag)
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:88 (include)
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  CMakeLists.txt:4 (find_package)
-- Added CUDA NVCC flags for: -gencode;arch=compute_53,code=sm_53;-gencode;arch=compute_62,code=sm_62;-gencode;arch=compute_72,code=sm_72
CMake Warning at /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
  static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
  /home/jet/.local/lib/python3.6/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:127 (append_torchlib_if_found)
  CMakeLists.txt:4 (find_package)
-- Found Torch: /home/jet/.local/lib/python3.6/site-packages/torch/lib/libtorch.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/jet/example-app/build

The cmake build command

$cmake --build . --config Release
[ 50%] Building CXX object CMakeFiles/example-app.dir/example-app.cpp.o
[100%] Linking CXX executable example-app
/usr/bin/ld: cannot find -lCUDA_cublas_LIBRARY-NOTFOUND
clang: error: linker command failed with exit code 1 (use -v to see invocation)
CMakeFiles/example-app.dir/build.make:110: recipe for target 'example-app' failed
make[2]: *** [example-app] Error 1
CMakeFiles/Makefile2:82: recipe for target 'CMakeFiles/example-app.dir/all' failed
make[1]: *** [CMakeFiles/example-app.dir/all] Error 2
Makefile:90: recipe for target 'all' failed
make: *** [all] Error 2

Use this reference https://github.com/pytorch/pytorch/blob/master/docs/libtorch.rst#building-libtorch-using-cmake

There is a way to use Libtorch on the nvidia jetson platform.

The Pytorch official has not provided an aarch64 Libtorch package for downloading, It only provides an x86-64 Libtorch package。

In the Nvidia Jetson platform, Nvidia provides the PyTorch python package, which contains Libtorch in it.
So, We can use Libtorch by installing the Pytorch Python package.

Firstly, you should download PyTorch for Jetson on below link

https://forums.developer.nvidia.com/t/pytorch-for-jetson/72048

Secondly, install the “whl” file following this guide

[Installing PyTorch for Jetson Platform - NVIDIA Docs]

After you finished the above steps. The LibTorch has been installed on your device.

How to Use?

where is LibTorch installed?

Commonly, it installs at ~/.local/lib/python3.6/site-packages/torch/
In this dir, you can see bin, lib, and include dir, It is organized just like other C++ libraries.

How to use LibTorch with CMake?

for example, add these two lines to your CMakeLists.txt

// set PyTorch path
set(Torch_DIR /home/nx/.local/lib/python3.6/site-packages/torch/share/cmake/Torch)
// use find_package to use libtorch
find_package(Torch REQUIRED)