LoadLibrary error while using libtorch with ROS2

I am trying to build and run a ROS2 package which uses libtorch.

In CMakeLists.txt I wrote

set(Torch_DIR ~/libtorch/share/cmake/Torch/)
find_package(Torch)

  ament_auto_add_library(my_lib SHARED  
    src/node.cpp)
  
  target_link_libraries(my_lib 
    ${TORCH_LIBRARIES}
  )

Package can be built without errors, but there is an error while running it:

terminate called after throwing an instance of 'class_loader::LibraryLoadException'
  what():  Could not load library LoadLibrary error: libtorch_cuda.so: cannot open shared object file: No such file or directory, at /tmp/binarydeb/ros-foxy-rcutils-1.1.5/src/shared_library.c:84

If I add path to libtorch to LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/libtorch/lib/
there is another error while running package:

terminate called after throwing an instance of 'class_loader::LibraryLoadException'
  what():  Could not load library LoadLibrary error: /home/arseniy.marin@nami.local/libtorch/lib/libgomp-75eea7e8.so.1: version `GOMP_4.5' not found (required by /lib/x86_64-linux-gnu/libpcl_common.so.1.10), at /tmp/binarydeb/ros-foxy-rcutils-1.1.5/src/shared_library.c:84

Libtorch 1.8.1
ROS2 Foxy
Ubuntu 20.0.4