Unable to access CUDA using torch on aarch64 architecture

C++
1

I have a custom yocto build with aarch64 architecture running Linux. I have installed pytorch using Python. When I check torch.cuda.is_available(), it always returns false even though I have a CUDA capable GPU (nvidia A6000).
Also, I am unable to install libtorch on it, I get an error saying “wrong file format”.

2

The PyTorch ARM binaries do not support CUDA at the moment as they are built for Mac.
You could use the PyTorch NGC container which is built for x86 and ARM and supports CUDA.

3

Oh okay. Does installing pytorch from source support cuda? Is there any container for libtorch as well?

4

Yes, our source builds support CUDA as mentioned before. libtorch is also built and used from the Python API. To build a standalone libtorch application you can use

cmake -DCMAKE_PREFIX_PATH=python -c 'import torch;print(torch.utils.cmake_prefix_path)'

which would find the right libraries.

5

Okay, I will try this. Thank you for your help!

6

Hey @ptrblck, I am building pytorch from source on arm64 but I do not have a conda environment, however I have a python virtual environment. Do you guys have any documentation for a non-conda installation?

Instructions at GitHub - pytorch/pytorch: Tensors and Dynamic neural networks in Python with strong GPU acceleration need conda.

7

You don’t need conda and can install magma-cuda from source (if needed) e.g. via cloning the repository and running:

cmake -H. -Bbuild -DUSE_FORTRAN=OFF -DGPU_TARGET="All" -DBUILD_SHARED_LIBS=OFF -DCMAKE_CXX_FLAGS="-fPIC" -DCMAKE_C_FLAGS="-fPIC" -DCUDA_NVCC_FLAGS="-Xfatbin;-compress-all;-DHAVE_CUBLAS;-std=c++11;--threads=0;" -GNinja
cmake --build build --target install

and could also install MKL from PyPI although it’s irrelevant for ARM.

8

I got this error

CMake Error at cmake/Dependencies.cmake:1454 (add_dependencies):
  Cannot add target-level dependencies to non-existent target "gloo_cuda".

  The add_dependencies works for top-level logical targets created by the
  add_executable, add_library, or add_custom_target commands.  If you want to
  add file-level dependencies see the DEPENDS option of the add_custom_target
  and add_custom_command commands.
Call Stack (most recent call first):
  CMakeLists.txt:728 (include)

I have also attached the CMakeError.log