Could NOT find CUDA (missing: CUDA_CUDART_LIBRARY) (found version "12.5")

1

With CMake version 3.30.2 and CUDA 12.5.1, I get the following error when trying to build PyTorch 2.4.0:

-- Could NOT find CUDA (missing: CUDA_CUDART_LIBRARY) (found version "12.5")
CMake Warning at cmake/public/cuda.cmake:31 (message):
  Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
  or a Caffe2 dependent library, the next warning / error will give you more
  info.
Call Stack (most recent call first):
  cmake/Dependencies.cmake:43 (include)
  CMakeLists.txt:853 (include)


CMake Warning at cmake/Dependencies.cmake:74 (message):
  Not compiling with CUDA.  Suppress this warning with -DUSE_CUDA=OFF.

Perhaps this is a CMake issue? I didn’t have the problem with CMake 3.29.4, and I’ve since upgraded to 3.30.2.

2

Might be as cmake==3.30.2 was just released a week ago. Am I understanding your claim correctly that cmake==3.29.4 works for you?

3

I downgraded CMake to 3.29.4, but I’m still getting the same issue:
disabling CUDA because USE_CUDA is set false
Yet I have USE_CUDA explicitly set: export USE_CUDA=1 and -DUSE_CUDA=ON.

4

It does say: Found CUDA with FP16 support, compiling with torch.cuda.HalfTensor
But then it says disabling CUDA because NOT USE_CUDA is set.

5

I haven’t seen this issue myself and can find a few references pointing to a wrong docker run command not allowing GPU access. I don’t know if this is related to your issue or not.
You could check if you are able to source build any other CUDA application (e.g. the CUDA samples) first.

6

Yes. I can CUDA example (which doesn’t use CMake). It seems to be a CMake issue.

7

Hi, I mentioned this on the issue and this is a noob guess, but could try recompiling after cleaning the build? e.g., add the line right python3 setup.py clean before python3 setup.py install?

8

Where exactly? This is how the SlackBuild script does it:

cd build
  unshare -n cmake \
    -G Ninja \
    -DCMAKE_C_FLAGS:STRING="$SLKCFLAGS" \
    -DCMAKE_CXX_FLAGS:STRING="$SLKCFLAGS" \
    -DCMAKE_CXX_STANDARD=17 \
    -DCMAKE_INSTALL_PREFIX="/usr" \
    -DLIBSHM_INSTALL_LIB_SUBDIR="lib$LIBDIRSUFFIX" \
    -DTORCH_INSTALL_LIB_DIR="lib$LIBDIRSUFFIX" \
    -DPYTHON_EXECUTABLE=$(which python3) \
    -DBUILD_CUSTOM_PROTOBUF=OFF \
    -DBUILD_TEST=OFF \
    -DUSE_FFMPEG=ON \
    -DUSE_GOLD_LINKER=ON \
    -DUSE_OPENCL=ON \
    -DUSE_OPENCV=ON \
    -DUSE_VULKAN=ON \
    -DCMAKE_BUILD_TYPE=Release ..
  "${NINJA:=ninja}"
  DESTDIR=tmpxxx $NINJA install/strip

  mkdir -p $PKG/usr/{share,lib$LIBDIRSUFFIX}
  mv tmpxxx/usr/bin $PKG/usr
  mv tmpxxx/usr/include $PKG/usr
  mv tmpxxx/usr/share/cmake $PKG/usr/share
  mv tmpxxx/usr/lib$LIBDIRSUFFIX/*.so $PKG/usr/lib$LIBDIRSUFFIX
cd ..
python3 setup.py install --root=$PKG

Adding python3 setup.py clean right before the last line above didn’t change anything. I still get:

Found CUDA with FP16 support, compiling with torch.cuda.HalfTensor
and
disabling CUDA because NOT USE_CUDA is set

9

Hi, did you solve the issue? I also met the same problem.

10

Not yet.

I should also mention that I fixed the

Building wheel torch-2.4.0a0+gitUnknown
-- Building version 2.4.0a0+gitUnknown
Could not find any of CMakeLists.txt, Makefile, setup.py, LICENSE, LICENSE.md, LICENSE.txt in /tmp/SBo/pytorch-v2.4.0/third_party/QNNPACK
Did you run 'git submodule update --init --recursive'?

by running sed -i '/"QNNPACK"/d' setup.py. I’m wondering if that triggered the CUDA issue I’m having?