No such file or directory #include <cublas_v2.h>

I’m trying to write a C++ version of maskrcnn based on libtorch, then when compiling my project, I got an error like this. I have seen this kind of error when compiling Caffe. However, that solution can not be used when talking about libtorch. Is there anyone have cues? thank you in advance.

-- Found Threads: TRUE  
-- Found CUDA: /usr/local/cuda-10.1 (found version "10.1") 
-- Caffe2: CUDA detected: 10.1
-- Caffe2: CUDA nvcc is: /usr/local/cuda-10.1/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda-10.1
-- Caffe2: Header version is: 10.1
-- Found CUDNN: /usr/lib/x86_64-linux-gnu/libcudnn.so  
-- Found cuDNN: v7.6.4  (include: /usr/include, library: /usr/lib/x86_64-linux-gnu/libcudnn.so)
-- Autodetected CUDA architecture(s):  5.2 5.2
-- Added CUDA NVCC flags for: -gencode;arch=compute_52,code=sm_52
-- Found torch: /home/tengfeihan/maskrcnn_benchmark.cpp/lib/libtorch/lib/libtorch.so  
-- Found OpenCV: /usr/local (found version "3.4.2") 
CUDA FOUND
-- The CUDA compiler identification is NVIDIA 10.1.243
-- Check for working CUDA compiler: /usr/local/cuda-10.1/bin/nvcc
-- Check for working CUDA compiler: /usr/local/cuda-10.1/bin/nvcc -- works
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Found PythonInterp: /home/tengfeihan/anaconda3/bin/python (found version "3.7.4") 
-- Found GTest: /home/tengfeihan/maskrcnn_benchmark.cpp/lib/libtorch/lib/libgtest.a  
-- Configuring done
-- Generating done
-- Build files have been written to: /home/tengfeihan/maskrcnn_benchmark.cpp/build
(base) ┌─[tengfeihan@tengfeihan-X10SRA-F] - [~/maskrcnn_benchmark.cpp/build] - [Sat Mar 14, 19:06]
└─[$] <git:(master*)> make
Scanning dependencies of target layers
[  1%] Building CUDA object source/rcnn/layers/CMakeFiles/layers.dir/cuda/ROIAlign_cuda.cu.o
In file included from /home/tengfeihan/maskrcnn_benchmark.cpp/source/rcnn/layers/cuda/ROIAlign_cuda.cu:3:0:
/home/tengfeihan/maskrcnn_benchmark.cpp/lib/libtorch/include/ATen/cuda/CUDAContext.h:7:10: fatal error: cublas_v2.h: No such file or directory
 #include <cublas_v2.h>
          ^~~~~~~~~~~~~
compilation terminated.
source/rcnn/layers/CMakeFiles/layers.dir/build.make:62: recipe for target 'source/rcnn/layers/CMakeFiles/layers.dir/cuda/ROIAlign_cuda.cu.o' failed
make[2]: *** [source/rcnn/layers/CMakeFiles/layers.dir/cuda/ROIAlign_cuda.cu.o] Error 1
CMakeFiles/Makefile2:337: recipe for target 'source/rcnn/layers/CMakeFiles/layers.dir/all' failed
make[1]: *** [source/rcnn/layers/CMakeFiles/layers.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2

hey, I encountered the exact same problem, have you find a solution yet? Thanks!

Which CUDA version are you using to build PyTorch from source?

10.1, I encountered this issue while building detectron2.

Could you search for this file in your CUDA directory?
I’m seeing the file in:

find /usr/local/ -name cublas_v2.h
/usr/local/cuda-11.0/targets/x86_64-linux/include/cublas_v2.h
2 Likes

cublas_v2.h is not in my CUDA_HOME /usr/local/cuda-10.1/, but I find it at /usr/include/

Hey,
I am facing the same issue, my cublas_v2.h file is located at /usr/include/.
How did you solve the issue? Did you change the path to a different one?

Hi,

I had a similar issue trying to build apex with cuda10.2 and pytorch 1.7 nightly. You can use symbolic linking like the following to resolve the issue:
ln -s /usr/include/cublas_v2.h /usr/local/cuda-10.2/targets/x86_64-linux/include/cublas_v2.h . Please use appropriate cuda version and target for the target link. Cheers!

I have the same issue here. I tried ln -s /usr/include/cublas_v2.h /usr/local/cuda-10.2/targets/x86_64-linux/include/cublas_v2.h but I get the output:


ln: failed to create symbolic link '/usr/local/cuda-10.2/targets/x86_64-linux/include/cublas_v2.h': File exists

and my issue is still there. Do you have any solution?

2 Likes

I am compiling apex with Pytorch 1.6. For my case it’s the other way around: cublas_v2.h is located at /usr/local/cuda-10.1/targets/x86_64-linux/include/cublas_v2.h but compiler expects it to be at /usr/include/cublas_v2.h.

I’ve got the very same problem for the very same repository (NSVF). I tried installing different cuda versions, reinstalling them, linking,… nothing works.

Edit: Ok, this worked for me. Make sure cublas_v2.h is located somewhere on your system, you can use what @ptrblck suggested, i.e find /usr/local/ -name cublas_v2.h If it doesn’t find anything you have to reinstall cuda - I recommend this tutorial Installing CUDA 10.1 on Ubuntu 20.04 | by Stephen Gregory | Medium

Do find /usr/local/ -name cublas_v2.h once again to make sure cublas_v2.h is there, if you followed the aforementioned tutorial cublas_v2.h will be in cuda-10-2 folder. No worries, we will symlink it and everything else that is necessary. Just do the following:

sudo ln -s /usr/local/cuda-10.2/targets/x86_64-linux/include/cublas_v2.h /usr/local/cuda-10.1/targets/x86_64-linux/include/cublas_v2.h

and

sudo ln -s /usr/local/cuda-10.2/targets/x86_64-linux/include/cublas_api.h /usr/local/cuda-10.1/targets/x86_64-linux/include/cublas_api.h

You should be good to go. In my case I got:

$ pip install --editable ./
Obtaining file:///home/milos/Projects/repositories/NSVF
Installing collected packages: fairnr
  Running setup.py develop for fairnr
Successfully installed fairnr

Let me know if it works :slight_smile:

1 Like

This failed because the file order is reversed. It should be ln -s path/to/src path/to/dst so you want

ln -s /usr/local/cuda-10.2/targets/x86_64-linux/include/cublas_v2.h /usr/include/cublas_v2.h

then do the same for cublas_api.h.

ln -s /usr/local/cuda-10.2/targets/x86_64-linux/include/cublas_api.h /usr/include/cublas_api.h

This worked for me.

3 Likes

@thanhmvu tips are very helpful. Now I encounter the errors for the links in the libraries for cublas_api.h

/usr/include/cublas_api.h:76:10: fatal error: cuda_bf16.h: 

I guess your cublas installations is newer than the CUDA toolkit, which is why the cuda_bf16.h cannot be found.

Thanks! I solved the problem by cublas for 10.1 is missing - #18 by phillip3m - CUDA Setup and Installation - NVIDIA Developer Forums (You might want to change the order of symbolic)