I’m trying to replicate Pedro Cuenca’s code on accelerated diffusers using pytorch 2.0 + CUDA 11.8 on Ubuntu 22.04. However, I’m getting the following error:
Could not load library libcudnn_cnn_infer.so.8. Error: libnvrtc.so: cannot open shared object file: No such file or directory
Here is the relevant code:
from diffusers import StableDiffusionPipeline
from diffusers.models.cross_attention import AttnProcessor2_0
print('diffusers version', diffusers.__version__)
print('torch version', torch.__version__)
print('cudnn version', torch.backends.cudnn.version())
pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2")
prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images
I’m on diffusers version 0.14.0, torch version 2.0.0+cu118 and cudnn version 8700.
I’m using a conda env. Tried installing pytorch using both pip and conda. The required library (libcudnn_cnn_infer.so.8) does seem to be properly installed within the virtual env. I also tried this before running the script, but I still get the same error:
Thanks @ptrblck! I fixed it by installing cuda-11-8 from Nvidia’s developer repo:
sudo apt install cuda-11-8
I was wrongly understanding that the missing library was libcudnn_cnn_infer.so.8, which is correctly installed by the pytorch pip/conda packages, but it is in fact libnvrtc.so, which is not. After installing it I can now run the example code.
If you think this is a packaging bug I can submit an issue at github.
Yes, the issue in your environment is raised by a missing libnvrtc.so, which is shipped in python3.8/site-packages/torch/lib/libnvrtc-672ee683.so.11.2, but I guess libcudnn misses this dependency assuming cuDNN’s runtime fusion API is used.
Let me check my workflow with LD_DEBUG to see where my environment is loading this library from and I can follow up with an issue and fix if needed in the pytorch/builder repository.
>>> print('torch version', torch.__version__)
torch version 2.0.0+cu117
>>> print('cudnn version', torch.backends.cudnn.version())
cudnn version 8500
I have cuda toolkit 12.1 installed currently
Thank You it’s working with the nightly build, but I have this error now
File "/home/prabhat/.local/share/virtualenvs/GroundingSam-IWDf2mQY/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 388, in _check_cuda_version
raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
The detected CUDA version (12.1) mismatches the version that was used to compile
PyTorch (11.8). Please make sure to use the same CUDA versions.
As I’ve also mentioned in the linked topic your locally installed CUDA toolkit will be used if you build PyTorch from source or a custom CUDA extension as is apparently the case causing your error.
Pure PyTorch code does not need a locally installed CUDA toolkit.