Could not load library libcudnn_cnn_infer.so.8

1

I’m trying to replicate Pedro Cuenca’s code on accelerated diffusers using pytorch 2.0 + CUDA 11.8 on Ubuntu 22.04. However, I’m getting the following error:

Could not load library libcudnn_cnn_infer.so.8. Error: libnvrtc.so: cannot open shared object file: No such file or directory

Here is the relevant code:

from diffusers import StableDiffusionPipeline
from diffusers.models.cross_attention import AttnProcessor2_0
import diffusers
import torch
print('diffusers version', diffusers.__version__)
print('torch version', torch.__version__)
print('cudnn version', torch.backends.cudnn.version())

pipe = StableDiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-2")
pipe.to("cuda")
pipe.unet.set_attn_processor(AttnProcessor2_0())
prompt = "a photo of an astronaut riding a horse on mars"
image = pipe(prompt).images[0]

I’m on diffusers version 0.14.0, torch version 2.0.0+cu118 and cudnn version 8700.

I’m using a conda env. Tried installing pytorch using both pip and conda. The required library (libcudnn_cnn_infer.so.8) does seem to be properly installed within the virtual env. I also tried this before running the script, but I still get the same error:

export LD_LIBRARY_PATH=~/miniconda3/envs/stable-diffusion/lib/python3.10/site-packages/torch/lib/
2

I cannot reproduce the issue using 2.0.0+cu118 with a 3090 and get:

...
>>> pipe.unet.set_attn_processor(AttnProcessor2_0())
>>> prompt = "a photo of an astronaut riding a horse on mars"
>>> image = pipe(prompt).images[0]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 50/50 [00:16<00:00,  3.00it/s]
>>> image
<PIL.Image.Image image mode=RGB size=768x768 at 0x7F1E306F7FA0>

while I assume the last posted line of code raises the error for you?
Just to double check, these are my versions:

>>> print('diffusers version', diffusers.__version__)
diffusers version 0.14.0
>>> print('torch version', torch.__version__)
torch version 2.0.0+cu118
>>> print('cudnn version', torch.backends.cudnn.version())
cudnn version 8700

and I needed to install some dependencies such as transformers.

3

Thanks @ptrblck! I fixed it by installing cuda-11-8 from Nvidia’s developer repo:

sudo apt install cuda-11-8

I was wrongly understanding that the missing library was libcudnn_cnn_infer.so.8, which is correctly installed by the pytorch pip/conda packages, but it is in fact libnvrtc.so, which is not. After installing it I can now run the example code.

If you think this is a packaging bug I can submit an issue at github.

4

Yes, the issue in your environment is raised by a missing libnvrtc.so, which is shipped in python3.8/site-packages/torch/lib/libnvrtc-672ee683.so.11.2, but I guess libcudnn misses this dependency assuming cuDNN’s runtime fusion API is used.
Let me check my workflow with LD_DEBUG to see where my environment is loading this library from and I can follow up with an issue and fix if needed in the pytorch/builder repository.

5

You’re right, the library is there. Symlinking libnvrtc-672ee683.so.11.2 to libnvrtc.so (and apt removing cuda-11-8) worked:

cd yourenv/lib/python3.10/site-packages/torch/lib
ln -s libnvrtc-672ee683.so.11.2 libnvrtc.so
6

We are tracking the issue here.

1 Like