Loss.backward() -> RuntimeError: cuDNN error: CUDNN_STATUS_EXECUTION_FAILED

22

Could you post a code snippet to reproduce this issue, please?

23

I upgrade to torch 1.5 and it works now.

1 Like
24

Encountered the same issue with my backward pass:
loss.backard()

My environment

PyTorch 1.4.0
TorchVision 0.5.0
CUDA 10.2
CUDNN 7.6.03

Adding this line helped but slowed down the training

torch.backends.cudnn.enabled = False

Upgrading to Torch 1.5.0 and Torchvision to 0.6.0 solved the issue
pip install torch==1.5.0+cu101 torchvision==0.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html

1 Like
25

Upgrading Torch and Torchvision also seemed to have solved it for me!
Here’s the conda command:

conda install pytorch==1.5.0 torchvision==0.6.0 -c pytorch

And with that, we don’t have to disable the cuddn backend.