RuntimeError: derivative for _cudnn_rnn_backward is not implemented

1

I have no idea to solve this problem.
I impletemented torch.autograd.grad to get the gradient penalty loss, but this error just show again and again , did there any one has same problem?

Thanks,
Peter

1 Like
2

Unfortunately, double backward for cudnn rnn is not supported, there’s an upstream issue tracking this https://github.com/pytorch/pytorch/issues/5261. Recommended way of doing what you want to do is writing custom rnns with torch script https://pytorch.org/blog/optimizing-cuda-rnn-with-torchscript/