Gradient for input when evaluate

Hi,

I want to use a pretrained model to supervise another model, which need to set the batch normalization into eval mode. However, with lstm in my pretrained model, it arises follow message:

Traceback (most recent call last):
  File "main.py", line 153, in val
    cost.backward()
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/autograd/variable.py", line 145, in backward
    self._execution_engine.run_backward((self,), (gradient,), retain_variables)
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/autograd/function.py", line 208, in _do_backward
    result = super(NestedIOFunction, self)._do_backward(gradients, retain_variables)
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/autograd/function.py", line 216, in backward
    result = self.backward_extended(*nested_gradients)
  File "/home/jrmei/.local/lib/python2.7/site-packages/torch/nn/_functions/rnn.py", line 199, in backward_extended
    self._reserve_clone = self.reserve.clone()
AttributeError: 'CudnnRNN' object has no attribute 'reserve'

Do you have any suggestions? Thanks very much.

your pretrained model does not have reserve looks like it.
Is there a way I can reproduce your problem? Do you have a small 30 line script that will give the same problem?

I see the bug. I’ll fix it today.

import torch
import torch.nn as nn
from torch.autograd import Variable

rnn = nn.Sequential(nn.LSTM(10, 10, 2, bidirectional=True))
rnn = rnn.cuda()
rnn.eval()

i = torch.Tensor(2, 1, 10).cuda()
i = Variable(i, requires_grad=True)

o, _ = rnn(i)
o = o.mean()
o.backward(torch.ones(1).cuda())
1 Like

May I ask when will be the next update?
My work are really interrupted by some bugs.

It will be today. I will be building binaries late today.

Thanks very much. (~~~~~~~~~~~~~~~~~~)