I have a issue with implement my own
Consider the following example:
class Exp(torch.autograd.Function): @staticmethod def forward(ctx, i): result = i.exp() ctx.save_for_backward(result) print(result) # print1 here for debug return result @staticmethod def backward(ctx, grad_output): result, = ctx.saved_tensors return grad_output * result
Now; I have found the following behaviours
from torch.autograd import Variable x = Variable(torch.Tensor([3.]),requires_grad=True) e.apply(x) tensor([20.0855]) # here comes from the print1 Out: tensor([20.0855], grad_fn=<ExpBackward>) # here is the return value
What I have found is that in
forward() we seems loosing the track of gradient information. The print shows that
result is not
grad_fn. However, the actually returned value is.
Can anyone help me to reasoning about it ?
In my true implementation, due to the above issue, I seems to loss some variable (that is during the
backward(), some variable has zero gradient and hence no update )