Using grad to calculate derivative of two outputs w.r.t. the same input

[InnovArul (Arul)]

Make sure to zero the gradients (if x.grad: x.grad.zero_()) before calculating new y.

Some more info:

Why do we need to set the gradients manually to zero in pytorch?

y.backward() doesn’t just assign the value of y’(x) to x.grad (say y depends on x). It actually adds y’(x) to the current value of x.grad (think it as x.grad += true_gradient). In the following example, y.backward() is called 5 times, so the final value of x.grad will be 5*cos(0)=5. import torch from torch.autograd import Variable x = Variable(torch.Tensor([[0]]), requires_grad=True) for t in range(5): y = x.sin() y.backward() print(x.grad) # shows 5 Calling x.grad.data.zero_…