Explanation autograd, chain rule and Variables

autograd
1

I’m trying to understand how autograd works. Suppose I have the following code:

x = Variable(torch.arange(0, 4).double(), requires_grad=True)

y = Variable(x * 2, requires_grad=True)

z = y * x

z.backward(torch.ones(y.size()).double())
y.sum().backward()

print(x.grad, 2*x)

It seems to be working correctly. The problem is that if I change the 3rd line to z = Variable(y*x, requires_grad=True), then it does not work anymore. What’s the difference? Thank you.

2

Is your error something along the lines of?

RuntimeError: Trying to backward through the graph a second time ...

?

That’s because the graph gets deleted after it’s been evaluated via .backward(). If you want to backpropate through the graph a second time. You can set the argument "retain_graph=True in your first backward call.

Also, please note that Variable has been deprecated since PyTorch 0.4 (multiple versions ago) and it’s better to use the torch.tensor constructor. E.g., the following should work:

x = torch.tensor(torch.arange(0, 4).double(), requires_grad=True)

y = x.detach().requires_grad_(True)*2

z = y * x

z.backward(torch.ones(y.size()).double(), retain_graph=True)

y.sum().backward()

print(x.grad, 2*x)
1 Like