Call backward() twice does not incur errors

x=torch.rand(2,2,requires_grad=True)
y=x+2
# y=x*2
z=torch.sum(y)
zz=z*2
z.backward()
zz.backward()

call backward() twice does not incur errors. However, if i use y=x*2 rather than y=x+2, the error occurs, why?

x=torch.rand(2,2,requires_grad=True)
#y=x+2
y=x*2
z=torch.sum(y)
zz=z*2
z.backward()
zz.backward()

x * 2 saves a value (2) for backward, and the first time backward happens, this value is cleared
this causes the second backward to error unless retain_graph=True

x + 2 does not have the same problem because it doesn’t need to save any values in order to compute the backward

1 Like