Why do I use net.zero_grad() and then go backward on loss. The gradient before and after execution is the same
print(out)
target = Variable(t.arange(0,10)).float()
criterion = nn.MSELoss()
loss = criterion(out, target)
print(loss)
#梯度清0
net.zero_grad()
print("反向传播之前的conv1.bias的梯度")
print(net.conv1.bias.backward)
loss.backward(retain_graph=True)
print("反向传播之后的梯度")
print(net.conv1.bias.backward)
反向传播之前的conv1.bias的梯度
<bound method Tensor.backward of Parameter containing:
tensor([ 0.0900, 0.1849, -0.1985, -0.0836, -0.1590, 0.0689],
requires_grad=True)>
反向传播之后的梯度
<bound method Tensor.backward of Parameter containing:
tensor([ 0.0900, 0.1849, -0.1985, -0.0836, -0.1590, 0.0689],
requires_grad=True)>