I’m new to PyTorch and I’m taking a liking to it, but a small doubt I have is whether or not gradients are calculated when the forward method is run on a model is used for prediction.
If so (gradients are calculated) does running loss.backward() just populate those gradients? or is it like running the loss.backward() method does both of them together (calculate gradients and populate them too)
Thanks in Advance!
No, forward doesn’t calculate gradients.
I don’t know what you mean by populate, do you mean propagate? loss.backward() does the back propagation, and compute all the gradients for the network weights.
By populate, I mean allocating the value of the gradients in the tensor. Thanks for clarifying my issue, I now understand that model.forward() only does the forward propagation and no gradient calculation is involved.