When i test my model, do I have to use model.eval() even though I am using 'with torch.no_grad() ?
These two have different goals:
model.eval()will notify all your layers that you are in eval mode, that way, batchnorm or dropout layers will work in eval model instead of training mode.
torch.no_grad()impacts the autograd engine and deactivate it. It will reduce memory usage and speed up computations but you won’t be able to backprop (which you don’t want in an eval script).
Thank you very much for your quick and clear explanation.
Hey, this implies I should definitely do
"model.eval" while validating.
And, if memory and speed are not constraints;
"torch.no_grad()" can be ignored. Right?
Ahh with torch.no_grad() you’ll have much higher speeds and can use larger validation batch sizes so it’s useful if not recommended
Why do something that takes 5x more memory (the 5 here is for the example, not actual number in practice) and is slow, if you can just add one extra line to avoid it?
torch.no_grad() is actually the recommended way to perform validation !
yeah alright I meant to write “if not compulsory”
Thank you for the explanation!