'model.eval()' vs 'with torch.no_grad()'

(Dong Wook Kim) #1

When i test my model, do I have to use model.eval() even though I am using 'with torch.no_grad() ?

(Alban D) #2


These two have different goals:

  • model.eval() will notify all your layers that you are in eval mode, that way, batchnorm or dropout layers will work in eval model instead of training mode.
  • torch.no_grad() impacts the autograd engine and deactivate it. It will reduce memory usage and speed up computations but you won’t be able to backprop (which you don’t want in an eval script).

(Dong Wook Kim) #3

Thank you very much for your quick and clear explanation.

(Jayakrishna Rudra) #4

Hey, this implies I should definitely do "model.eval" while validating.

And, if memory and speed are not constraints; "torch.no_grad()" can be ignored. Right?

(Naman Jain) #5

Ahh with torch.no_grad() you’ll have much higher speeds and can use larger validation batch sizes so it’s useful if not recommended

(Alban D) #6

Why do something that takes 5x more memory (the 5 here is for the example, not actual number in practice) and is slow, if you can just add one extra line to avoid it?
@Naman-ntc using torch.no_grad() is actually the recommended way to perform validation !

(Naman Jain) #7

yeah alright I meant to write “if not compulsory” :sweat_smile:

(Jayakrishna Rudra) #8

Thank you for the explanation!