In validation process we set
model.eval() to tell BN and Dropouts that we are in val mode.
But what’s the difference if we use train mode?
In the docs it said:
During training, this layer keeps a running estimate of its computed mean and variance. The running sum is kept with a default momentum of 0.1. During evaluation, this running mean/variance is used for normalization.
what if I set
requires_grad=False for the whole model, which means that the running mean/var will not be modified.
In that case, is training mode same as validation mode?
Thanks in advance.
upd: This question came to me when using vgg16 as perceptual loss. Do we need to set vgg16 model to eval mode?