Trying to understand the meaning of model.train() and model.eval()

[anis016 (Sayed Anisul Hoque)]

maybe these should clear you out.

Model.train() and model.eval() vs model and model.eval()

Yes, they are the same. By default all the modules are initialized to train mode (self.training = True). Also be aware that some layers have different behavior during train/and evaluation (like BatchNorm, Dropout) so setting it matters. Also as a rule of thumb for programming in general, try to explicitly state your intent and set model.train() and model.eval() when necessary.

By default all the modules are initialized to train mode (self.training = True). Also be aware that some layers have different behavior during train/and evaluation (like BatchNorm, Dropout) so setting it matters.

Do need to use model.eval() when I test?

Sure, Dropout works as a regularization for preventing overfitting during training. It randomly zeros the elements of inputs in Dropout layer on forward call. It should be disabled during testing since you may want to use full model (no element is masked)

Dropout works as a regularization for preventing overfitting during training.
It randomly zeros the elements of inputs in Dropout layer on forward call.
It should be disabled during testing ( model.eval() ) since you may want to use full model (no element is masked)