On model.train() and model.eval()

Hi all,

I have a basic Pytorch question on how to use model.train() and .eval(). Let’s say I have a code like that:

For epochs: 
   For training mini-batches:
      model.train()
      .
      .
      .
   For validation data:
      model.eval()
      .

Is this correct?
I am feeling that using .train() and .eval() like above causes problem because at some points both of them are true.

Thank you!

Yes it is correct. For example if you have dropout in your model, model.eval() will disable it. You have to explicitly change the model state back to training again by calling model.train() To enable dropout during training loop.

2 Likes