What does nn.Module's train(True)/train(False) do

What do nn.Module’s train(True) and train(False) do internally?

1 Like

Set the training attribute of Module and subModules to True or False.
As far as I know, when I’m using Dropout layers, it randomly sets some output values to zero in training mode, but not in evaluation mode.

Does it affect any layers other than dropout?

Yes, for example BatchNorm

Other than dropout and BatchNorm? Would be good if there’s a list of things in the framework where the training (True/False) that can make a difference.

4 Likes

I am still confuse what is the different of model.train(False) and model.eval()? DO I need to do both model.train(False) andmodel.eval() for every validation and test step?
Simply I have a model which deploy dropout layers. I am doing like this:

  1. In training phase -> model.train(True)
  2. In validation phase -> model.train(False)
  3. In testing phase -> model.eval()

However I found that my model is not working properly. I must remove model.eval() to get the best result. Later I tried in validation phase -> model.train(False) followed by model.eval(). However again the result is not good I must remove model.eval() in testing phase. Could anyone explain this phenomena?, what should I do in validation and testing phase?, is it enough if we only usemodel.train(False)?

2 Likes

model.train(False) and model.eval() have the same effect, as the latter calls in fact the former function as seen in this line of code.

If you see any differences, I guess this might be some random effects in your training.

13 Likes

I see thank you for your explanation. I have that problem because I applied nn.DropOut2d which a different way of computation in training and testing.