What does nn.Module's train(True)/train(False) do

1

What do nn.Module’s train(True) and train(False) do internally?

1 Like
2

Set the training attribute of Module and subModules to True or False.
As far as I know, when I’m using Dropout layers, it randomly sets some output values to zero in training mode, but not in evaluation mode.

1 Like
3

Does it affect any layers other than dropout?

4

Yes, for example BatchNorm

1 Like
5

Other than dropout and BatchNorm? Would be good if there’s a list of things in the framework where the training (True/False) that can make a difference.

5 Likes
6

I am still confuse what is the different of model.train(False) and model.eval()? DO I need to do both model.train(False) andmodel.eval() for every validation and test step?
Simply I have a model which deploy dropout layers. I am doing like this:

  1. In training phase -> model.train(True)
  2. In validation phase -> model.train(False)
  3. In testing phase -> model.eval()

However I found that my model is not working properly. I must remove model.eval() to get the best result. Later I tried in validation phase -> model.train(False) followed by model.eval(). However again the result is not good I must remove model.eval() in testing phase. Could anyone explain this phenomena?, what should I do in validation and testing phase?, is it enough if we only usemodel.train(False)?

3 Likes
7

model.train(False) and model.eval() have the same effect, as the latter calls in fact the former function as seen in this line of code.

If you see any differences, I guess this might be some random effects in your training.

17 Likes
8

I see thank you for your explanation. I have that problem because I applied nn.DropOut2d which a different way of computation in training and testing.