What do nn.Module’s train(True) and train(False) do internally?
training attribute of Module and subModules to True or False.
As far as I know, when I’m using
Dropout layers, it randomly sets some output values to zero in training mode, but not in evaluation mode.
Does it affect any layers other than dropout?
Yes, for example BatchNorm
Other than dropout and BatchNorm? Would be good if there’s a list of things in the framework where the training (True/False) that can make a difference.
I am still confuse what is the different of
model.eval()? DO I need to do both model.train(False) and
model.eval() for every validation and test step?
Simply I have a model which deploy dropout layers. I am doing like this:
- In training phase ->
- In validation phase ->
- In testing phase ->
However I found that my model is not working properly. I must remove
model.eval() to get the best result. Later I tried in validation phase ->
model.train(False) followed by
model.eval(). However again the result is not good I must remove
model.eval() in testing phase. Could anyone explain this phenomena?, what should I do in validation and testing phase?, is it enough if we only use
model.eval() have the same effect, as the latter calls in fact the former function as seen in this line of code.
If you see any differences, I guess this might be some random effects in your training.
I see thank you for your explanation. I have that problem because I applied
nn.DropOut2d which a different way of computation in training and testing.