.eval() mode performance degradation strangely

autograd
1

I know there is lots of difference between .eval() and .train() in Dropout layer and BatchNorm layer.
So I write two simple method to locate the reason.
def dropout_disable(m):
print(m)
if type(m)==nn.Dropout:
m.eval()

def bn_disable(m):
print(m)
if type(m)==nn.BatchNorm1d:
m.eval()

Following is my experiment.
net.train() test，f1_score=0.77
net.apply(dropout_disable),test，f1_score=0.78
net.apply(bn_disable)，test，f1_score=0.77
net.eval(),test ,f1_score=0.77 ,It is same with the above result.
net.train()，test，f1_score=77%（the result is different each time as there is randomness in dropout layer）
net.eval(),test,f1_score=0.70 .This time the result is degradation largely.
I don’t know why this result is differ from the above result 0.77???
Is there other difference between .train() and .eval()?
or it is a bug of pytorch???

2

I delete all dropout layer in my network,the performance becomes great.
but I don’t know why.
why the disable of dropout layer would destroy my model?

3

Maybe it’s related to the BatchNorm layers.
Since you switched your model to .train() and used the test dataset on it, the running stats will be upgraded.
After this run, you switched back to .eval() and got a bad result.

However, this does not explain your second observation, that removing all Dropout layers yields a higher accuracy.