Could I simply use `affine=False` to bypass a batchnorm layer

Hi ,
Here I have a simple question: when I assign affine=False in nn.BatchNorm2d, I noticed that the weight and bias terms vanish in the state_dict. So does this mean the gamma and beta of the BN equation are fixed to be 1 and 0, which means to fix BN in the training?
Do I need to do other things in the eval phase?

The running estimates will still be calculated and updated during training while only applied in evaluation. Therefore you should still call model.eval() before starting the evaluation.
Let me know, if I’ve misunderstood your question.

Thanks, what if I need not only disable the update of gamma and beta, but also the running mean and var, since affine=False only helps to disable the training of gamma and beta ?

If you don’t want to update the running estimates, you can call .eval() on the BatchNorm layer. This will make sure to apply the estimated mean and std on the current batch without updating them.

2 Likes