Dear all,
I am using some model with BatchNorm2d with default eps=1e-5. But dumping the model trained parameters, it turns out that the running variances as a mean value of the order of 1e-6.
So the denominator of the BN sqrt(var + eps) is dominated by the eps factor.
What do you think ? should I put eps at 1e-7 level for instance ?
Thanks