How does batch norm behave in pytorch depending if one is in training mode or eval/inference mode?

[Brando_Miranda (MirandaAgent)]

I was looking at:

http://pytorch.org/docs/stable/_modules/torch/nn/modules/batchnorm.html#BatchNorm2d

and I didn’t see how how F.batch_norm works. I looked in the docs but could find it and perhaps it seems that it might be written in C which I dont know.

My question is how does batch_norm updates its parameters during training vs inference? When are the mean variance for example updated? What happens to them during inference mode? Are they just fixed?

[Brando_Miranda (MirandaAgent)]

How does pytorch’s batch norm know if the forward pass its doing is for inference or training?

how does pytorch’s batch norm know if the forward pass its doing is for inference or training? I am evaluating the the test performance of my net but realized that I’m not sure how my net knows if its training phase or inference phase. How does pytorch handle this?