Batch Normalization weights

Hi I was trying to figure out the internals of the batch_normalization in terms of what represents what. Couple questions: answers appreciated

http://pytorch.org/docs/master/nn.html#torch.nn.BatchNorm1d

1.Does gamma(1/std) and beta(mean) in the formula corresponds to .weight and .bias respectively?

Yes, gamma and beta correspond to weight and bias, respectively.

3 Likes