Getting batchnorm layer statistics

vision
1

Hi!
How can I extract the mean and variance of a given batch in batchnorm layer, is it the running_mean\running_var just without model.evel()
Thanks a lot!

2

Running mean would contain the exponential average over previous batches. To get the mean and variance of the current batch you have to compute them using separate functions.

3

What do you mean by separate functions

4

Use separate functions to compute mean like torch.mean().

5

but how can I extract the inputs to the batchnorm

6

What do you mean?

If you want to get the current running_mean and running_var in a pretrained model, use model.{your_batch_norm_module}.running_mean and model.{your_batch_norm_module}.running_var.

If you want to get the running_mean and running_var in a pretrained model after forward x, use torch.nn.functional.batch_norm or just forward that layer.

If you donot have a pretrained model, and want to get the running_mean and running_var, init running_mean to 0 and running_var to 1 then use torch.nn.functional.batch_norm or torch.batch_norm.

1 Like
7

I want the mean and the variance of a batch as the normbatch layer calculate

8

Use torch.mean() and torch.std(). There is also torch.normalize() function that you can look into.

9

thanks a lot but how can I get the tensor of the batchnorm, X

1 Like
10

Hi, have you found a way to do it?