# Weird BatchNorm behaviour

**URL:** <https://discuss.pytorch.org/t/weird-batchnorm-behaviour/7082>\
**Category:** Uncategorized\
**Created:** [September 5, 2017, 10:04pm UTC](https://discuss.pytorch.org/t/weird-batchnorm-behaviour/7082 "2017-09-05T22:04:07Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![asanakoy](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/asanakoy/32/1234_2.png) [@asanakoy](https://discuss.pytorch.org/u/asanakoy)\
**Post date:** [September 5, 2017, 10:04pm UTC](https://discuss.pytorch.org/t/weird-batchnorm-behaviour/7082/1 "2017-09-05T22:04:07Z")

</div>

I have a pretrained CNN model which has several `BatchNorm2d` layers.

I take a batch with 1 image, set `model.eval()` and run forward pass.  
If I run the same image but with `model.train()` then I get different output!

As I understood the only difference of the `BatchNorm` behaviour in `eval` and `train` mode is that in `eval` it doesn’t update moving averages. It means that in my experiment I should have got the same output in both cases!  
Could you please point me out to my mistake?

---

<div class="post-metadata">

**Author:** ![ngimel](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ngimel/32/14766_2.png) [@ngimel](https://discuss.pytorch.org/u/ngimel)\
**Post date:** [September 6, 2017, 6:39pm UTC](https://discuss.pytorch.org/t/weird-batchnorm-behaviour/7082/2 "2017-09-06T18:39:38Z")

</div>

In the train mode, batch normalization calculates the statistics for the batch you provide, normalizes by _calculated_ statistics and applies affine transformation. In the eval mode, instead of calculating statistics, batch norm uses provided moving averages to normalize. Having different output in eval and train mode is expected.