# Model.eval() gives incorrect loss for model with batchnorm layers

**URL:** <https://discuss.pytorch.org/t/model-eval-gives-incorrect-loss-for-model-with-batchnorm-layers/7561>\
**Category:** Uncategorized\
**Created:** [September 19, 2017, 4:25am UTC](https://discuss.pytorch.org/t/model-eval-gives-incorrect-loss-for-model-with-batchnorm-layers/7561 "2017-09-19T04:25:03Z")\
**Posts on this page:** 1\
**Showing post:** 39

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [June 25, 2019, 3:52pm UTC](https://discuss.pytorch.org/t/model-eval-gives-incorrect-loss-for-model-with-batchnorm-layers/7561/39 "2019-06-25T15:52:48Z")

</div>

The high validation loss is due to the wrong estimates of the running stats.  
Since you are feeding a constant tensor (`batchone`: mean=1, std=0) and a random tensor (`batchtwo`: mean~=0, std~=1), the running estimates will be shaky and wrong for both inputs.

During training the current batch stats will be used to compute the output, so that the model might converge.  
However, during evaluation the batchnorm layer tries to normalize both inputs with skewed running estimates, which yields the high loss values.  
Usually we assume that all inputs are from the same domain and thus have approx. the same statistics.

If you set `track_running_stats=False` in your BatchNorm layer, the batch statistics will also be used during evaluation, which will reduce the eval loss significantly.

---

_[View the full topic](https://discuss.pytorch.org/t/model-eval-gives-incorrect-loss-for-model-with-batchnorm-layers/7561)._