# Changing batch size

**URL:** <https://discuss.pytorch.org/t/changing-batch-size/56700>\
**Category:** Uncategorized\
**Created:** [September 24, 2019, 7:12am UTC](https://discuss.pytorch.org/t/changing-batch-size/56700 "2019-09-24T07:12:04Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![slavavs](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/slavavs/32/15429_2.png) [@slavavs](https://discuss.pytorch.org/u/slavavs)\
**Post date:** [September 24, 2019, 7:12am UTC](https://discuss.pytorch.org/t/changing-batch-size/56700/1 "2019-09-24T07:12:04Z")

</div>

Why, when changing batch size, do the weights of neurons change? If I test new data with a batch size equal to the size with which I trained NN, then the results are good. If you change the batch size, the results are bad.

```auto
class MyResNet(ResNet):
    def __init__ (self):
        super(MyResNet, self). __init__ (BasicBlock, [2, 2, 2, 2], num_classes=3)
        self.conv1 = torch.nn.Conv2d(1, 64, 
            kernel_size=(7, 7), 
            stride=(2, 2), 
            padding=(3, 3), bias=False)
...
model.load_state_dict(torch.load('save.pth'))
criterion = nn.CrossEntropyLoss(reduction='sum')
optimizer = torch.optim.AdamW(model.parameters(), lr=learning_rate)
...
outputs = model(x)
loss1 = criterion(outputs, y)
optimizer.zero_grad()
loss1.backward()
optimizer.step()

```

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [September 24, 2019, 12:04pm UTC](https://discuss.pytorch.org/t/changing-batch-size/56700/2 "2019-09-24T12:04:00Z")

</div>

> [@slavavs](#):
>
> Why, when changing batch size, do the weights of neurons change?

The weights do not change based on the batch size or the forward pass alone.  
Could you explain this issue a bit more?

> [@slavavs](#):
>
> If I test new data with a batch size equal to the size with which I trained NN, then the results are good. If you change the batch size, the results are bad.

Make sure to call `model.eval()` before evaluating your model, as otherwise e.g. the running estimates of batchnorm layers will be updated, which depends on the used batch size.

---

<div class="post-metadata">

**Author:** ![slavavs](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/slavavs/32/15429_2.png) [@slavavs](https://discuss.pytorch.org/u/slavavs)\
**Post date:** [September 24, 2019, 12:19pm UTC](https://discuss.pytorch.org/t/changing-batch-size/56700/3 "2019-09-24T12:19:25Z")

</div>

I don’t know how, but my can teaches samples in batch.  
I did not shuffle.  
After training, if I change the batch size, then the network can no longer correctly predict.  
If I call model.eval () the result is very bad

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [September 25, 2019, 2:25pm UTC](https://discuss.pytorch.org/t/changing-batch-size/56700/4 "2019-09-25T14:25:21Z")

</div>

This might be due to skewed running estimates in your batch norm layer.  
Try to use a higher batch size during training or adapt the `momentum`.