# Slightly different results for different batch sizes **URL:** \ **Category:** vision\ **Created:** [April 24, 2019, 1:09pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458 "2019-04-24T13:09:40Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![zhaopku](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/ecc23a/32.png) [@zhaopku](https://discuss.pytorch.org/u/zhaopku)\ **Post date:** [April 24, 2019, 1:09pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/1 "2019-04-24T13:09:40Z")
I am using a VGG19 network to classify CIFAR10 images, yet I have slightly different results for data of different batch sizes. Code: > f\_image = net.forward(Variable(image, requires\_grad=True)).data > > f\_image2 = net.forward(Variable(image[0].unsqueeze(0), requires\_grad=True)).data > > f\_image3 = net.forward(Variable(image[0].unsqueeze(0), requires\_grad=True)).data > print(‘-----------------------------------------’) > print(‘f\_image - f\_image2 = {}’.format((f\_image[0] - f\_image2).cpu().data.numpy())) > print(‘f\_image - f\_image3 = {}’.format((f\_image[0] - f\_image3).cpu().data.numpy())) > print(‘f\_image2 - f\_image3 = {}’.format((f\_image2 - f\_image3).cpu().data.numpy())) > print(‘-----------------------------------------’) Specifically, image is of size `[batch_size, n_channels, height, width]`. And I expect `f_image[0].unsqueeze(0)` to be the same as `f_image2` and `f_image3`. However, they are slightly different: > * * * > > f\_image - f\_image2 = [[ 0.0000000e+00 3.7252903e-09 -1.8626451e-09 3.7252903e-09 > 0.0000000e+00 -3.7252903e-09 3.7252903e-09 -1.8626451e-09 > 3.7252903e-09 -1.3969839e-09]] > f\_image - f\_image3 = [[ 0.0000000e+00 3.7252903e-09 -1.8626451e-09 3.7252903e-09 > 0.0000000e+00 -3.7252903e-09 3.7252903e-09 -1.8626451e-09 > 3.7252903e-09 -1.3969839e-09]] > f\_image2 - f\_image3 = [[0. 0. 0. 0. 0. 0. 0. 0. 0. 0.]] > -----------------------------------------] The difference is very small, is it a bug or expected behavior of PyTorch? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [April 24, 2019, 1:59pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/2 "2019-04-24T13:59:10Z")
Values smaller than `1e-6` are due to the limited floating point precision, so you shouldn’t worry about it. Unrelated to this, but `Variables` are deprecated since PyTorch `0.4.0`, so you can just use `tensors` now. Also, I wouldn’t recommend to use `.data` anymore, as this might have some negative side effects, since Autograd cannot track the changes made to the `.data` attribute. This is not really important in your current code, but might lead to some issues in the future. 😉 ---
**Author:** ![zhaopku](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/ecc23a/32.png) [@zhaopku](https://discuss.pytorch.org/u/zhaopku)\ **Post date:** [April 24, 2019, 2:12pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/3 "2019-04-24T14:12:26Z")
But even for the same data sample (image[0] in my case), there are small differences. Does the rounding policy of PyTorch have different behaviors when it comes to different batch sizes? ---
**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\ **Post date:** [April 24, 2019, 2:15pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/4 "2019-04-24T14:15:00Z")
Yes, even for the same data, you might run into floating point precision limits. You could try to enable deterministic behavior as described [here](https://pytorch.org/docs/stable/notes/randomness.html), but I’m not sure if your results will get more deterministic. ---
**Author:** ![zhaopku](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/ecc23a/32.png) [@zhaopku](https://discuss.pytorch.org/u/zhaopku)\ **Post date:** [April 24, 2019, 2:21pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/5 "2019-04-24T14:21:56Z")
Thanks! Actually I have enabled the deterministic behavior. The difference still exists. The deterministic flags ensure that I have the same difference in different runs, but do not eliminate the differences. ---
**Author:** ![zhaopku](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/z/ecc23a/32.png) [@zhaopku](https://discuss.pytorch.org/u/zhaopku)\ **Post date:** [April 24, 2019, 2:27pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/6 "2019-04-24T14:27:05Z")
So I guess PyTorch might have some “trade-off” policy for samples in the same batch. If you want maximal precision for the 1st sample in a batch, you might lose more precision in other samples of the batch. And the “trade-off” policy is why I slightly different results for different batch sizes. Is my understanding correct? ---
**Author:** ![fanlu](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/fanlu/32/11195_2.png) [@fanlu](https://discuss.pytorch.org/u/fanlu)\ **Post date:** [April 24, 2019, 2:38pm UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/7 "2019-04-24T14:38:56Z")
Could you help me to check my problem, [Different batch sizes give different test accuracies](https://discuss.pytorch.org/t/different-batch-sizes-give-different-test-accuracies/24577), Thanks ---
**Author:** ![water](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/w/a698b9/32.png) [@water](https://discuss.pytorch.org/u/water)\ **Post date:** [December 27, 2019, 11:51am UTC](https://discuss.pytorch.org/t/slightly-different-results-for-different-batch-sizes/43458/8 "2019-12-27T11:51:26Z")
hello,do you solve this problem? For different results, I am very affected in semantic segmentation, I got a very different result for different batch sizes during prediction