Hi all,

It seems like models in eval mode return different outputs depending on the batch size of the input provided.

It is my understanding this is not expected.

I already checked here, here (and in other 4 posts but the new user limit limits me to 2 links) but the answer is not obvious to me.

I created some dummy code to demonstrated the problem I’m having.

Any Idea if this is expected or why it is happening?

torch version: 1.10.1

python version: 3.9.7

```
import torch
import torchvision
model = torchvision.models.resnet18(pretrained=True, progress=True)
model.eval()
a = torch.ones((1, 3, 96, 96))
b = torch.ones((64, 3, 96, 96))
with torch.no_grad():
a_out = model(a)
b_out = model(b)
diff = torch.absolute(b_out[0] - a_out[0])
total_diff = torch.sum(diff)
average_diff = torch.mean(diff)
print(f"equal: {torch.equal(a_out[0], b_out[0])}")
print(f"rtol=0.00001: {torch.allclose(a_out[0], b_out[0])}")
print(f"rtol=0.0001: {torch.allclose(a_out[0], b_out[0], rtol=0.0001)}")
print(f"rtol=0.001: {torch.allclose(a_out[0], b_out[0], rtol=0.001)}")
print(f"rtol=0.01: {torch.allclose(a_out[0], b_out[0], rtol=0.01)}")
print(f"rtol=0.1: {torch.allclose(a_out[0], b_out[0], rtol=0.1)}")
print(f"total diff {total_diff}")
print(f"avg diff {average_diff}")
```

And this is the output:

```
equal: False
rtol=0.00001: False
rtol=0.0001: False
rtol=0.001: True
rtol=0.01: True
rtol=0.1: True
total diff 0.04874414950609207
avg diff 7.616273478561197e-07
```