I follow the tutorial to train a cnn model on CIFAR10, and when I use this model to validate on test_data, I got different accuracy when I use different batch_size on test_data, is it normal? As you can see below, as the batch_size increased to 280, the accuracy of this model has declined. I use batch size of 64 to train this model, I don’t know why just changing the batch size of test data will get different accuracy.
# Validate on test_data
model.eval()
def test_accu(batch_size):
test_loader = DataLoader(test_data, batch_size=batch_size, shuffle=False)
correct_count = 0.0
for i, data in enumerate(test_loader, 0):
img, labels = data
x = Variable(img)
y = Variable(labels)
if torch.cuda.is_available():
x = x.cuda()
y = y.cuda()
outs = model(x)
_, pred = torch.max(outs, -1)
correct_count += (pred == y).sum().data[0]
return correct_count
for i in range(200, 500, 10):
correct_count = test_accu(i)
print('BatchSize: {}, Accu: {}'.format(i, correct_count/len(test_data)))
BatchSize: 200, Accu: 0.90894
BatchSize: 210, Accu: 0.90894
BatchSize: 220, Accu: 0.90894
BatchSize: 230, Accu: 0.90894
BatchSize: 240, Accu: 0.90894
BatchSize: 250, Accu: 0.90894
BatchSize: 260, Accu: 0.90894
BatchSize: 270, Accu: 0.90382
BatchSize: 280, Accu: 0.4891
BatchSize: 290, Accu: 0.08974
BatchSize: 300, Accu: 0.06414
BatchSize: 310, Accu: 0.08462
BatchSize: 320, Accu: 0.11022
BatchSize: 330, Accu: 0.13582
BatchSize: 340, Accu: 0.1563
BatchSize: 350, Accu: 0.17678
BatchSize: 360, Accu: 0.19726
BatchSize: 370, Accu: 0.21774
BatchSize: 380, Accu: 0.23822
BatchSize: 390, Accu: 0.25358
BatchSize: 400, Accu: 0.26894
BatchSize: 410, Accu: 0.2843
BatchSize: 420, Accu: 0.29966
BatchSize: 430, Accu: 0.31502
BatchSize: 440, Accu: 0.32526
BatchSize: 450, Accu: 0.34062
BatchSize: 460, Accu: 0.35086
BatchSize: 470, Accu: 0.36622
BatchSize: 480, Accu: 0.37646
BatchSize: 490, Accu: 0.3867