Please reproduce the behaviour and explain why this happens.
resnet50 = models.resnet50(pretrained=True) # load pretrained model
resnet50.eval() # set model to eval mode
# batch size > 1
for index, batch in enumerate(imagenet_val_loader): # iterate over dataloader
output = resnet50(batch) # predict for a batch of images
_, pred = output.topk(1, 1, True, True) # get predicted classes
print(pred) # show predicted classes
# single image or batch size = 1
for index, image in enumerate(imagenet_val_dataset): # iterate over dataset
output = resnet50(image) # predict for a single image
_, pred = output.topk(1, 1, True, True) # get predicted class
print(pred) # show predicted class
Make sure that the loader is not on shuffle
to reproduce the behaviour.
- Do you get the same predictions from the loader and the dataset?
- Try again without
resnet50.eval()
(comment it), is the prediction the same as before? If not, is it swapped?
What I observe:
With eval, the model predicts correctly for the image in the dataset
loop (single) and predicts incorrectly for the images in the loader
loop (batches). Without eval, the performance is reversed.
What I expect:
Without eval, it works for batches and not for individual images. (Not ideal but I understand why - no problem here).
With eval, the model should predict correctly for single images (does do) and batches of images (does not do).