Low Validation Score on Pretrained Alexnet from Pytorch models for ImageNet 2012 dataset

vision
#1

I am using pre-trained AlexNet network to validate some prior work.

The code is as follows:

import os
import torch

import torchvision
import torchvision.datasets as datasets
import torchvision.models as models
import torchvision.transforms as transforms


model = torch.hub.load('pytorch/vision:v0.6.0', 'alexnet', pretrained=True)
model.eval()

batchsize = 50000
workers = 1
dataset_path = 'data/imagenet_2012/'
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
val_data = datasets.ImageFolder(root=os.path.join(dataset_path, 'val'), transform=transforms.Compose( [transforms.Resize(256), transforms.CenterCrop(224), transforms.ToTensor(), normalize,]))
val_loader = torch.utils.data.DataLoader(val_data, batch_size=batchsize, num_workers=workers)


batch = next(iter(val_loader))
images, labels = batch

with torch.no_grad():
    output = model(images)


for i in output:
    out_soft = torch.nn.functional.softmax(i, dim=0)
    print(int(torch.argmax(out_soft)))

When I execute this and compare with ILSVRC2012_validation_ground_truth.txt , I get top 1% accuracy of 5% only.

What am I doing wrong here?

Thank you.

#2

Instead of the fop loop you could directly use torch.argmax(output, dim=1), but your code should also work.
I guess the correspondence between the data and labels might be broken. Could you check some random examples from your ground truth file and the input image manually?

#3

Hi @ptrblck
Thank you for your reply.
And you are right!

I checked the images in the validation folder manually against the class names listed here. I noticed that the ILSVRC2012_validation_ground_truth.txt file has INCORRECT class names listed. For example, the first image in the folder is of a snake, but groundtruth lists it as a goldfish. Also, the second image was correctly classified when I checked manually, but groundtruth said otherwise.

In this case, I found out I should use the groundtruth files from here:

Thank you again.

update:
New validation accuracy based on the groundtruth in the zip file in the link:
Top_1 = 56.522%
Top_5 = 79.066%

#4

can you share the label file, i cannot find it and have the same issue

#5

Hi, did you read my message above? I shared the link in there.
Here, is the direct one: http://dl.caffe.berkeleyvision.org/caffe_ilsvrc12.tar.gz
Get this and untar it. Look up the *.txt files for labels.