Pytorch doesn't give expected output on correctly classified data

Firstly, a bunch of data is classified by the CNN model. Then, I’m trying to make prediction on correctly classified data from first step, which is expected to give an accuracy of 100%. However, I found the result is unstable, sometimes 99+%, but not 100%. Is there anybody know what is the problem with my code? Thank you very much in advance, it has troubled me several days ~ ~

torch.version

‘0.3.1.post2’

 import numpy as np
    import torch 
    import torch.nn as nn
    from torch.autograd import Variable

    n = 2000
    data = np.random.randn(n, 1, 10, 10)
    label = np.random.randint(2, size=(n, ))

    def test_pred(model, data_test, label_test):

        data_batch = data_test
        labels_batch = label_test

        images = torch.autograd.Variable(torch.FloatTensor(data_batch))
        labels = torch.autograd.Variable(torch.FloatTensor(labels_batch))

        outputs = model(images)

        _, predicted = torch.max(outputs.data, 1)

        correct = (np.array(predicted) == labels_batch).sum()

        label_pred = np.array(predicted)

        acc = correct/len(label_test)
        print(" acc:", acc)

        return acc, label_pred

    class CNN(nn.Module):
        def __init__(self):
            super(CNN, self).__init__()
            self.layer1 = nn.Sequential(
                nn.Conv2d(1, 16, kernel_size=5, padding=2),
                nn.BatchNorm2d(16),
                nn.ReLU(),
                nn.MaxPool2d(2))
            self.layer2 = nn.Sequential(
                nn.Conv2d(16, 32, kernel_size=5, padding=2),
                nn.BatchNorm2d(32),
                nn.ReLU(),
                nn.MaxPool2d(2))
            self.fc = nn.Linear(128, 2)

        def forward(self, x):
            out = self.layer1(x)
            out = self.layer2(out)
            out = out.view(out.size(0), -1)
            out = self.fc(out)
            return out

    cnn = CNN()

    [_, label_pred] = test_pred(cnn, data, label)

    print("Acc:", np.mean(label_pred==label))
    # Given the correctly classified data in previous step, expect to get 100% accuracy
    # Why it sometimes doesn't give a 100% accuracy ?
    print("Using selected data size {}:".format(data[label_pred==label].shape))
    _, _ = test_pred(cnn, data[label_pred==label], label[label_pred==label])

You should set your model to evaluation by calling model.eval() in your test method.
This makes sure that some layers like BatchNorm and Dropout are switched to evaluation mode, i.e. for BatchNorm the running statistics are used instead of being updated.

The previously correctly classified samples will have different mean and std values and thus you are updating the BatchNorm layers, which might yield different results.

1 Like

Thank you so much for help, it works pretty well and the explanation makes perfect sense to me.:crazy_face: