Confusion matrix output not printing correctly

1

I am trying to make a confusion matrix for my model, but it is not coming outright. There are 3 classes but my matrix is displaying 2x2 instead of 3x3. Another issue I am having is that the numbers in my matrix do not equal the total number of images in my test set. Can someone help em figure out my error?

Run the testing batches

with torch.no_grad():
    for b, (pics, names) in enumerate(test_loader):
        
        if torch.cuda.is_available():
            pics = pics.cuda()
            names = names.cuda()
            
        b+=1

        # Apply the model
        y_val = MobileNet(pics)
        val_loss = criterion(y_val, names)
        # Tally the number of correct predictions
        test_predicted = torch.max(y_val.data, 1)[1] 
        tst_corr += (test_predicted == names).sum()
        
        test_correct.append(tst_corr)
        Val_accuracy = tst_corr.item()*100/(b*batch)
        # Print Validation results
        if b%7 == 0:
            print(f'test_epoch: {i:2}  batch: {b:4}  Val_loss: {val_loss.item():10.8f}  \Val_accuracy: {Val_accuracy:7.3f}%')
            test_acc.append(Val_accuracy)
            test_losses.append(val_loss)
            print("\n")

names = names.cpu()
test_predicted = test_predicted.cpu()
arr = confusion_matrix(names.view(-1), test_predicted.view(-1))
#df_cm = pd.DataFrame(arr, class_names, class_names)
plt.figure(figsize = (9,6))
sn.heatmap(df_cm, annot=True, fmt=“d”, cmap=‘BuGn’)
plt.xlabel(“prediction”)
plt.ylabel(“label (ground truth)”)
plt.show();

2

Could you post the code for confusion_matrix?
If you are using unique inside this method, the created confusion matrix might be smaller, if your model does not predict all classes.

3

I did the text underneath the code snippet is the rest of the code.

4

How many samples of each class is in your test dataset?

5

I think it’s the scikit-learn libraries confusion_matix, seaborn libraries heatmap then plotted with matplot lib.

1 Like
6

class 1: 31
class 2 & 3: 100

7

Check the unique values in both tensors for the current batch via print(torch.unique(names)) and print(torch.unique(test_predicted)). If the current batch does not contain all classes, the confusion matrix will be created by only using the found class indices.

You could force the output of the non-existent classes via the labels argument:

x = torch.randint(0, 2, (10,))
y = torch.randint(0, 2, (10,))

confusion_matrix(x, y, labels=[0, 1, 2])
> array([[4, 2, 0],
         [1, 3, 0],
         [0, 0, 0]], dtype=int64)
8

Thanks I figure out what I was doing wrong. I was using test_predicted and names in the confusion matrix calculations these values keep changing at each iteration of the data loader after each batch. to fix this I created a list to store each of them and attempted
value =
valuename =
with torch.no_grad():
for b, (pics, names) in enumerate(test_loader):

        if torch.cuda.is_available():
            pics = pics.cuda()
            names = names.cuda()
            
        b+=1

        # Apply the model
        y_val = MobileNet(pics)
        val_loss = criterion(y_val, names)
        # Tally the number of correct predictions
        test_predicted = torch.max(y_val.data, 1)[1] 
        value.append(test_predicted)
        valuename.append(names)
        tst_corr += (test_predicted == names).sum()
        
        test_correct.append(tst_corr)

confusion_matrix(value, valuename)
Unfortunately, that gives me this: ValueError: unknown is not supported

9

names is on cuda. So, you might to want to detach it and then convert to numpy array before feeding it into confusion_matrix.