Problem with transfer learning

#1

Hello everybody,

I am new to Pytorch and I am have a problem, it is probably something basic. I have just trained a simple model for FashionMNIST (as an example), then transfer the learning to a new CIFAR problem. The code for FashionMNIST is:

CNN(
  (conv1): Conv2d(3, 16, kernel_size=(5, 5), stride=(1, 1))
  (conv2): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))
  (conv3): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1))
  (fc): Linear(in_features=512, out_features=64, bias=True)
  (classifier): Linear(in_features=64, out_features=10, bias=True)
)

class_names = trainset.classes
dataiter = iter(trainloader)
inputs, classes = dataiter.next()

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model_ft = best_model_FashionMNIST

num_ftrs = model_ft.fc.in_features
model_ft.fc = nn.Linear(num_ftrs, 64) 

model_ft = model_ft.to(device)

criterion = nn.CrossEntropyLoss()

optimizer_ft = optim.Adam(model_ft.parameters(), lr=0.0001)

model_ft = train_model(model_ft, criterion, optimizer_ft, device, num_epochs=20)

I train the new probem with parameters from FashionMNIST:

model_ft.eval()
classes = testset.classes 

with tqdm.notebook.tqdm(total=len(testloader), unit='batch', desc=f'Evaluation', 
                        position=100, leave=True) as pbar:  
    for X, Y in testloader: 

        X = X.to(device)
        Y = Y.to(device)

        _, Y_ = torch.max(model_ft(X).data, 1)

        for y, y_ in zip(Y, Y):
          if y == y_:
              correct[y] += 1
          total[y] += 1
          acc[y] = correct[y]/total[y]

        pbar.set_postfix(accuracy=sum(acc)/len(classes))
        pbar.update()   

for class_name,class_acc in zip(classes,acc):
    print(f'Accuracy for class {class_name:5s}: {class_acc:.2f}')
print(f'Accuracy: {sum(acc)/len(classes):.2f}')

When trying to solve, I do not understand the reason that I get only two classes in the prediction, could someone tell me why it is due please?

The results is:
Y_:
tensor([0, 0, 1, 0, 1, 0, 1, 1, 0, 1, 1, 0, 1, 1, 1, 1, 1, 1, 0, 0, 1, 0, 0, 1,
0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 1, 0, 1, 0, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0,
0, 0, 1, 0, 1, 1, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0, 1, 0, 0, 1, 0, 0, 0,
0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1,
0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 0, 0, 0, 0,
0, 0, 0, 1, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 0, 1, 1, 0, 0, 0, 0, 1, 0, 0,
0, 0, 1, 1, 1, 1, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 1, 0, 1, 0, 0,
0, 0, 1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 1, 0, 1, 1, 0,
1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 0, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1,
1, 0, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 1,
1, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 1, 1, 0, 1, 0], device=‘cuda:0’)

Y:
tensor([3, 8, 8, 0, 6, 6, 1, 6, 3, 1, 0, 9, 5, 7, 9, 8, 5, 7, 8, 6, 7, 0, 4, 9,
5, 2, 4, 0, 9, 6, 6, 5, 4, 5, 9, 2, 4, 1, 9, 5, 4, 6, 5, 6, 0, 9, 3, 9,
7, 6, 9, 8, 0, 3, 8, 8, 7, 7, 4, 6, 7, 3, 6, 3, 6, 2, 1, 2, 3, 7, 2, 6,
8, 8, 0, 2, 9, 3, 3, 8, 8, 1, 1, 7, 2, 5, 2, 7, 8, 9, 0, 3, 8, 6, 4, 6,
6, 0, 0, 7, 4, 5, 6, 3, 1, 1, 3, 6, 8, 7, 4, 0, 6, 2, 1, 3, 0, 4, 2, 7,
8, 3, 1, 2, 8, 0, 8, 3, 5, 2, 4, 1, 8, 9, 1, 2, 9, 7, 2, 9, 6, 5, 6, 3,
8, 7, 6, 2, 5, 2, 8, 9, 6, 0, 0, 5, 2, 9, 5, 4, 2, 1, 6, 6, 8, 4, 8, 4,
5, 0, 9, 9, 9, 8, 9, 9, 3, 7, 5, 0, 0, 5, 2, 2, 3, 8, 6, 3, 4, 0, 5, 8,
0, 1, 7, 2, 8, 8, 7, 8, 5, 1, 8, 7, 1, 3, 0, 5, 7, 9, 7, 4, 5, 9, 8, 0,
7, 9, 8, 2, 7, 6, 9, 4, 3, 9, 6, 4, 7, 6, 5, 1, 5, 8, 8, 0, 4, 0, 5, 5,
1, 1, 8, 9, 0, 3, 1, 9, 2, 2, 5, 3, 9, 9, 4, 0], device=‘cuda:0’)

I appreciate your comments

Greetings
K.

#2

Do you see these two-class output only during validation or was it also the case during training?
Could you check the unique values of each target tensor during training and make sure it’s containing all expected 10 labels throughout the training?

#3

Thank you very much for the answer @ptrblck. In train for Fashion MNIST, I have 10 classes. My result is in train:

Accuracy para la clase T-shirt/top: 0.85
Accuracy para la clase Trouser: 0.97
Accuracy para la clase Pullover: 0.90
Accuracy para la clase Dress: 0.90
Accuracy para la clase Coat : 0.76
Accuracy para la clase Sandal: 0.97
Accuracy para la clase Shirt: 0.61
Accuracy para la clase Sneaker: 0.95
Accuracy para la clase Bag : 0.97
Accuracy para la clase Ankle boot: 0.95
Accuracy promedio (balanceado): 0.88

If I train for CIFAR with no transfer learning, the result is the same.

Accuracy para la clase airplane: 0.69
Accuracy para la clase automobile: 0.78
Accuracy para la clase bird : 0.43
Accuracy para la clase cat : 0.48
Accuracy para la clase deer : 0.46
Accuracy para la clase dog : 0.57
Accuracy para la clase frog : 0.66
Accuracy para la clase horse: 0.70
Accuracy para la clase ship : 0.66
Accuracy para la clase truck: 0.69
Accuracy promedio (balanceado): 0.61

I take the learned model “best_model_FashionMNIST” and try to transfer learning for CIFAR problem. I get two classes as indicated above.

I will appreciate the comments

Greetings
K

#4

Are these metrics calculated using the validation dataset? If so, did you verify that the targets contain all labels during training?

#5

Thank you very much @ptrblck for the comments, I was finally able to solve it. There was a problem in assigning the dataloader.