Why my accuracy values don´t change on my train model

Hi, I am using transfer learning to classify histological breast cancer images with VGG16 pre-trained. I have 7900 images y I separated in train, val and test. I have made a custom dataset to read the images and edit the train model function from https://pytorch.org/tutorials/beginner/transfer_learning_tutorial.html

My accuracy in train and val doesn´t change and I don´t know why, can you please chek it out, than you. Also, my loss it´s so high!

I´m pretty new in this

The code I use is here:

class BreakHistDataset(Dataset):
        def __init__(self, csv_file, root_dir,transform=None):
            self.df = pd.read_csv(csv_file)
            self.root_dir = root_dir
            self.transform = transform
        def __len__(self):
           return len(self.df)
        def __getitem__(self, idx):
            img_name = os.path.join(self.root_dir,self.df.iloc[idx, 3])
            image = io.imread(img_name)
            labels = self.df.iloc[idx, 4]
            if self.transform:
                image = self.transform(image)

            return image,labels

transforme_train = transforms.Compose([
     transforms.Normalize([0.7890, 0.6167, 0.7618],[0.1047, 0.1388, 0.0894])])

transforme_val = transforms.Compose([
   transforms.Normalize([0.7890, 0.6167, 0.7618],[0.1047, 0.1388, 0.0894])])

transforme_test = transforms.Compose([
   transforms.Normalize([0.7890, 0.6167, 0.7618],[0.1047, 0.1388, 0.0894])])

bh_dataset_train = BreakHistDataset(csv_file='/content/gdrive2/My Drive/breakhis/train1.csv',root_dir='/content/gdrive2/My Drive/breakhis/BreaKHis_v1', transform = transforme_train)
bh_dataset_val = BreakHistDataset(csv_file='/content/gdrive2/My Drive/breakhis/val.csv',root_dir='/content/gdrive2/My Drive/breakhis/BreaKHis_v1', transform = transforme_val)
bh_dataset_test = BreakHistDataset(csv_file='/content/gdrive2/My Drive/breakhis/test.csv',root_dir='/content/gdrive2/My Drive/breakhis/BreaKHis_v1', transform = transforme_test)

train_loader = DataLoader(bh_dataset_train, batch_size=32,shuffle=True)
val_loader = DataLoader(bh_dataset_val, batch_size=32,shuffle=True)
test_loader = DataLoader(bh_dataset_test, batch_size=32,shuffle=True)

use_gpu = torch.cuda.is_available()

def train_model(model, criterion, optimizer, scheduler, num_epochs=10):
      since = time.time()
      best_model_wts = model.state_dict()
      best_acc = 0.0
      p_epoch = tqdm(range(num_epochs))

      for epoch in p_epoch:
        print('Epoch {}/{}'.format(epoch, num_epochs - 1))
        print('-' * 10)

        loss_train = 0.0
        corrects_train = 0
        loss_val = 0.0
        corrects_val = 0

         w = []
         l = []

        for img, labels in tqdm(train_loader):
           if use_gpu:
              img = img.cuda()
              labels = labels.cuda()
               img, labels = img, labels
           outputs = model(img)
           _, preds = torch.max(outputs.data, 1)
           loss = criterion(outputs, labels)

           loss_train += loss 
           corrects_train += (preds == labels.data).sum().item()
      epoch_loss = loss_train / len(train_loader)
      epoch_acc = corrects_train / len(train_loader)

      plt.title('Loss vs Epocas')
      plt.plot(l,label = 'loss train')
      plt.plot(figsize=(20, 10))

       print('{} Loss: {:.4f} Acc: {:.4f} RunCorr: {:.4f}'.format('train', epoch_loss, 
       r = []
       m = []   

     for img, labels in tqdm(val_loader):

         if use_gpu:
            img = img.cuda()
            labels = labels.cuda()
            img, labels = img, labels


        outputs = model(img)
        _, preds = torch.max(outputs.data, 1)
        loss = criterion(outputs, labels)
       loss_val += loss
       corrects_val += (preds == labels.data).sum().item()
   epoch_loss_val = loss_val / len(val_loader)
   epoch_acc_val = corrects_val / len(val_loader)
   m.append( epoch_acc_val)

   print('{} Loss: {:.4f} Acc: {:.4f}'.format('val', epoch_loss_val, epoch_acc_val))

   if epoch_acc_val > best_acc:
     best_acc = epoch_acc_val
     best_model_wts = model.state_dict()
time_elapsed = time.time() - since
print('Training complete in {:.0f}m {:.0f}s'.format(time_elapsed // 60, time_elapsed % 60))
print('Best val Acc: {:4f}'.format(best_acc))

 # load best model weights
return model

num_classes = 8
model_conv = torchvision.models.vgg16(pretrained=True)
for param in model_conv.parameters():
param.requires_grad = False

num_features = model_conv.classifier[6].in_features
features = list(model_conv.classifier.children())[:-1] # Remove last layer
features.extend([nn.Linear(num_features, num_classes)])# Add our layer with 4 outputs
model_conv.classifier = nn.Sequential(*features) # Replace the model classifier

if use_gpu:
  model_conv = model_conv.cuda()

criterion = nn.CrossEntropyLoss()
optimizer_conv = optim.Adam(model_conv.classifier.parameters(), lr=0.0001,betas = [0.9, 
exp_lr_scheduler = lr_scheduler.StepLR(optimizer_conv, step_size=7, gamma=0.1)
model_conv = train_model(model_conv ,criterion, optimizer_conv,exp_lr_scheduler )

Results for all the epochs

train Loss: 1.6865 Acc: 14.3810 
val Loss: 1.6122 Acc: 15.5312

The code looks generally alright.
There is one issue, which might create a growing memory usage and could thus trigger an out of memory error.
In these lines of code it seems you are directly storing the loss, which is still attached to the computation graph:

loss_train += loss 

Store the detached tensor, if the loss list is just used to calculate the average loss etc.:

loss_train += loss.item()

Have you tried to unfreeze the model and train it completely?
If so, what was the highest accuracy you could achieve?
Note that the pretrained models were trained on ImageNet12. The input images were thus normalized with:

normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                 std=[0.229, 0.224, 0.225])

While it seems you’ve calculated the image statistics manually based on your dataset or where do these values come from?

Thanks for the answer!! I’m gonna check the loss.
Q1: I haven’t tried to train it completely, but i will try it.
Q2: The highest accuracy achived is 60%.

Q3: Yes I’ve calculated the statistics manually based on the dataset with this code:

mean = 0.
std = 0.
for images in train_loader:
    batch_samples = images.size(0) 
    images = images.view(batch_samples, images.size(1), -1)
    mean += images.mean(2).sum(0)
    std += images.std(2).sum(0)

mean /= len(train_loader.dataset)
std /= len(train_loader.dataset)

Hi again, I change in my train function this: loss_train += loss.item(), also I noticed that the classes of the dataset were unbalanced so I used this code to balance them. Even doing that my accuracy doesn´t change. What can I do? Here is the code for the balance:

class_weights_train = 1./torch.Tensor(class_sample_counts_train)
class_weights_train = class_weights_train.double()

train_targets = [sample[1] for sample in bh_dataset_train]
train_samples_weight = [class_weights_train[class_id] for class_id in train_targets]

class_sample_counts_val = [48,1138,331,324,245,189,235,251]

class_weights_val = 1./torch.Tensor(class_sample_counts_val)
class_weights_val = class_weights_val.double()

val_targets = [sample[1] for sample in bh_dataset_val]
val_samples_weight = [class_weights_val[class_id] for class_id in val_targets]

train_sampler = torch.utils.data.sampler.WeightedRandomSampler(train_samples_weight, len(bh_dataset_train))
val_sampler = torch.utils.data.sampler.WeightedRandomSampler(val_samples_weight, len(bh_dataset_val))

You could scale down the problem and try to (perfectly) overfit a small data snippet, e.g. just 10 samples from your dataset.
If your model is not able to overfit these samples, you might need to further fine tune the hyperparameters.
Let me know, if this still doesn’t work, as there might then be subtle code bugs, which I’m missing at the moment.