How to solve ResNet Overfitting

vision
1

I am training a Boat Type classifier using dataset (https://www.kaggle.com/clorichel/boat-types-recognition) with 75-25 train-validation split.

Dataset is imbalanced.
to counter imbalance I did

  1. used train_test_split with stratify to divide the data.
  2. Over/Under Sampled the dataset using WeightedRandomSampler
  3. Added data augmentation for train and validation sets. (CenterCrop, RandomRotation([0,30]), etc …)

I used pretrained resnet18 as the backbone, and modified the last fc layer for classification.
I am using ReduceLROnPlateau scheduler.

still model is getting overfitted.
train (3)

2

Not sure exactly if this is overfitting. Validation loss didn’t increase by that much. You should run this for longer and see if the validation loss increases further.

One way to reduce overfitting in transfer learning is to freeze the initial layers and then train your network. In the case of ResNet, you can freeze the conv1, conv2, and conv3 layers and see if that helps.

3

I tried

  1. Freezing Resnet18 upto layer3 module (got train_acc 90% and validation acc 68%)
  2. Freezing resNet18 upto layer4 module and changed fc layer (got train_acc 99% and validation acc 79%)

Same problem I faced with Inceptionv3, resnet34 and resnet50

4

I see. In that case, you can try increasing augmentation (ColorDistortion, Solarize, ColorJitter, GaussianFilter etc.) and introduce Dropout in the layers.

Hope that helps.

5

Here is my current implementation https://www.kaggle.com/iabhyuday/notebook5b80d89625
Have a look, I’ve tried everything still don’t know whats wrong

6

[UPDATE]

Got better results after switching to SGD from Adam Optimizer. But validation accuracy and loss saturate early .
Learning curves (before early stopping)
train