Model starts to overfit right away

Hi, I am using a pretrianed 3d resnet on brain mri images to do binary classification. My problem is that validation error and auc never decreases while training error decreases very fast and auc of training goes up to 1.

Im using a batch size of 2 and SGD optimizer with lr 0.001. I also do minMax normalization on the input.
Here is what the train/val loss graph looks like:

This is a sample of my data:

You could try to add more regularization via (more) dropout, weight decay etc.
Data augmentation might also help.

1 Like