Debugging loss plot

vision
1

I’m using a densenet121 for multilabel image classification.
Loss function is beta_score.
I used iterative stratification to balance distributions of multilabel data across splits for cross validation.
In the plot, the validation loss is lower than train loss.
I saw this link: https://www.pyimagesearch.com/2019/10/14/why-is-my-validation-loss-lower-than-my-training-loss/
They gave many reasons. But I’m still unsure.
And I guess I also don’t know if it’s something to be corrected.

I saw in https://machinelearningmastery.com/learning-curves-for-diagnosing-machine-learning-model-performance/ that if training loss is still decreasing than it means the model is underfitting.
I just want to make sure that this still holds true in my case (loss_train > loss_val)
This is a snapshot of the loss plot where blue is train and red is val.
I smoothed the plot (0,999).
smoothed_losses_densenet121

EDIT:
I deleted Dropout and added more epochs. But the plot shows the peak in the beginning. And I have no idea what it means.
loss_no_dropout_more_epochs

2

Did you check the data splits between the training and validation sets?
Could maybe “simpler” samples have landed in the validation set or are you using aggressive data augmentation techniques during training?

3

I used iterative stratification to balance distributions of multilabel data across splits for cross validation. And then I chose 1 fold/5folds to be the validation fold. So, normally there is no data leakage. Now, concerning the “simpler” samples, I have no idea. Do people normally go through the samples one by one to see?