Training accuracy is decreasing and validation is constant but very less then training accuracy

vision
1

I am working on crop disease with 20k images. I have imbalance dataset, so downsampled it but when I applied the resnet model on this I am getting training accuracy initially with 86% but later epochs it is decreasing and val acc to 7% is constant with different batch_size, weight_decay. I have augmented data with brightness, rescaled, etc.

2

Hi,

Could it be that the model overfits on your training set given that you only have 20k images and use a resnet? Does a very simple model give similar behavior?