Without using data augmentation gives results better than using data augmentation

I am a beginner to deep learning, I’m doing the image classification problem on a small self plant disease imaging dataset (400 images)composed of 7 classes. I am using transfer learning(pretrained Resnet152) model and have applied some image augmentation techniques to increase the size of training data(random rotation, flipping etc.) but the performance was worse than not using data augmentation and the model get overfitted. The first image shows the curves which are generated with data augmentation and the second one without data augmentation Can anyone help me to know what is the problem and how to solve it? Thanks