Augmentation lead a worst loss

Dear All, I try to do some transfer learning on 3d data (mri) using 3d resnet.
I have tried to do augmentation on data loader only in training using Elastic function and random rotation. I found a worst result using augmentation than the use of no augmented data.

So can interpeter this as suggestion my endpoint I wan to classify are not equaly distribute? Thanks in advance for any help

If the posted curves represent the losses, I would recommend to make sure your model is training and the loss decreases.
The augmented images might be “harder examples”, which might create a higher loss, but it seems that your model is not learning anything at all at the moment.