Loss function(MSE)

My loss function’s values using the MSE criterion and the Adam optimizer descends very slow and have Periodic surges.How can i eliminate the sudden increase?

What type of model do you train? What do you train to do?
What is your training data? Your learning rate?
???