Loss suddenly increases using Adam optimizer

9

Yes, I use a combination of gradient clipping and batch normalization which has pretty much ensured that this never occurs again.