Hi, What is the meaning of this jumps in the loss function?
I am decreasing the learning rate at iterations [300, 600, 900, 1200]
Hi, @lycaenidae. Some questions/comments, but fair warning… I’m not very experienced with pytorch. 
- What is the scale of your plot?
- Are you calculating loss as an instantaneous value? A moving average?
- What is your use case? Without some context, it’s tough to tell what could be going on.
- Are you shuffling the samples in your training set? If not, your instantaneous loss values could definitely jump around like that. Datasets are sometimes “clumpy”. Imagine hitting your model with a large “clump” of a novel class after you’ve been training for a while.