Is my loss function normal?

1

my loss changes along iteration as the figure.

2025-05-05 10-32-20屏幕截图
2025-05-05 10-32-20屏幕截图918×717 87.3 KB

Is my loss normal?

I use “optimizer = optim.SGD(parameters, lr = args.learning_rate, weight_decay = args.weight_decay_optimizer)”, and I train three standalone models simultaneously (the loss depends on all three models dont share any parameters).

Why my loss trend differs from the curves at many papers which decrease in a stable manner?

2

Help me out. Is the horizontal axis per epoch or per batch?

If per batch(highly likely), that just shows that some batches contribute more loss while others less, progressively getting better.

If per epoch, I’m not sure, that would be an odd chart - perhaps LR is set too high? Definitely going to have over-fitting with 10k+ epochs(and a pricey power bill).