I am training a
preactresnet18 network on the cifar100 dataset. I have adjusted all training settings but still, I can see spikes in my training loss and accuracy. I have no idea why it’s happening. I have changed the learning rate batch size but still i cannot solve it.
Single GPU (NVIDIA 3080
The batch size is
0.001, momentum 0.9 weight decay