I am training a preactresnet18 network on the cifar100 dataset. I have adjusted all training settings but still, I can see spikes in my training loss and accuracy. I have no idea why it’s happening. I have changed the learning rate batch size but still i cannot solve it.
Training Configuration
Single GPU (NVIDIA 3080 12 GB)
The batch size is 384
Epochs 400
Learning rate 0.001, momentum 0.9 weight decay 5e-4
Results
