How to implement torch.optim.lr_scheduler.CosineAnnealingLR?

vision
5

oh…right…missed that…thanks

However, can I implement the SGDR with warm restarts in PyTorch?
Right now, it does not seem to be restarting. I want the cosine annealing to happen inside every epoch and then restart for the next epoch.