Hi everyone. I’m trying to create an LR scheduler that combines cosine annealing and exponential decrease. My objective is to have something similar to the left image (see bottom of the post)
To do that, I combine the CosineAnnealingLR and the ExponentiaLR using the ChainedScheduler.
The code is something similar
scheduler_list = [
torch.optim.lr_scheduler.CosineAnnealingLR(optimizer, T_max = 9, eta_min = 1e-5),
torch.optim.lr_scheduler.ExponentialLR(optimizer, gamma = 0.92)
]
lr_scheduler = torch.optim.lr_scheduler.ChainedScheduler(schedulers_list, optimizer)
But the final result (for 80 epochs) is the right image (see bottom of the post)
I have the initial decrease, but after that, it always rebounds at the same value.
How can I obtain my initial goal? Am I doing something wrong with the parameter settings? Or is it directly the choice of schedulers?
