Check out : https://github.com/bluesky314/Cyclical_LR_Scheduler_With_Decay_Pytorch
Shouldn’t this be merged within pytorch?
1 Like
@apaszke can help. I know cyclic learning rate with cyclic momentum can become a huge problem to implement, but you can also use fastai source code to easily create your own functions.