Lr Scheduler is not starting from the base_lr

1

I used CLR scheduler and it is working weirdly. It is not starting from the base_lr.
My base_lr = 1e-06 and max_lr =1. But it starts from 0.01. If I increase my step size it then, it starts from 0.001, which I think step size should not effect the start of lr scheduler.

I found a way around. For a range of 10^-6. I did 2 cycle with different base_lr and max_lr.
1st cycle : base_lr = 1e-06 ; max_lr = 1e-.3
2nd cycle : base_lr = 1e-03 ; max_lr = 1

It seems it is working when the range is less.

Did any one else found issue using CLR scheduler. ? Is it a bug or am I doing it incorrectly

1 Like