any recommend setting for CosineAnnealingWarmRestarts
as the lr is high to low in cosine way with each turn and then restart
high = 1e-3
low = 1e-6
T_max = 8
this lr scheduler per batches or per epoches are common / perform better ?
if we consider the aims of escape local optima, per batches or per epoches will be better?
and convergence also ?
anyone can give some hints on this setting