Hi Tom,
you are right, it’s torch 1.4.
the reproduce step is easy, simply create the class and run step:
from torch import optim
from torchvision.models import resnet50
from torch.optim import lr_scheduler
model = resnet50()
optimizer = optim.SGD(model.parameters(), lr=0.05)
for param_group_dict in optimizer.param_groups:
param_group_dict[‘initial_lr’] = param_group_dict[‘lr’]
lr_scheduler.CosineAnnealingWarmRestarts(optimizer=optimizer, T_0=50, T_mult=1, eta_min=0, last_epoch=0)
thanks for your replay.
the use case can be found at Pytorch latest update(1.4) broke MultiStepLR: wrong LR after step from _get_closed_form_lr