Pytorch latest update(1.4) broke CosineAnnealingWarmRestarts: T_cur is not define

1

Hello,
maybe i missing something, but my tests failed on CosineAnnealingWarmRestarts creation,

the problem is:

    super(CosineAnnealingWarmRestarts, self).__init__(optimizer, last_epoch)

    self.T_cur = self.last_epoch

the super class call the override step function that use self.T_cur that is not defined.

i open a bug report(https://github.com/pytorch/vision/issues/2119), but if i missed something please let me know
thanks

2

Two quick comments:

  • Is this something regarding PyTorch itself or torchvision? As far as I can tell, the scheduler is defined in torch.optim, which would make it a PyTorch thing.

  • In any bug, it would be helpful if you provided code to produce the error.

    • If you found a bug, it makes digging into it much easier.
    • If it is something with your particular use of the class - which may be uncommon yet something that should work, you avoid someone making up an example, saying “works for me” and move on.
    • If it is something where your use is not supported (i.e. bug in your code), it’ll help finding that, too.

    So it would seem to be a win in any case. :slight_smile:

Best regards

Thomas

3

Hi Tom,
you are right, it’s torch 1.4.

the reproduce step is easy, simply create the class and run step:
from torch import optim
from torchvision.models import resnet50
from torch.optim import lr_scheduler

model = resnet50()
optimizer = optim.SGD(model.parameters(), lr=0.05)
for param_group_dict in optimizer.param_groups:
param_group_dict[‘initial_lr’] = param_group_dict[‘lr’]
lr_scheduler.CosineAnnealingWarmRestarts(optimizer=optimizer, T_0=50, T_mult=1, eta_min=0, last_epoch=0)

thanks for your replay.

the use case can be found at Pytorch latest update(1.4) broke MultiStepLR: wrong LR after step from _get_closed_form_lr