How to apply PyTorch lr scheduler to my self-defining optimizer

1

I want to apply torch.optim.lr_scheduler.CosineAnnealingWarmRestarts() to my self-defining optimizer. However, the error occurs TypeError: XXX is not an Optimizer(XXX is my custom optimizer). I creat my optimizer by modifying torch.optim.SGD and it works well. How to solve this problem?

2

pytorch/lr_scheduler.py at master · pytorch/pytorch · GitHub shows that the TypeError will be raised when your customized optimizer is not an instance Optimizer.

Does your customized optimizer subclass the base class Optimizer?

3

Yes. My customized optimizer is an instance Optimizer. I directly modified the torch.optim.SGD code to create my customized optimizer.

4

You should subclass the base Optimizer class imported from torch.optim.optimizer, i.e., from torch.optim.optimizer import Optimizer

5

Thanks! That works! I find that I import the old version optimizer: from Grad_optimizer import Optimizer.