Pytorch implementation of RAdam and Look Ahead

#1

Is there a way to include RAdam and Look Ahead in pytorch models.
I tried the below approach from is-there-a-pytorch-implementation-of-radam-lookahead

from radam import RAdam
from optimizer import Lookahead
base_optim = RAdam(model.parameters(),lr = 0.001) 
optimizer =  Lookahead(base_optim, k=5, alpha=0.5)

but getting ModuleNotFoundError: No module named 'optimizer'.

#2

Your error is because you import: from optimizer import Lookahead but optimizer is not an actual package I think. You need to get the file from the repo that you linked.

1 Like