Does deepcopying optimizer of one model works across the model? or should I create new optimizer every time?

autograd
2

deepcopying the optimiser won’t work because it will either give you another optimiser for model1, or more likely simply break the references to the model parameters.

Here is what I would do…

model2 = Model() # get new instance
model2.load_state_dict(model1.state_dict()) # copy state
opt2 = torch.optim.Adam(model2.parameters(), lr = 0.0001) # get new optimiser
opt2.load_state_dict(opt1.state_dict()) # copy state
4 Likes