After building optimizer, moving model to other device and move back, does optimizer still work?

1

as thisissue say that: every time you move model to different device, you should build optimizer again, so

model = Model()
model.cuda()
optimizer = optim.Adam(model.parameters())

for d, gt in trn_dataloader:
    # train
    ... 
    optimizer.step()
    model.cpu() # move to cpu
    # eval or do other things
    ...
    model.cuda()  # but finnally, move back

does optimizer run as expected?

1 Like