i want to resume the saved model and continue training. justing wondering what’s the exact procedure to load optimizer and scheduler, then use them on gpu.
question 1: after loading model state dict, is my model still on gpu?
here’s my code
model = Modelclass() device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu') model.to(device) optimizer = optim.Adam(model.parameters(), lr = 'lr') scheduler = optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.95) if os.path.isfile("model.pth.tar"): checkpoint = torch.load("model.pth.tar") model.load_state_dict(checkpoint['model_state_dict']) optimizer.load_state_dict(checkpoint['optimizer_state_dict']) scheduler.load_state_dict(checkpoint['scheduler_state_dict'])
question 2: is my optimizer is still on GPU after that?
i saw some tutorials sad that the following code should be added, if you want to use optimizer on gpu:
for state in optimizer.state.values(): for k, v in state.items(): if torch.is_tensor(v): state[k] = v.cuda()
but it seems that if use Adam this is not working
question3: did this scheduler need additional steps to be loaded? such as scheduler._step_count?In my case, i use ExponentialLR.