Load and save of optimizer and scheduler

1

i want to resume the saved model and continue training. justing wondering what’s the exact procedure to load optimizer and scheduler, then use them on gpu.

question 1: after loading model state dict, is my model still on gpu?
here’s my code

    model = Modelclass()
    device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')
    model.to(device)  

    optimizer = optim.Adam(model.parameters(), lr = 'lr')
    scheduler = optim.lr_scheduler.ExponentialLR(optimizer, gamma=0.95)

    if os.path.isfile("model.pth.tar"):
        checkpoint = torch.load("model.pth.tar")
        model.load_state_dict(checkpoint['model_state_dict'])
        optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
        scheduler.load_state_dict(checkpoint['scheduler_state_dict'])

question 2: is my optimizer is still on GPU after that?
i saw some tutorials sad that the following code should be added, if you want to use optimizer on gpu:

        for state in optimizer.state.values():
            for k, v in state.items():
                if torch.is_tensor(v):
                    state[k] = v.cuda()

but it seems that if use Adam this is not working
question3: did this scheduler need additional steps to be loaded? such as scheduler._step_count?In my case, i use ExponentialLR.

thanks!