I see that in the tutorials, from PyTorch website, we have a way to save the entire model along with the loss and optimizer parameters. That being said, I don’t understand what is the “loss” in this code below (from PyTorch website - https://pytorch.org/tutorials/beginner/saving_loading_models.html#saving-loading-a-general-checkpoint-for-inference-and-or-resuming-training) :
model = TheModelClass(*args, **kwargs) optimizer = TheOptimizerClass(*args, **kwargs) checkpoint = torch.load(PATH) model.load_state_dict(checkpoint['model_state_dict']) optimizer.load_state_dict(checkpoint['optimizer_state_dict']) epoch = checkpoint['epoch'] loss = checkpoint['loss'] model.eval() # - or - model.train()
Is it the final value of the loss before stopping training? Or is it the entire loss history (aka, a list of losses per epoch)?
I also have a StepRL, can its state be saved like the others? Since I need to recover the lr decay.