Pytorch loss does not decrease

[jpeg729]

You can save and load the optimizer state in a similar way to the model state.

torch.save(optimizer.state_dict(), filename)
optimizer.load_state_dict(torch.load(filename))