Yes, this link (Saving and loading a model in Pytorch?) has an example of optimizer as well. So, basically, you create a dictionary and save the checkpoint as follows:
save_checkpoint({
'epoch': epoch + 1,
'arch': args.arch,
'state_dict': model.state_dict(),
'best_prec1': best_prec1,
'optimizer' : optimizer.state_dict(),
}, is_best)
Note that I just copied this from that link.