Saving you best model during stochastic training

Is there a canonical way of caching the best model weights during training? I am training using an nn.Module class, using SGD with momentum and learning rate. On rare occasions, the model with the smallest loss() occurred somewhere in the middle of the epochs. I would like to somehow set aside that model for later use, rather than just using the model weights from the final epoch.