How to save two best models for differnet dataset in pytorch

Hello all
I am new to pytorch. I have a deep learning pytorch model. After each training epoch I check the accuracy for validation set and if it is greater than the last best accuracy, I save the respective model and checkpoints. Now, I want to use the same model in which already trained in the simple dataset and fine-tune it on a more difficult dataset (by resuming training on the second dataset). obviously I do not reach to the same good accuracy that I have gotten from simple dataset, but I do get a better accuracy than only training on difficult dataset (without pretraining on simple dataset). My problem is how to manage and save the best model for the second dataset(the difficult one).

I appreciate your guide.

You could save the model of relative better accuracy of the more difficult dataset as what you did in the simple dataset. Does this way fit your use case?