I am using the epoch validation accuracy as a metric to save the model current status if there is a progress but I was wondering if it is the right way or should I use the lowest epoch validation loss or a combination of both ?
I am working on multi class classification problem
From my own experience, I always save all model after each epoch so that I can select the best one after training based on validation accuracy curve, validation loss curve and training loss curve.
1 Like
yes but how do you proceed if you dont have the same epoch with the lowest validation loss and the highest validation accuracy?
When everything else is normal, I prefer accuracy.
Others are welcome to answer this question.
1 Like
in my case the validation plots accuracy and loss are not looking like the exemplar curve I suspect an overffiting so i am only releying on the accuracy so far
As what I know, selecting the checkpoint at the lowest loss is the most common way. And as @Eta_C says, I would also recommend to save all checkpoints and do model averaging after the training is done. Here is an example fairseq/average_checkpoints.py at main · pytorch/fairseq (github.com)
let us say I care more about accuracy this is what I should use as improvement and I dont get the best epoch in which the lowest loss is and the highest validation , + the fact that I am getting overfitting
@huahuanZ torchcontrib.optim.SWA also implements model averaging 