Hello
I was following this tutorial. I have a question that after training for multiple epochs and over all mini batches of complete training set. Which set of weights are selected for the network when we are testing it.
Is it the weights of the epoch with lowest running loss or the last epoch?
If it is the weight for last epoch which function should i use to save the model parameters for lowest running loss or epoch loss and use that for testing?
Often you would want to save the state_dict of your model for the lowest validation loss (or highest validation accuracy).
Depending on your use case, you could split the training data into a training and validation set (e.g. using SubsetRandomSampler or Subset), observe the validation loss, and save the best model.
After saving you could calculate the test accuracy to get a signal of your model’s performance on unseen data.
Ok. Just to confirm. After training through each epoch i should run the validation set through the model and see validation loss. And should use the model for least validation loss after training through all epochs is complete. Right?
Also if my above lines are not coded in algo, model will use parameters that are set from the last training epoch for testing. As i referred to the tutorial same is being done in that. Weights for last epoch are being used for testing. Am i missing something in this?