Dear PyTorch community,
I observed that the frequency of evaluation steps is affecting the final accuracy of my PyTorch model. When I evaluate the model after each epoch of training (this is with a dummy example, where I am only training the model for an epoch), I get an accuracy of 0.7551 and a loss of 1.0198748077556585.
Like below -
for epoch in range(num_epochs): train_one_epoch() evaluate()
However, when I evaluate the model before training, followed by training and then evaluation again, I get an accuracy of 0.7656 and a loss of 0.9733265159995692.
for epoch in range(num_epochs): evaluate() train_one_epoch() evaluate()
Similarly, if I evaluate the model twice before training, followed by training and then evaluation again, I get an accuracy of 0.7631 and a loss of 0.9899978056834762.
I would like to know why this is happening and how to ensure consistent results regardless of the order of training and evaluation steps. I would appreciate any suggestions or insights from the PyTorch community.
Here is the full code - example.py