When to believe the Accuracy

vision
1

When I train the classification task, I don`t know when to believe the Accuracy.
Sometimes within the 10 epochs, there is a high accuracy but It is not stable.(After this epoch, Acc is fall.)
The accuracy is high and loss is small enough to believe in early epochs, Can I trust this accuracy?
or this is meaningless score?

Epoch : 4
Classification Acc : 70%
Epoch : 5 
**Classification Acc : 90%**
...
Epoch : 6
Classification Acc : 75%
...
Epoch : 50
**Classification Acc : 90%**
Epoch : 51
Classification Acc : 89%
...
According to the Learning rate, Acc is changed or held.
2

Did you calculate the accuracy on the training or validation data?
How large is your data in general? If your dataset is quite small, your accuracy might be quite noisy as just a few misclassified samples might change the accuracy.

Generally speaking, if you have a good training, validation, test splitting (data from approx. same domain, not too few samples), and prevented all data leaks, you can trust your accuracy.

3

@ptrblck
The accuracy is from validatation data during training process.
The dataset is digit data like MNIST, … from pytorch.
Of course MNIST dataset is quite easy to classification, but my network have only 2-3 Conv layers so I’m not sure that I can trust the acc from on early epochs.