I am new to Neural Networks and currently doing a project for university. I am trying to train a CNN using frames that portray me shooting a ball through a basket. And my aim is for the network to be able to classify the result( hit or miss) correctly. When I train the network, the training accuracy increases slowly until it reaches 100%, while the validation accuracy remains around 65% (It is important to mention here that 65% is the percentage of shots that have a Miss label. So the network gives the highest Validation accuracy when it predicts all frames are a miss) Does anyone have experience with a similar problem?
I am using PyTorch and Resnet18 ( have tried other architectures as well but they all gave the same result). My frames are jpg images of sie 224.
As an optimizer, both Adam and SGD gave the same result
Thank you in advance
Hi Wassim,
Sounds like your model is over fitting to the training set. You could try adding regularization or dropout during training to avoid it. You can also read up more about how else to avoid overfitting to the training set online.
Hope this helps,
Best,
Prerna
Hi Wassim,
In addition to what @Prerna_Dhareshwar said, do have a look at your training data to make sure there are no biases or features in the image that would allow the network to “cheat”. One example would be ratio of hits and misses in your training data, which ideally should be 1(called a balanced dataset). Another example, if you collected the training data for “hit” during the day, training data for “miss” during the night, and all validation data during the night, your network could just be predicting day or night depending on the lighting conditions, and get 100% accuracy on your training data.
Hope this helps!
Hi! In addition to previous answers I would like to suggest you to use data augmentations. This will help you to increase your training set and will have a regularization effect.
I am having the same issue. I am doing 3D medical image synthesis and train loss(red) and valid loss(blue) looks as below plot.
The valid loss doesnt drop. I tried standardizing and normalizing and changed the validation sets.
I would request to look at my discussions lately for more details (having trouble to paste links from phone)
Would appreciate any feedbacks.
@Mazhar_Shaikh Thank you for your input. My data is quite unbalanced (around 65% miss and 35% hit). Is the unbalance large enough to cause this error? It seems like, during validation, the model tries to predict the outcome but gets a very low accuracy, so it goes back to predicting all shots to be a miss and gets stuck on 65% accuracy.
@banikr @Prerna_Dhareshwar Thank you for the tips. I will give it a try 