Is overfitting a good idea for testing and debuging?

vision
1

Hi,

I’m training a model on text-image pairs. I wondered if overfitting to a small subset of your training data is a valid method of knowing if your model will perform the task?

Thank you in advance.

2

Hey,

My technique when training a model is to overfit it to the train set to see if my model is capable enough to learn the data. However, after I see it, I will try to mitigate the overfitting effect to make my model more general to unseen data.

3

Hi @Andre_Amaral_IST

Thank you for your answer.

Do you train your model on the entire training set or a single batch?

I’ve trained my model on a single batch, and loss shows that the model converges, but when evaluating the model on the same data as it was trained on, it shows nothing significant (my model is not able to classify any labels)

4

In my case since I have 90 million of data, I train my model using batches. To see if the model is learning the training data I evaluate the model using the train data and if it is not good enough I change the hyperparameters unit it is perfect. After that, I use the test data and check whether exists or not the overfit.

In your case it seems that your model is not learning at all, it might be a problem of the model structure (layers or activation functions) not being complex enough for your data.
The other problem might be the hyperparameters, such as the learning rate, number of neurons, etc…

1 Like
5

Thank you, that’s insightful. I’ll try modifying the hyperparameters.

I didn’t understand if you train your model on the full 90M or a subset (e.g. 1000 samples) to overfit?

I’ve only tested it on a single batch on 16 text-image pairs.

6

I feed the model with 90M of data, however, it is fed in batches of 10.000, otherwise, my computer would not be capable to do it :slight_smile:

16 pairs seems a bit low, but my suggestion is for you o test the hyperparameters and see if with that only 16 examples your model is capable to learn something.

7

Thanks so much for your advice. I’ll give that a try :slight_smile:

8

You’re welcome, if the model keeps not learning just use this post and the community and I will try to help you!
Keep up the good work!

1 Like