What are wrong in the figures?

1

I want to know what are wrong in the plotted diagrams.They are train loss and validation loss.If it is informal,please answer how can I solve the problem.Thanks in advance!![loss|640x476](upload://rqPczFdZ8naAVCLpj9kTtKB0fSloss1 t.png)

2

The figure shows noisy losses and indicates that your model is not training.
Generally, if you are stuck and your training isn’t working properly, I would recommend to start by overfitting a small dataset, e.g. just 10 samples, and make sure it’s working fine by playing around with hyperparameters.

3

Thanks for answering!

5

Please explain that model is not training.How can I overfit small dataset.Thanks a lot in advance!

6

The model is not training since the graph shows the loss is not decreasing, and the goal of a ML model is to minimize the loss between the actual and predicted values. If the model were training correctly, you would see a decrease in the loss as the epochs go up (it may still be a bit noisy, but it will be a notable decrease).

Onto the overfitting part, say you have 10000 samples. Use only 100 of them and train your model on that subset. Play around with number of hidden layers, number of neurons in each layer, different optimizers (and their parameters) until you can achieve near perfect accuracy. The reply to this question gives a comprehensive in-depth explanation.

2 Likes
7

Thanks a lot for explaining!

8

If I use deep learning and samples are only about 2000,what parameters should be played.Does the figure improve slightly or not.Thanks a lot in advance!loss

9

No, your model still isn’t learning. The loss is still extremely noisy. The parameters that you can look into are:

  1. Number of Hidden Layers.
  2. Number of Neurons in the Hidden Layers.
  3. Batch Sizes.
  4. Add Dropout Layers in the Network
  5. Optimizers (and their parameters like learn rate, weight decay, etc).

Do all of the above only on the small subset of data. The goal is to overfit the model on this tiny batch.

Losses
Losses1465×2507 190 KB

I’ve attached an image from one of my projects. As you can see, it is still noisy, but there is a definite decrease. This is the kind of curve you should expect.

10

Thanks a lot for answering!!