No change in my loss

1

Hi,

I am trying to train my classification model (Inception V3), even though my loss from first epoch is very low, there is no decrease in the loss.
I played with learning rate, but still this is the issue. Here is the result:

> 
>  Epoch 1/50, Train_acc: 0.2847 , Train_loss: 0.0143 , Val_AUC: 0.4803, Val_acc: 0.3855, Val_loss: 0.0111
> ----------
> Epoch 2/50, Train_acc: 0.3234 , Train_loss: 0.0114 , Val_AUC: 0.4677, Val_acc: 0.4217, Val_loss: 0.0111
> ----------
> Epoch 3/50, Train_acc: 0.5634 , Train_loss: 0.0111 , Val_AUC: 0.5714, Val_acc: 0.6225, Val_loss: 0.0114
> ----------
> Epoch 4/50, Train_acc: 0.6891 , Train_loss: 0.0110 , Val_AUC: 0.4764, Val_acc: 0.6165, Val_loss: 0.0123
> ----------
> Epoch 5/50, Train_acc: 0.6942 , Train_loss: 0.0111 , Val_AUC: 0.5312, Val_acc: 0.5944, Val_loss: 0.0113
> ----------
> Epoch 6/50, Train_acc: 0.7560 , Train_loss: 0.0109 , Val_AUC: 0.5369, Val_acc: 0.6185, Val_loss: 0.0116
> ----------
> Epoch 7/50, Train_acc: 0.7616 , Train_loss: 0.0110 , Val_AUC: 0.5084, Val_acc: 0.6145, Val_loss: 0.0115
> ----------
> Epoch 8/50, Train_acc: 0.7621 , Train_loss: 0.0109 , Val_AUC: 0.4869, Val_acc: 0.6145, Val_loss: 0.0117
> ----------
> Epoch 9/50, Train_acc: 0.7621 , Train_loss: 0.0109 , Val_AUC: 0.4779, Val_acc: 0.6145, Val_loss: 0.0118
> ----------
> Epoch 10/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5094, Val_acc: 0.6145, Val_loss: 0.0118
> ----------
> Epoch 11/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4887, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 12/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4546, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 13/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5634, Val_acc: 0.6145, Val_loss: 0.0118
> ----------
> Epoch 14/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5575, Val_acc: 0.6145, Val_loss: 0.0118
> ----------
> Epoch 15/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5435, Val_acc: 0.6145, Val_loss: 0.0118
> ----------
> Epoch 16/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4879, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 17/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5334, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 18/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5144, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 19/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5072, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 20/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4981, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 21/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4811, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 22/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5246, Val_acc: 0.6145, Val_loss: 0.0118
> ----------
> Epoch 23/50, Train_acc: 0.7621 , Train_loss: 0.0107 , Val_AUC: 0.5560, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 24/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4935, Val_acc: 0.6145, Val_loss: 0.0120
> ----------
> Epoch 25/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5321, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 26/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4758, Val_acc: 0.6145, Val_loss: 0.0120
> ----------
> Epoch 27/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4604, Val_acc: 0.6145, Val_loss: 0.0120
> ----------
> Epoch 28/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5021, Val_acc: 0.6145, Val_loss: 0.0120
> ----------
> Epoch 29/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4892, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 30/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5390, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 31/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4862, Val_acc: 0.6145, Val_loss: 0.0120
> ----------
> Epoch 32/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.4736, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 33/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5286, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 34/50, Train_acc: 0.7621 , Train_loss: 0.0107 , Val_AUC: 0.4850, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 35/50, Train_acc: 0.7621 , Train_loss: 0.0108 , Val_AUC: 0.5115, Val_acc: 0.6145, Val_loss: 0.0119
> ----------
> Epoch 36/50, Train_acc: 0.7621 , Train_loss: 0.0107 , Val_AUC: 0.4829, Val_acc: 0.6145, Val_loss: 0.0120

What is your interpretation of these results?

2

It just means that ur model can’t learn any further and u should change the architecture of ur model it may also be due to ur weight initialization

3

Thanks for your reply, but I am not initializing an weights!

4

Try using learning rate decay while training to reduce ur learning rate by a factor while training.
If it doesn’t work try changing ur network