NN stuck in the validation set

nlp
1

Hello,

I am training a model on a standard dataset (SemEvalABSA) were the validation set accuracy constantly gets stuck at exactly 17.25% and then at 65%. Two curious things happen when i train it:

  • Most of the times I train it for like 50 epochs and it I still get 65% in the validation set. The training acc fluctuates arround that number but the validation acc remains constant at exactly 65.00%.

  • One rare time i got in 4 epochs arround 80% acc in the validation set with the same hyperparameters.

Is this a normal behavior? anyone knows why could this be happening?

2

Hi,

How big is your dataset and are you using batchnorm? This happened to me on a smaller dataset (hundreds) with batchnorm, removing it from the network helped.