Loss is increasing from first epoch itself

1

I am training my siamese network for nlp. I have used lstm in it. and BCELoss. My loss is increasing from the first epoch. The first 36 epoch loss is
error after 0 is
272.4357
[torch.FloatTensor of size 1]

error after 1 is
271.8972
[torch.FloatTensor of size 1]

error after 2 is
271.5598
[torch.FloatTensor of size 1]

error after 3 is
271.6979
[torch.FloatTensor of size 1]

error after 4 is
271.7315
[torch.FloatTensor of size 1]

error after 5 is
272.3965
[torch.FloatTensor of size 1]

error after 6 is
273.3982
[torch.FloatTensor of size 1]

error after 7 is
275.1197
[torch.FloatTensor of size 1]

error after 8 is
275.8228
[torch.FloatTensor of size 1]

error after 9 is
278.3311
[torch.FloatTensor of size 1]

error after 10 is
277.1054
[torch.FloatTensor of size 1]

error after 11 is
277.8418
[torch.FloatTensor of size 1]

error after 12 is
279.0189
[torch.FloatTensor of size 1]

error after 13 is
278.4090
[torch.FloatTensor of size 1]

error after 14 is
281.8813
[torch.FloatTensor of size 1]

error after 15 is
283.4077
[torch.FloatTensor of size 1]

error after 16 is
286.3093
[torch.FloatTensor of size 1]

error after 17 is
287.6292
[torch.FloatTensor of size 1]

error after 18 is
297.2318
[torch.FloatTensor of size 1]

error after 19 is
307.4176
[torch.FloatTensor of size 1]

error after 20 is
304.6649
[torch.FloatTensor of size 1]

error after 21 is
328.9772
[torch.FloatTensor of size 1]

error after 22 is
300.0669
[torch.FloatTensor of size 1]

error after 23 is
292.3902
[torch.FloatTensor of size 1]

error after 24 is
300.8633
[torch.FloatTensor of size 1]

error after 25 is
305.1822
[torch.FloatTensor of size 1]

error after 26 is
333.9984
[torch.FloatTensor of size 1]

error after 27 is
346.2062
[torch.FloatTensor of size 1]

error after 28 is
354.6148
[torch.FloatTensor of size 1]

error after 29 is
341.3568
[torch.FloatTensor of size 1]

error after 30 is
369.7580
[torch.FloatTensor of size 1]

error after 31 is
366.1615
[torch.FloatTensor of size 1]

error after 32 is
368.2455
[torch.FloatTensor of size 1]

error after 33 is
391.4102
[torch.FloatTensor of size 1]

error after 34 is
394.3190
[torch.FloatTensor of size 1]

error after 35 is
401.0990
[torch.FloatTensor of size 1]

error after 36 is
422.3723
[torch.FloatTensor of size 1]

2

Most likely culprit here is exploding gradients due to a high learning rate.

Try lowering your learning rate by one or two orders of magnitude and see if it still grows.

2 Likes
3

Thanks for the reply. Will try.

4

Thanks, you helped me with my problem too.