Pytorch model sometimes trains and sometimes does not learn

1

hi, i have a simple feed forward NN for doing forecasting on tabular time series data. i transformed the data and feed it through the network in following way:

  1. if the tabular data has n fields. and i want to use t times to predict T future, i transform the t rows to t * n features in first layer. than i ahve 10 cells in next layer and have T cells in next layer.

this simple model works very well sometimes and understands and fits the data really well sometimes. but most of the time it does not learn at all. it does not diverge but loss seem to get stuck and not reduce further. this does not happen always.

any ideas why it would not behave in-consistently. (mostly bad and sometimes really learns well)

thanks much,
Manish