Hello everyone! I have a problem that my model cannot correctly predict the numbers I need. Tried various functions of activation, regularization, scaler, scheduler and others. In vandb, according to me, the calculation of the loss of MaE goes like this: 1 epoch of loss 23, 2 epoch 35, 3 epoch 1000, 4 epoch 50, 5 epoch 1000, 6 epoch 25 and so on without changing. But the accuracy steadily moves from 0 to 90%. And when transferring data to the model that it has not seen or seen, the result is bad. Also, I have already logged the gradients. I use a 3-layer linear model, the first layer is 10000 (and this size is mandatory), the second is 5000 (but I tried to experiment) and the last is 100.
Maybe someone has an idea what my problem is?