Hello,

I am trying to obtain a wave model which requires simultaneous solutions of multiple ODE’s.

My input is has 2 input features. First one is Stimuli which only initiates the wave, and mostly 0’s.

However, my model can’t produce a specific part of the wave.

As you can see, most of the wave is predicted very accurately.

What I tried is,

- Changing the hidden dimension
- Changins the sequence length
- Changing the batch size
- Adding additional FC layers
- Increasing and decreasing LSTM Layer numbers
- Using MSE and L1 Loss functions
- Increasing data sizes

When I try to make my model more complex (such as increasing number of layers & size of hidden dimension) It tends to overfit,

Exact opposite effect occurs when model is simplified or regularization is added.

How can it learn the area after initial sharp increase ? Is there a known approach for this ? I am at a loss at this point.

Thank you kindly.