Which is better? Loss + optimizer question

1

I am testing my program on a subset of my data to make sure everything works. I’m experimenting with different loss functions and optimizers. I’m getting the following results:

The figure below uses MSE as the loss, Adam as the optimizer, and takes 250-300 epochs for the loss to plateau. The plateau sits around a loss of 225.

loss_rmse_adam
loss_rmse_adam800×600 24.8 KB

The figure below uses MSE as the loss, Adam as the optimizer, and takes 50-100 epochs for the loss to plateau. The plateau sits around a loss of 214.

loss_rmse_sgd
loss_rmse_sgd800×600 21.1 KB

The figure below uses L1 as the loss, Adam as the optimizer, and takes ~250 epochs for the loss to plateau.

loss_rmse_L1
loss_rmse_L1800×600 27.1 KB

What am I supposed to prefer? Fewer epochs? Or lower loss? Edit: batch size and learning rate for all of the above are exactly the same.

I also am not sure about why my RMSE dips slightly, then increases a little and then levels out. Is this dip normal?

2

I’m following the procedure outlined in this article below:

However, I’m using smaller images of 40x40 pixels, and only 3 columns of data in my table.