RMSE vs SD Error

I currently have a NN regression model that, upon training, my RMSE steadies out and is unable to be reduced further than ~1.78. Unsure of the issue I compared it to the standard deviation of the targets and noticed a similar value of 1.79. Any help with diagnosing this is appreciated. I tried both SGD and LBFGS as optimizers and got the same issue. Thanks!