# RMSE loss function

**URL:** <https://discuss.pytorch.org/t/rmse-loss-function/16540>\
**Category:** Uncategorized\
**Created:** [April 17, 2018, 12:20pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540 "2018-04-17T12:20:18Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![ddd24](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/d/90db22/32.png) [@ddd24](https://discuss.pytorch.org/u/ddd24)\
**Post date:** [April 17, 2018, 12:20pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/1 "2018-04-17T12:20:18Z")

</div>

Hi all,

I would like to use the RMSE loss instead of MSE. From what I saw in pytorch documentation, there is no build-in function. Any ideas how this could be implemented?

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [April 17, 2018, 8:11pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/2 "2018-04-17T20:11:20Z")

</div>

Wouldn’t it work, if you just call `torch.sqrt()` in `nn.MSELoss`?

```auto
x = Variable(torch.randn(5, 10), requires_grad=True)
y = Variable(torch.randn(5, 10))

criterion = nn.MSELoss()
loss = torch.sqrt(criterion(x, y))
loss.backward()
print(x.grad)

```

---

<div class="post-metadata">

**Author:** ![pierre](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pierre/32/6867_2.png) [@pierre](https://discuss.pytorch.org/u/pierre)\
**Post date:** [October 24, 2018, 2:08pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/3 "2018-10-24T14:08:55Z")

</div>

The solution of @ptrblck is the best I think (because the simplest one).  
For the fun, you can also do the following ones:

```auto
# create a function (this my favorite choice)
def RMSELoss(yhat,y):
    return torch.sqrt(torch.mean((yhat-y)**2))

criterion = RMSELoss
loss = criterion(yhat,y)

```

```auto
# create a nn class (just-for-fun choice :-) 
class RMSELoss(nn.Module):
    def __init__ (self):
        super(). __init__ ()
        self.mse = nn.MSELoss()
        
    def forward(self,yhat,y):
        return torch.sqrt(self.mse(yhat,y))

criterion = RMSELoss()
loss = criterion(yhat,y)

```

---

<div class="post-metadata">

**Author:** ![YannDubs1](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yanndubs1/32/6325_2.png) [@YannDubs1](https://discuss.pytorch.org/u/YannDubs1)\
**Post date:** [January 31, 2019, 9:52pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/4 "2019-01-31T21:52:26Z")

</div>

You should be careful with `NaN` which will appear if the `mse=0`. Something like this would probably be better :

```auto
class RMSELoss(nn.Module):
    def __init__ (self, eps=1e-6):
        super(). __init__ ()
        self.mse = nn.MSELoss()
        self.eps = eps
        
    def forward(self,yhat,y):
        loss = torch.sqrt(self.mse(yhat,y) + self.eps)
        return loss

```

---

<div class="post-metadata">

**Author:** ![Damien\_Lancry](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/damien_lancry/32/11393_2.png) [@Damien\_Lancry](https://discuss.pytorch.org/u/Damien_Lancry)\
**Post date:** [April 9, 2019, 2:14am UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/5 "2019-04-09T02:14:49Z")

</div>

sqrt of 0 is 0, not nan

```auto
>>> torch.sqrt(torch.zeros(1))
tensor([0.])

```

---

<div class="post-metadata">

**Author:** ![YannDubs1](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/yanndubs1/32/6325_2.png) [@YannDubs1](https://discuss.pytorch.org/u/YannDubs1)\
**Post date:** [April 19, 2019, 12:37pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/6 "2019-04-19T12:37:01Z")

</div>

> [@Damien\_Lancry](#):
>
> torch.sqrt(torch.zeros(1))

Of course, the issue is during the backward pass as you multiply 0 by infinity (derivative of sqrt at 0).

```auto
>>> mse = nn.MSELoss()
>>> yhat = torch.zeros(1, requires_grad=True)
>>> y = torch.zeros(1)
>>> loss = torch.sqrt(mse(yhat,y))
>>> loss.backward()
>>> yhat.grad
tensor([nan])

```

Using the simple module I wrote above

```auto
>>> rmse = RMSELoss()
>>> yhat = torch.zeros(1, requires_grad=True)
>>> y = torch.zeros(1)
>>> loss = rmse(yhat,y)
>>> loss.backward()
>>> yhat.grad
tensor([0.])

```

---

<div class="post-metadata">

**Author:** ![windson](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/windson/32/39596_2.png) [@windson](https://discuss.pytorch.org/u/windson)\
**Post date:** [July 15, 2019, 11:45am UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/7 "2019-07-15T11:45:01Z")

</div>

Hi, I wonder if that’s exactly the same as RMSE when dealing with batch size more than 1 tensor.  
i.e. target and prediction are [2,0,256,256] tensor  
MSE\_0 = MSE(prediction[0,:,:,:], target[0,:,:,:])  
MSE\_1 = MSE(prediction[1,:,:,:], target[2,:,:,:])

RMSE what we want is:  
SQRT( MSE\_0) + SQRT( MSE\_1)  
torch.sqrt(nn.MSELoss(x,y)) will give:  
SQRT( MSE\_0 + MSE\_1)  
so:  
sqrt(M1+M2) is not equals to sqrt(M1) + sqrt(M2)

with reduction is even off, we wanna  
Mean[Mean (sqrt (MSE\_0) ) + Mean(sqrt (MSE\_1) )]  
what will get with reduction = ‘mean’ instead, I think is:  
sqrt (Mean(MSE\_0) + Mean(MSE\_1) )  
so:  
[sqrt(M1) / N + sqrt(M2)/N] /2 is not equals to sqrt (M1/N + M2/N)

please correct me if my understanding is wrong. Thanks 😉

---

<div class="post-metadata">

**Author:** ![JamesHowlettLA](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/j/f04885/32.png) [@JamesHowlettLA](https://discuss.pytorch.org/u/JamesHowlettLA)\
**Post date:** [September 18, 2019, 5:06pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/8 "2019-09-18T17:06:46Z")

</div>

Try to add eps, such as eps = 1e-8, according to your precision.,

---

<div class="post-metadata">

**Author:** ![sudhanshu\_rastogi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/sudhanshu_rastogi/32/62232_2.png) [@sudhanshu\_rastogi](https://discuss.pytorch.org/u/sudhanshu_rastogi)\
**Post date:** [January 26, 2025, 4:51am UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/10 "2025-01-26T04:51:42Z")

</div>

This implementation is according to the definition of RMSE Error

```auto
class RMSELoss(nn.Module):
    def __init__ (self):
        super(). __init__ ()
        self.eps=1e-6

    def forward(self,ground_truth,prediction):
        loss=torch.mean(torch.sqrt(torch.sum(torch.square(ground_truth-prediction),axis=-1))) + self.eps
        return loss

```

---

<div class="post-metadata">

**Author:** ![Pytorch-Noob](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/pytorch-noob/32/78054_2.png) [@Pytorch-Noob](https://discuss.pytorch.org/u/Pytorch-Noob)\
**Post date:** [August 28, 2025, 5:58am UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/11 "2025-08-28T05:58:03Z")

</div>

So in summary one one should implement **[ptrblck](https://discuss.pytorch.org/u/ptrblck)**’s implementation by simply taking a square root over the mean square error. However, you need to be aware of Nan resulting in the backward pass.

Hence incredibly important to follow **[YannDubs1](https://discuss.pytorch.org/u/yanndubs1)**’s advice and add a very small non-zero number like 1e-6 or even smaller like 1e-8. This does not effect neuron’s weights as this small number has vanishing nature aka neurons just ignore it.

So the **simplest solution** is to add following code to **ptrblck** ’s implementation solution:

`criterion = nn.MSELoss()`  
`loss = torch.sqrt(criterion(x, y) + eps) # added eps to be 1e-6`

Correct this Noob if wrong!

---

<div class="post-metadata">

**Author:** ![c-foschi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/c-foschi/32/78555_2.png) [@c-foschi](https://discuss.pytorch.org/u/c-foschi)\
**Post date:** [October 28, 2025, 1:52pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/12 "2025-10-28T13:52:15Z")

</div>

You should be warned that minimising RMSE, as @windson correctly assessed, is NOT the same as minimising MSE, and that one should always use MSE as loss function, because RMSE is just the square root of MSE computed over batches, which doesn’t make any sense if batches are randomized as in SGD.  
If someone wanted to compute RMSE over single data points instead, that would just be absolute errors (as long as output is a scalar - because no averaging is actually performed) and therefore MAE is recommended.

---

<div class="post-metadata">

**Author:** ![c-foschi](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/c-foschi/32/78555_2.png) [@c-foschi](https://discuss.pytorch.org/u/c-foschi)\
**Post date:** [October 28, 2025, 2:28pm UTC](https://discuss.pytorch.org/t/rmse-loss-function/16540/13 "2025-10-28T14:28:59Z")

</div>

Btw, I found this thread while searching for a method to safely record and log RMSE instead of MSE on the test set.  
If you are using lightning, like I do, that method exists and is `torchmetrics.regression.MeanSquaredError(squared= False)`. This can be used for training too, as the gradients will still be correctly computed on the MSE. This class will however accumulate the computed MSE and present the square root when logging it.