# Torch.var() and torch.std() return nan

**URL:** <https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884>\
**Category:** Uncategorized\
**Created:** [March 4, 2019, 2:27pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884 "2019-03-04T14:27:01Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![jmaronas](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jmaronas/32/2816_2.png) [@jmaronas](https://discuss.pytorch.org/u/jmaronas)\
**Post date:** [March 4, 2019, 2:27pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/1 "2019-03-04T14:27:01Z")

</div>

When torch tensor has only one element this call returns a nan where it should return a 0.

```auto
import torch
a=torch.tensor([0.2550])
print(a.var())

```

---

<div class="post-metadata">

**Author:** ![albanD](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/alband/32/215_2.png) [@albanD](https://discuss.pytorch.org/u/albanD)\
**Post date:** [March 4, 2019, 2:32pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/2 "2019-03-04T14:32:04Z")

</div>

Hi,

Isn’t the variance based on a set of n samples supposed to be:

```auto
var = sum_i (x_i - mean(x)) / (n-1)

```

---

<div class="post-metadata">

**Author:** ![jmaronas](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jmaronas/32/2816_2.png) [@jmaronas](https://discuss.pytorch.org/u/jmaronas)\
**Post date:** [March 4, 2019, 2:52pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/3 "2019-03-04T14:52:02Z")

</div>

well, that is the unbiased estimate of the maximum likelihood estimator of the variance of a Gaussian distribution as far as I know. The empirical variance is defined as:

```auto
1/N sum_i (xi -mean)**2

```

edit: I have checked over the internet and it seems that, under certain conditions, the unbiased estimate of the variance you mentioned also holds

---

<div class="post-metadata">

**Author:** ![rasbt](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rasbt/32/60_2.png) [@rasbt](https://discuss.pytorch.org/u/rasbt)\
**Post date:** [March 4, 2019, 3:26pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/4 "2019-03-04T15:26:45Z")

</div>

Sorry for nitpicking, but the variance is defined as E[(population mean - x)^2] and both formulas above are for empirical variances. But like you said, one is a biased and one an unbiased estimator.

I would prefer if PyTorch would implement the biased estimator without the Bessel’s correction term by default as it would then be similar to NumPy and would avoid the division by zero. In 99.9% of the cases in DL, we don’t care about biased/unbiased estimates of the variance anyway because our goal is to have unit variance for other numerical reasons.

That being said, I just see that it’s possible to turn it off:

```
function:: var(input, dim, keepdim=False, unbiased=True, out=None)
```

---

<div class="post-metadata">

**Author:** ![jmaronas](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jmaronas/32/2816_2.png) [@jmaronas](https://discuss.pytorch.org/u/jmaronas)\
**Post date:** [March 4, 2019, 3:32pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/5 "2019-03-04T15:32:59Z")

</div>

In pattern recognition and machine learning from Bishop page 94 I checked that this formula is the variance of the maximum likelihood estimate of a Gaussian distribution.

The empirical variance is defined as I mentioned, because in the limit of N-\>infinite it will converge to the true variance definition… Empirical statistics are statistics computed over the empirical distribution, instead of the true one. You could see it as a monte carlo estimate of the true variance.

---

<div class="post-metadata">

**Author:** ![rasbt](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rasbt/32/60_2.png) [@rasbt](https://discuss.pytorch.org/u/rasbt)\
**Post date:** [March 4, 2019, 3:44pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/6 "2019-03-04T15:44:33Z")

</div>

Will check later, but like you said, he is probably talking about an estimate (hence empirical). Gaussian distribution is is not involving discrete variables, hence you need an integral to define the non-empirical variance. So, everything with a “sum” is empirical as it is over a sample.

But in any case, this is beside the main point you raise with the 1/(n-1) term.

---

<div class="post-metadata">

**Author:** ![jmaronas](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/jmaronas/32/2816_2.png) [@jmaronas](https://discuss.pytorch.org/u/jmaronas)\
**Post date:** [March 4, 2019, 3:51pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/7 "2019-03-04T15:51:54Z")

</div>

Yes, that correction can be applied. Anyway I think that the empirical variance should be computed as I mentioned or at least make it explicit in the documentation.

---

<div class="post-metadata">

**Author:** ![rasbt](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/rasbt/32/60_2.png) [@rasbt](https://discuss.pytorch.org/u/rasbt)\
**Post date:** [March 4, 2019, 3:56pm UTC](https://discuss.pytorch.org/t/torch-var-and-torch-std-return-nan/38884/8 "2019-03-04T15:56:48Z")

</div>

I agree with you that “1/N sum\_i (xi -mean)\*\*2” should be the default. Then it would also avoid the “nan” issue you encounter by default (plus be more intuitive for people coming from NumPy)

Just checking the documentation:

```auto
.. function:: var(input, dim, keepdim=False, unbiased=True, out=None) -> Tensor
...
If :attr:`unbiased` is ``False``, then the variance will be calculated via the
biased estimator. Otherwise, Bessel's correction will be used.

Args:
...
    unbiased (bool): whether to use the unbiased estimation or not
...

```

It is mentioned there though.