Why I keep getting nan?

vision
1

Sorry guys, I have a silly question.I am using python3.5 and pytorch 3.1, when I type

torch.Tensor(1000, 1000).mean()

I get the following result:

>>> import torch
>>> torch.Tensor(1000, 1000).mean()
0.0
>>> torch.Tensor(1000, 1000).mean()
nan
>>> torch.Tensor(1000, 1000).mean()
nan

nan stands for not a number, right?
but when I type this:

>>> torch.Tensor(1000, 1000)

 0.0000e+00  4.3397e+02  0.0000e+00  ...   0.0000e+00  0.0000e+00  2.2500e+00
 0.0000e+00  0.0000e+00  0.0000e+00  ...   0.0000e+00  1.0842e-19  1.9776e+00
 0.0000e+00  0.0000e+00  0.0000e+00  ...   0.0000e+00  0.0000e+00  0.0000e+00
                ...                   ⋱                   ...                
 0.0000e+00  0.0000e+00  0.0000e+00  ...   0.0000e+00  0.0000e+00  0.0000e+00
 0.0000e+00  0.0000e+00  0.0000e+00  ...   0.0000e+00  0.0000e+00  0.0000e+00
 0.0000e+00  0.0000e+00  0.0000e+00  ...   0.0000e+00  0.0000e+00  0.0000e+00
[torch.FloatTensor of size 1000x1000]

The tensor is filled with numbers.

So, two questions:

  1. How come 1000000 numbers’ mean is not a number?

  2. And why sometimes I get nan, but sometimes I get numbers?Like this:

>>> torch.Tensor(1000, 1000).mean()
-1.0074733384568384e+32

Thanks in advance.

2

That constructor doesn’t do anything to the storage memory, it’s all uninitialized. So the data is just garbage.

1 Like
3

But still, garbage have a value that can be represented by a number, right? If they have a number, how come pytorch knows it’s garbage or a useful number? And why sometims I get a number for calculate the mean, and sometimes I get a nan?

4

The behavior of calling mean on an uninitialized tensor is undefined. I believe pytorch is interpreting the data as if it were valid numbers, which is why you get a result. However, there’s no guarantees for the data that is going to be in the tensor, so that’s why you get such unpredictable results. One possible explanation for why you get nan is one of the entries is interpreted by pytorch as nan, and the mean of a tensor with a nan element is always nan.

2 Likes