`torch.Tensor` generation is not random

>>> import torch
>>> torch.Tensor(5)
tensor([3.7398e-14, 1.8478e+20, 8.3280e+35, 1.8888e+31, 1.2872e+22])
>>> torch.Tensor(5)
tensor([2.6207e-20, 3.0620e-41, 1.7231e-18, 3.0620e-41, 8.9683e-44])
>>> torch.Tensor(5)
tensor([1.7242e-18, 3.0620e-41, 2.6306e-20, 3.0620e-41, 0.0000e+00])
>>> torch.Tensor(5)
tensor([1.7243e-18, 3.0620e-41, 1.7233e-18, 3.0620e-41, 1.1210e-43])
>>> torch.Tensor(5)
tensor([2.1218e-14, 4.5569e-41, 2.1218e-14, 4.5569e-41, 4.4842e-44])
>>> torch.Tensor(5)
tensor([0., 0., 0., 0., 0.])
>>> torch.Tensor(5)
tensor([1.7220e-18, 3.0620e-41, 1.7242e-18, 3.0620e-41, 1.1210e-43])
>>> 

As you can see it is not entirely random, a lot of values are repeating

>>> import torch
>>> torch.manual_seed(412)
<torch._C.Generator object at 0x7fa74dd01b70>
>>> torch.Tensor(4)
tensor([0., 0., 0., 0.])
>>> 
>>> torch.Tensor(4)
tensor([1.0700e+09, 4.5793e-41, 1.0700e+09, 4.5793e-41])
>>> torch.Tensor(4)
tensor([-3.3252e+28,  3.0782e-41, -3.3290e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([-3.3304e+28,  3.0782e-41, -3.3302e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([ 5.7405e+17,  7.0065e-45, -3.3290e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([-3.3310e+28,  3.0782e-41, -3.3308e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([-3.3288e+28,  3.0782e-41,  1.0700e+09,  4.5793e-41])
>>> torch.Tensor(4)
tensor([-3.3299e+28,  3.0782e-41, -3.3311e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([-3.3303e+28,  3.0782e-41, -3.3282e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([ 1.0700e+09,  4.5793e-41, -3.3306e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([ 5.7405e+17,  7.0065e-45, -3.3282e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([ 1.0700e+09,  4.5793e-41, -3.3310e+28,  3.0782e-41])
>>> torch.Tensor(4)
tensor([ 5.7405e+17,  7.0065e-45, -3.3282e+28,  3.0782e-41])
>>> 

same when I manually seed

I just I am an idiot, torch.Tensor is not to be a random generator

torch.randn is instead

>>> torch.manual_seed(10)
<torch._C.Generator object at 0x7f9583585b70>
>>> torch.randn(5)
tensor([-0.6014, -1.0122, -0.3023, -1.2277,  0.9198])
>>> torch.randn(5)
tensor([-0.3485, -0.8692, -0.9582, -1.1920,  1.9050])
>>> torch.randn(5)
tensor([-0.9373, -0.8465,  2.2678,  1.3615,  0.0157])
>>> torch.manual_seed(10)
<torch._C.Generator object at 0x7f9583585b70>
>>> torch.randn(5)
tensor([-0.6014, -1.0122, -0.3023, -1.2277,  0.9198])
>>> torch.randn(5)
tensor([-0.3485, -0.8692, -0.9582, -1.1920,  1.9050])
>>> torch.randn(5)
tensor([-0.9373, -0.8465,  2.2678,  1.3615,  0.0157])
1 Like

Good to see you’ve narrowed down the issue. A quick explanation for the issue you’ve seen by using torch.Tensor:
torch.Tensor is not a factory method and will use uninitialized memory. Depending which memory location is picked, the values could be repeated.