Both of them seems to return an uninitialized tensor with specified size
Hi,
torch.empty only returns uninitialized Tensor, torch.Tensor has more arguments: for examples, you can give it a python list and it will build a Tensor from it.
Sorry I didn’t make the question clear – what I was trying to ask is actually, are there any differences between these two APIs if we just want to create an uninitialized tensor?
Ho, in that case no there is no difference. For code clarity, you should probably use empty as it is clearer that you want and get a uninitialized Tensor.
I see, thanks for answering!
Can you clarify what “uninitialized” means?
Perhaps I am being pedantic, but I see the implementation returns small numbers rather than zero’s. Is the semantics of uninitialized intended to be zero?
I guess what I am really after is why it’s not exactly zero.
I printed some outputs and it was more confusing than clarifying
>>> torch.empty(3, dtype=torch.long)
tensor([ 140187750138624, 140187750063648, -5764607523034234880])
>>> torch.empty(3)
tensor([0.0000e+00, 1.4013e-45, 0.0000e+00])
Questions:
- the second one is trying to be zero. Why can’t it be exactly zero? I thought it was possible to represent zero exactly in float/double etc.
- why is the first one nothing resembling zero? What is the intended semantics there?
1 Like
Uninitialized means that the memory will be allocated and the corresponding tensor will represent its values using the bits which were already in the memory block before it was allocated.
This means the values should never be used without a proper initialization, as they might represent “everything” (valid values, NaN, Inf).
If you want to initialize the tensor with zeros, use torch.zeros.
The advantage of torch.empty is that it’s faster, because it will just assign the memory to the tensor without filling it with specific values. This is often used, if you need a buffer, which is guaranteed to be filled with other values later (or initialized somehow).
5 Likes