What's the benefit of using "out=" parameter in tensor operations?

1

In fairseq code, I saw some use of the “out=” argument for tensor operations similar to below. Are there any performance benefits of using “out=” parameter? It seems less natural and less readable. Does it offer performance gain like avoiding generating intermediate results? Or is it just good for specifying the destination tensor to store the result?

c = torch.tensor([])
a = torch.rand(3, 4)
torch.add(a, 3, out=c)   # Does this have any advantage over the equivalent way below?
# this is equivalent to the below
c[:] = a + 3

c = torch.empty(12)
a = torch.rand(3, 4)
torch.add(a, 3, out=c.view(3, 4))  # Does this have any advantage over the equivalent way below?
# the above is equivalent to the below
c[:] = (a + 3).view(12)
# or
c.view(3, 4)[:] = a + 3
2

If you create an empty tensor, then give it as out=, then it will be resized and filled with the result. This will be the same as c = a + 3 in your first example.
Doing c[:] = a + 3 will create an intermediary result and copy it into c (which will work only if c has been resized beforehand).

If can be usefull to prevent your operation to create a new tensor if you already have a Tensor of the right size that is not needed anymore.
Unless you work with very large Tensors or very specific use case, the performance gain is not going to be noticeable.

1 Like