What is `in-place operation`?

3

I initially found in-place operations in the following PyTorch tutorial:

Adding two tensors

import torch

>>> x = torch.rand(1)
>>> x

 0.2362
[torch.FloatTensor of size 1]


>>> y = torch.rand(1)
>>> y

 0.7030
[torch.FloatTensor of size 1]

Normal addition

# Addition of two tensors creates a new tensor.
>>> x + y

 0.9392
[torch.FloatTensor of size 1]


# The value of x is unchanged.
>>> x

 0.2362
[torch.FloatTensor of size 1]

In-place addition

# An in-place addition modifies one of the tensors itself, here the value of x.
>>> x.add_(y)

 0.9392
[torch.FloatTensor of size 1]


>>> x

 0.9392
[torch.FloatTensor of size 1]
15 Likes