What is `in-place operation`?

Yes true they both modify x. But in-place operation does not allocate new memory for x.

Eg. Normal operation vs In place operation

>>> x = torch.rand(1)
>>> y = torch.rand(1)
>>> x
tensor([0.2738])
>>> id(x)
140736259305336
>>> x = x + y   # Normal operation
>>> id(x)
140726604827672 # New location
>>> x += y
>>> id(x)
140726604827672 # Existing location used (in-place)
28 Likes