Loosely, tensors you create directly are leaf variables. Tensors that are the result of a differentiable operation are not leaf variables
For example:
w = torch.tensor([1.0, 2.0, 3.0]) # leaf variable
x = torch.tensor([1.0, 2.0, 3.0], requires_grad=True) # also leaf variable
y = x + 1 # not a leaf variable
(The PyTorch documentation for is_leaf contains a more precise definition.)
An in-place operation is something which modifies the data of a variable. For example:
x += 1 # in-place
y = x + 1 # not in place
PyTorch doesn’t allow in-place operations on leaf variables that have requires_grad=True (such as parameters of your model) because the developers could not decide how such an operation should behave. If you want the operation to be differentiable, you can work around the limitation by cloning the leaf variable (or use a non-inplace version of the operator).
x2 = x.clone() # clone the variable
x2 += 1 # in-place operation
If you don’t intend for the operation to be differentiable, you can use torch.no_grad:
with torch.no_grad():
x += 1