From fmassa’s answer somewhere else (source: How does one make sure that the parameters are update manually in pytorch using modules?):
Whenever we have an underscore in the end of a function in pytorch, that means that the function is in-place.
If you you don’t know what in place means see albanD’s answer (source: What is `in-place operation`?):
An in-place operation is an operation that changes directly the content of a given Tensor without making a copy. Inplace operations in pytorch are always postfixed with a
_, like.add_()or.scatter_(). Python operations like+=or*=are also inplace operations.
as a side comment, these sort of operations don’t get added to the computation graph, so they should be used with caution.