I’ve been seeing underscores at the end of many functions in pytorch. What does this mean?
I think I read in some chat or question that it means operations are done “in place” but I wanted to make sure that was correct (and provide a way for future people to search for this question) and to clarify what “in place” meant.
This has been replied in How does one make sure that the parameters are update manually in pytorch using modules?
From fmassa’s answer somewhere else (source: How does one make sure that the parameters are update manually in pytorch using modules?):
Whenever we have an underscore in the end of a function in pytorch, that means that the function is in-place.
If you you don’t know what in place means see albanD’s answer (source: What is `in-place operation`?):
An in-place operation is an operation that changes directly the content of a given Tensor without making a copy. Inplace operations in pytorch are always postfixed with a
_, like.add_()or.scatter_(). Python operations like+=or*=are also inplace operations.
as a side comment, these sort of operations don’t get added to the computation graph, so they should be used with caution.