Hi, I am reading this paper about pytorch . In section 3.1, they gave an example code:
y = x.tanh()
y.add_(3)
y.backward()
I could not understand the idea behind this code snippet. Does this snippet cause an error ? Or eventhough it does not cause an error, is it illegal to use such thing ?
Thanks
The idea is that since add_ is an inplace operation, in this case, y will be modified inplace and so the gradient computation for the tanh operation won’t be able to be performed.
This code snipped should cause an error during the backward pass. All these potential problems are tracked properly, so if no error is raised, the operation is valid and the result will be correct.