Variable modified by an inplace operation error although using non-inplace operation

[KFrank]

Hi Sam!

@Sam-gege

what about other functions that also requires input data for gradient calculation, such as sqrt (df/dx=0.5/sqrt(x))?

The point here is that sqrt() saves its output, rather than its input, for
use in the backward pass. (sqrt (x) could save its input, x, but then
in would have to recompute sqrt (x) from x in order to compute its
gradient. To be a little more efficient, it stores its output, sqrt (x),
instead.)

Try modifying sqrt (x)'s output, y, inplace, and see what happens.

Here is a post that goes through a few more examples of this sort of
thing:

When inplace operation are allowed and when not

Hi Jie (and Srishti)! The part that can be confusing is that a given tensor used in the forward pass may or may not be used in the backward pass. It depends on the math of the gradient computation (and the details of how the relevant backward() function is implemented). It’s not always obvious. Here are some explanatory examples: The derivative of exp (x) is exp (x). So it makes sense that pytorch saves the output of exp (x) when it is computed during the forward pass so that it can …

Best.

K. Frank