When I use nn.ReLU(inplace=True), there is an error that is:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
But this error just appears in some places. Why it occurs?
Because the inplace op overwrites some data that’s needed by some Function to compute the gradient. It has no way of doing that after you overwrite it. Just remove inplace=True in these places.
Ok. Thank you. It works after removing it.
I had the same issue, and noticed this post a bit late.
Could you automatically make inplace=False in train mode of models?
It is the default value. Just don’t override it.
2 Likes