One of the variables needed for gradient computation has been modified by an inplace operation

7

I should have asked for a stack trace as well. Nevertheless I will hasard a guess.

Have you tried replacing nn.ReLU(inplace=True) with nn.ReLU()?

Besides the line x = self.relu(x) is confused because it does two things.

  1. it applies relu to x inplace thus modifying the input data
  2. it then reassigns x to x which is pointless given that x has already been modified inplace.

The line out = self.relu(x) is worse because it does not make it clear that x is modified inplace by the relu operation. Readability is important.

My guess is that trying to optimise by using inplace operations will easily lead to confusion, frustration and probably not a huge improvement in efficiency. So I would suggest getting the code to work without inplace operations, and only then try adding some.

5 Likes