Inplace Operations when using Pyrfm

[KFrank]

Hi Huzaifa!

@Huzaifa_Capricorn

            loss.backward(retain_graph = True)
            optimizer.step()

I get the following error when I use a NN but no error when using BinaryLogisticRegression:

 RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [3, 1]], which is output 0 of TBackward, is at version 100; expected version 65 instead.

The likely immediate cause of your inplace-modification error is your use of
retain_graph = True. Try removing it.

Note that optimizer.step() counts as an inplace operation, so you normally
don’t want the “retained graph” still around when you call it.

If you think that you actually do need retain_graph = True, make sure you
that you understand why so that you can either avoid it or arrange things to
avoid the inplace-modification error.

I don’t really understand what your code is doing – you can find some suggestions
for debugging inplace-modification errors in this post:

"RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 1]], which is output 0 of AsStridedBackward0, is at version 3; expected version 2 instead. Hint: the backtrace further a

Hi Fahmyadan and Sangyoon! Here are some suggestions about how to track down (and maybe fix) inplace-modification errors. Note that an inplace modification in the forward pass is not necessarily* an error – it depends on whether and how the tensor that was modified is used in the backward pass. Note that inplace operations can be useful for saving memory – if you replace an innocent inplace operation with an out-of-place equivalent, your training will use more memory (and, to a minor e…

Good luck!

K. Frank