RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 512, 4, 4]] is at version 3; expected version 2 instead

I’m training my GAN code(pix2pixHD code

) and I have an error about inplace operation.
how can I correct this problem?

What is the output when you set

I didn’t set the torch.autograd.set_detect_anomaly(True).
Where should I write the code?

I think you can set it any time before you start model training.

Thanks for the advice!
I set the detect _anomaly code at the start of the training part and the here is the error.

Ok, it looks like the issue is in a conv layer of the model. Can you show the model definition?

Here is the layer code I implemented.


My best guess here would be that you compute all the forward and losses first, then you do a serie of backward/step for each loss one after the other?
The problem is that the step function modifies the parameters of your net inplace, but these values are needed to run the backward (by conv for example).

You want to make sure that you don’t do a step operation between the forward and the backward.