Runtime Error: One of the variables needed for gradient computation has been modified by an in place operation

I get the Runtine Error only when I run on a single gpu. The error magically disappears when I run it on multi gpu. Any suggestions?


This error happens when you perform inplace operations on tensors whose values are needed by another op for computing gradients.
Inplace ops can be t[indices] = xxx or t += xxx or relu(t, inplace=True) etc