Gradient computation modified by an inplace operation

[KFrank]

Hi Daniel!

@Daniel_Maier

RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1, 128, 128]], which is output 0 of SqrtBackward0, is at version 1; expected version 0 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).

…

        squared_distance = torch.sum((query - key) ** 2, dim=-1)  # (B, N, N)

        # Take the square root to get the Euclidean distance
        euclidean_distance = torch.sqrt(squared_distance)
        return euclidean_distance

The error message you posted gives two useful clues: First, the shape of
the tensor being modified inplace is [1, 128, 128], and second, it is the
output of a sqrt() operation.

Such an error could occur in parts of your code that you haven’t posted, but
it could also be happening to euclidean_distance, the result of a sqrt()
operation. Is the shape of euclidean_distance [1, 128, 128]?

I would look at what happens to euclidean_distance after it is returned
from the forward() function you posted. Do you see any inplace operations
being applied to it in the code that is subsequently called?

If this is the issue, then returning a clone() of euclidean_distance, e.g.,
return euclidean_distance.clone(), might be sufficient to fix the problem.

For further suggestions about how to debug such issues, please see this post:

"RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 1]], which is output 0 of AsStridedBackward0, is at version 3; expected version 2 instead. Hint: the backtrace further a

Hi Fahmyadan and Sangyoon! Here are some suggestions about how to track down (and maybe fix) inplace-modification errors. Note that an inplace modification in the forward pass is not necessarily* an error – it depends on whether and how the tensor that was modified is used in the backward pass. Note that inplace operations can be useful for saving memory – if you replace an innocent inplace operation with an out-of-place equivalent, your training will use more memory (and, to a minor e…

Best.

K. Frank