RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 50]], which is output 0 of AsStridedBackward0, is at version 2; expected version 1 instead

[KFrank]

Hi Jun!

@Jun_Park

        h_out = torch.zeros(2, x.shape[0], self.hidden_size, device=x.device)

        h_out[0] = self.gru_0(x, h_in[0])
        h_out[1] = self.gru_1(h_out[0], h_in[1])

Assigning into a tensor using indexing is an inplace operation, and this
is likely the cause of your error.

Try replacing the code I quoted with

h_0 = self.gru_0(x, h_in[0])
h_out = torch.stack (h_0, self.gru_1(h_0, h_in[1])

If that doesn’t fix your problem, take a look at this post that gives some
suggestions for debugging inplace-modification errors:

"RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 1]], which is output 0 of AsStridedBackward0, is at version 3; expected version 2 instead. Hint: the backtrace further a

Hi Fahmyadan and Sangyoon! Here are some suggestions about how to track down (and maybe fix) inplace-modification errors. Note that an inplace modification in the forward pass is not necessarily* an error – it depends on whether and how the tensor that was modified is used in the backward pass. Note that inplace operations can be useful for saving memory – if you replace an innocent inplace operation with an out-of-place equivalent, your training will use more memory (and, to a minor e…

Best.

K. Frank