I use the following code snippet to calculate attention weights and get the new hidden state input for my RNN.
association = torch.mv(hidden_states_other[l], hidden_states[i]) probs = torch.nn.functional.softmax(association) attention_val = torch.mv(torch.t(hidden_states_other[l]), probs) hidden_states_weights[i] = attention_val
Running this, gives me the error
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
I don’t see any inplace operation. Can someone explain why this is happening?
hidden_states : N x R tensor hidden_states_other : N x R tensor l : M, tensor hidden_states_weights : N x R tensor