I use the following code snippet to calculate attention weights and get the new hidden state input for my RNN.
association = torch.mv(hidden_states_other[l], hidden_states[i])
probs = torch.nn.functional.softmax(association)
attention_val = torch.mv(torch.t(hidden_states_other[l]), probs)
hidden_states_weights[i] = attention_val
Running this, gives me the error
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation
I don't see any inplace operation. Can someone explain why this is happening?
hidden_states : N x R tensor
hidden_states_other : N x R tensor
l : M, tensor
hidden_states_weights : N x R tensor