I would like to use an attention like this;
attention = torch.relu(self.w_fc1(x)) self.x = self.x * torch.softmax(self.w_fc2(attention), dim=0)
where x is input vector.
#I removed avg_pool of 1st stage in attention.
This is in forward path description.
self.register_buffer('x', torch.stack([torch.zeros(NUM_INPUT) for _ in range(NUM_HIDDEN)]))
self.w_fc1 = nn.Linear(NUM_INPUT, NUM_INPUT) self.w_fc2 = nn.Linear(NUM_INPUT, NUM_INPUT)
Then I meet an error:
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor ] is at version 56; expected version 28 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
What does mean
version 56 ?
When I try to do like this;
attention = torch.relu(self.w_fc1(x)) self.x = attention
Same error is happen.
I have no idea to solve and where I made a wrong.
Can anyone point out, or suggest to solve?