I have an encoder-decode model and a function that calculating something iteratively:
f(a,b,c): # a: [batchsize, n, channel] res_a = torch.empty(a.shape) res_b = torch.empty(b.shape) for i in range(a.shape): res_a[:,i] = balabala res_b[:,i] = balabala return torch.cat([res_a,res_b],-1)
This is the main procedure of forward function:
def forward(x): a, b, c = x[0:i], x[i:j], x[j:k] q = f(a,b,c) x' = decoder(encoder(x)) a, b, c = x'[0:i], x'[i:j], x'[j:k] q' = f(a,b,c) return q, q'
The loss is the difference of q and q1
Then an error occurs when backward for the computation of
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.FloatTensor [64, 3]], which is output 0 of SelectBackward, is at version 21; expected version 20 instead. Hint: enable anomaly detection to find the operation that failed to compute its gradient, with torch.autograd.set_detect_anomaly(True).
I really have no idea where I modify the variables for gradient computation, all I do in function
f is just indexing some data from input and conduct some computations (add, multiply), then store the result to a new tensor.