Hi!
I’m trying to learn a differential equation from data, for that I defined a net that approximates the derivative at a point. I then use the output of the net to approximate the next grid-point of the solution to the differential equation and use that as an input for the next step. The function calls of the net are therefore recursive. Now I also know already pretty exactly the formula of the differential equation i want to approximate and do some feature extraction in every step, precisely building some products, powers, squares, etc with the output of the net and additional parameters.
class ODEMultistep(nn.Module):
def __init__(self, args):
super(ODEMultistep, self).__init__()
self.fnn1 = Derivative_FNN().to(args.device)
self.fnn2 = Superpos_FNN(args.steps).to(args.device)
self.args = args
def forward(self, nr, iv, k, stop):
ni_pred = torch.zeros(nr.size(0), stop).to(self.args.device)
ni_pred[:, :self.args.steps] = iv.clone().detach()
ni_pred.requires_grad = True
# this loops over all available nodes and calculates the next node
for j in range(self.args.steps, stop - self.args.steps):
solutions = []
# this calculates all previous derivatives
for l in range(j - self.args.steps, j):
t1 = nr[:, l] / ni_pred[:, l] * nr[:, l] # here calculate features
td = nr[:, l].pow(2)
td -= ni_pred[:, l].pow(2) # in this line the gradient gets lost
t2 = k * nr[:, l]
t2 *= torch.sqrt(td)
t1, t2 = t1.view(-1, 1), t2.view(-1, 1)
derivative = self.fnn1(t1, t2)
solutions.append(derivative)
this_derivative = self.fnn2(solutions)
ni_pred[:, j] = ni_pred[:, j - 1] + self.args.stepsize * this_derivative
return ni_pred
Now I get this error message when running the code:
sys:1: RuntimeWarning: Traceback of forward call that caused the error:
File "/home/felixwagner/MEGAsync/Projektarbeit/Code/train_ODEMultistep.py", line 164, in <module>
train(args, model, criterion, train_loader, optimizer, epoch)
File "/home/felixwagner/MEGAsync/Projektarbeit/Code/lib/utils_ODEMultistep.py", line 27, in train
stop=i)
File "/home/felixwagner/miniconda3/envs/felix_ml/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
result = self.forward(*input, **kwargs)
File "/home/felixwagner/MEGAsync/Projektarbeit/Code/lib/utils_ODEMultistep.py", line 101, in forward
td -= ni_pred[:, l].pow(2)
Traceback (most recent call last):
File "/home/felixwagner/MEGAsync/Projektarbeit/Code/train_ODEMultistep.py", line 164, in <module>
train(args, model, criterion, train_loader, optimizer, epoch)
File "/home/felixwagner/MEGAsync/Projektarbeit/Code/lib/utils_ODEMultistep.py", line 36, in train
loss.backward()
File "/home/felixwagner/miniconda3/envs/felix_ml/lib/python3.7/site-packages/torch/tensor.py", line 107, in backward
torch.autograd.backward(self, gradient, retain_graph, create_graph)
File "/home/felixwagner/miniconda3/envs/felix_ml/lib/python3.7/site-packages/torch/autograd/__init__.py", line 93, in backward
allow_unreachable=True) # allow_unreachable flag
RuntimeError: one of the variables needed for gradient computation has been modified by an inplace operation: [torch.cuda.FloatTensor [1]], which is output 0 of SelectBackward, is at version 3; expected version 2 instead. Hint: the backtrace further above shows the operation that failed to compute its gradient. The variable in question was changed in there or anywhere later. Good luck!
Somehow the gradient gets destroyed when I pow(2) the outputs of the net and then re-insert it for computing the next step in the differential equation. Can somebody help me finding a solution to this problem? Can I do anything about it?
