I’m running a model where I need to get the gradient of the loss function w.r.t my input data so I can use it to update previous networks that are in series. I thought I was calling everything right, but I have encounted a problem where the gradients of my loss function w.r.t input data is zero or very close to it. What am I doing wrong? The gradients exist throughout the network, loss is nonzero, etc.
Here’s the code for the backpropagation. after_graph is the output of a graph neural network, and net2 is my graph.
after_graph2 = Variable(after_graph.clone(), requires_grad=True) after_net2 = self.net2(after_graph2) # back prop loss = self.loss_func(after_net2, y) # find gradients of net 2 loss.backward(retain_graph = True) loss_graph_output = after_graph2.grad
Here’s the net:
net2 = nn.Sequential( nn.Linear(input_size2, hidden_sizes2), nn.Tanh(), nn.Linear(hidden_sizes2, output_size2), nn.Tanh() )
What could be going wrong?