Pytorch’s autograd facility – which automatically calculates the gradients
that are then used to update your network weights when you call optimizer.step() – only tracks computations performed with pytorch
So as soon as you call numpy() (and detach(), for that matter) you
have “broken the computation graph” and pytorch can no longer
backpropagate through those calculations.
Rewrite the manipulations that you are performing with numpy using
pytorch tensor operations, and autograd should work and your weights
Thank you for your response, I have updated the training loop as such:
for epoch in range(EPOCHS):
for dd in range(0, len(x_train) - batch):
X = x_train[dd:dd + batch]
output = dpc(X)
input_ssm = torch.empty((1,21))
for t in range(0,20):
input_ssm[t] = X[t]
input_ssm = output
y = ss_model(input_ssm)
y = (y*(max_a - min_a)) + min_a
desired = torch.Tensor([])
loss = (F.mse_loss(y, desired))
combined_loss += loss
Yet the issue still persists, and the weights are not updating, am I doing this incorrectly?
First, let me note that this second version you posted no longer has loss.backward() in it. If that’s not just a typo, it would prevent your
weights from being updated.
Second, please post a simplified, fully-self-contained, runnable script
that illustrates your issue, together with its output. For example, it looks
like what you posted has two models in it. Can you reproduce your issue
with only one model?
I would also suggest that you try a single forward and backward pass
(rather than the loops over epochs). Does a single pass create grads
for your model weights (e.g., my_model.fc1.weight.grad)? Are these
gradients non-zero? Does optimizer.step() then modify the weights?