I want to update parameters a and b that build up a set of tensors. For simplicity assume this MRC:

import torch
a = torch.tensor([100.], requires_grad=True)
b = torch.tensor([50.], requires_grad=True)
c = torch.tensor([a/10, b/10], requires_grad=True)
c.retain_grad()
optim = torch.optim.Adam([a,b], lr=0.1)
d = torch.tensor([5,5])
for i in range(10):
L = (c-d).mean()
L.backward()
optim.step()
optim.zero_grad()
print(L.item())

The computation graph looks like this. So it seems to end before reaching a and b and I do not understand why.

unfortunately, I don’t know why a and be are not populated with gradients and why there is no update happening.
Any help would be much appreciated.
Thank you

c is a new leaf-tensor and there is no computation graph to a and b since you’ve re-created a tensor. Use torch.stack or torch.cat to create c instead if you want gradients in a and b.

Thank you, this is a valid point.
With further testing, I also realized that the torch.cat operation needs to be performed for every optimization loop to not break the graph and update the values properly.

(I try to keep this information here for reference if someone stumbles upon the same errors)

Code that works as expected:

import torch
a = torch.tensor([100.], requires_grad=True)
b = torch.tensor([50.], requires_grad=True)
optim = torch.optim.Adam([a, b], lr=0.1)
d = torch.tensor([5, 5], dtype=torch.float32) # Make sure the dtype matches
losses = []
for i in range(1000):
c = torch.cat([a, b])/ 10
L = torch.nn.functional.mse_loss(c, d)
L.backward()
optim.step()
optim.zero_grad()
losses.append(L.item())
print("Updated a:", a.item())
print("Updated b:", b.item())