I know that we can’t call backward on vector Tensor without reducing it to a scalar. However, I also need the gradient to be a vector, with each column is the gradient of each column of the Loss with respect to the parameters, i.e.,
import torch x = torch.tensor([4.0], requires_grad=True) L1 = torch.sin(x) * torch.cos(x) L2 = torch.sin(x**2) * torch.cos(x/2) L = torch.stack([L1, L2]) L.backward() print(x.grad)
I hope that x.grad equals [dL1/dx, dL2/dx].
Is there any way I can do this in a simple manner, without copy and pasting to a temporary variable multiple times (as many as the length of the Loss Tensor)?