Simply, I have a few variables, say
x1, x2, x3, with
requires_grad set to
True. Using torch.Tensor resulted in removing all the
ipdb> x1 tensor(0.6684, device='cuda:0', grad_fn=<L1LossBackward>) ipdb> x2 tensor(0.7662, device='cuda:0', grad_fn=<L1LossBackward>) ipdb> torch.Tensor((x1, x2)) tensor([0.6684, 0.7662])
I need to concatenate the variables into one tensor and preserve the grad history, so that I can use
backward pass later.
I have tried using torch.Tensor( (x1, x2, x3) ) but all the grad history has been lost.
torch.cat((x1,x2,x3)) did not work too, throwing the error message “RuntimeError: zero-dimensional tensor (at position 0) cannot be concatenated” .
One way around this, is to define the tensor
z = torch.ones(3, dtype=torch.float32, requires_grad=True).to(device)
z = z*x1 z = z*x2 z = z*x3
Any, ideas, alternatives?