Stack tensors keeping their automatic gradient information

autograd
1

Hi,

I want to stack some tensors keeping their automatic gradient information but screw up.
I have the following script where I want to compute the derivative with respect to x:

_k_plusplus = []
for aplus_idx, aplus_val in enumerate(aplus):
    state_plus = torch.tensor([x[0], aplus_val])[None, :]
    pred_mean = likelihood(gp_kplus(state_plus)).mean
    _k_plusplus.append(pred_mean)
    k_plusplus = torch.stack(_k_plusplus).flatten()

where print(k_plusplus) returns:

tensor([0.3481, 0.3481, 0.3481, 0.3481, 0.3481], dtype=torch.float64,
       grad_fn=<ViewBackward>)

It seems to be OK for me, but with this syntax, torch.autograd.gradcheck retunrs False.
Here note that pred_mean = likelihood(gp_kplus(state_plus)).mean is coming from GPyTorch.
print(pred_mean) retuns:

tensor([0.2027], dtype=torch.float64, grad_fn=<ViewBackward>)

again it seems to be reasonable for me.

My guess is when I stack the tensors by torch.stack or maybe at some points, I lose the automatic gradient information.
When I simplify the script:

k_plusplus = torch.empty(aplus.shape)
for aplus_idx, aplus_val in enumerate(aplus):
   k_plusplus[aplus_idx] = torch.tensor(0.1)

It passes the gradient check without changing the rest of my code.

Thank you in advance for your kind help.

2

Sorry this is my bad.
In the above snippet, type(x) is torch.tensor and I overwrite the tensor object by torch.tensor(x[0]...) where I lost the grad_fn information, I guess.
When I modify

state_plus = torch.stack([x[0], aplus_val])[None, :]

everything works well.