Grad value when copying tensor

autograd
1

I have a question about grad attribute.

I observed that grad value is not copied when using tensor slicing, clone(), to(), cuda() etc.

how can I copy tensors with grad value?

Below is the example:

a = torch.tensor([1., 2.], requires_grad=True)
loss = a.sum()
loss.backward()
b = a.cuda()    # change device
c = a[:1]       # slicing
d = a.clone()   # clone

print(a.grad)  # [1., 1.]
print(b.grad)  # None
print(c.grad)  # None
print(d.grad)  # None

Thank you.

2

The mentioned operations would create a differentiable operation and are thus not copying the .grad attribute. To do so, you could manually copy it back to the .grad attribute of the new tensor.

3

I solved this issue with your help.

Thank you!