Grad is None in simple example

autograd
1

Hello,
I modified the basic autograd example from the pytorch tutorial (found here: https://pytorch.org/tutorials/beginner/blitz/autograd_tutorial.html#sphx-glr-beginner-blitz-autograd-tutorial-py) and I now get a weird error where the calculated gradient is None. Code is as follows:

if torch.cuda.is_available():
    device=torch.device("cuda:1")
    x=torch.arange(4,device=device,dtype=torch.float32,requires_grad=True).view(2,2)
    print(x)
    y=x+2
    z=y*y*3
    out=z.mean()
    out.backward()
    print(x.grad)
    print(x.requires_grad)

Output is

tensor([[0., 1.],
        [2., 3.]], device='cuda:1', grad_fn=<ViewBackward>)
None
True

All I changed from the tutorial is x. What happened here? Also, slightly OT, but what is the tensor I’m able to pass to the backward function? It’s really not well explained in the tutorial.

2

From the tutorial the folllowing seems to apply here :

If Tensor is a scalar (i.e. it holds a one element data), you don’t need to specify any arguments to backward() , however if it has more elements, you need to specify a gradient argument that is a tensor of matching shape.

3

What do you mean by apply? My tensor z is a scalar, so leaving it blank should be ok. It’s the same in the tutorial.

4

Yes you are right, my bad.

view(2,2) applied on x is causing the problem. This makes x a non-leaf variable.
If that is removed the code works.

You may also add one more step which applies view operation on x, then also it will work.

1 Like