Does my loss function need a tensor with a gradient?

Hi,

so my Neural Network’s forward() returns a tensor of shape: z.size() torch.Size([64, 1]) whereas

z=
tensor([[0.000],
[0.264],

[0.000]], grad_fn=)

Now my original y is of shape y.size()= torch.Size([64]) looking like

y = tensor([0., 0., 0. … 0. ])

Now my y_pred = model(X) has shape torch.Size([64, 1]).

Now I want to do loss = loss_fn(y_pred, y) but I need the same dimensions. I know I could use z.view(-1) to get the correct reshaping but then I loose grad_fn= no? Isn’t that problematic for the backpropagation?

Sorry for the missing formatting but I don’t get that forum here. Does it support markdown? Latex?

Hi Hyt!

No, you won’t lose your grad_fn when using .view(). Here is a
short example:

'1.9.0.dev20210504'
>>> t = torch.randn (4, 1, 2, requires_grad = True)
>>> y_pred = t.sum (dim = 2)
>>> y_pred
tensor([[ 0.6060],
        [-0.4268],
        [-0.6957],
        [ 1.3155]], grad_fn=<SumBackward1>)
>>> y_pred.view (-1)
tensor([ 0.6060, -0.4268, -0.6957,  1.3155], grad_fn=<ViewBackward>)

It is true that the “root node” of the computation graph now has
grad_fn=<ViewBackward>, but the grad_fn=<SumBackward1>
(in my example) is still there – it’s just one node up from the root in
the computation graph and not printed out as part of the root-node
tensor.

To answer the question in your title: Yes, to backpropagate usefully
you do need to feed a tensor with a gradient (requires_grad = True)
to your loss function. But, as noted above, .view() doesn’t spoil this.

Best.

K. Frank