How to get gradient of loss with respect to predicted output

1

I’m trying to get the gradient of the final output of a nn with respect to the loss function like so:

x = torch.FloatTensor(...)
x = Variable(x, requires_grad=True)
y = torch.FloatTensor(...)
y = Variable(y)

model = torch.nn.Sequential(
    torch.nn.Linear(D_in, H, False),
    torch.nn.ReLU(),
    torch.nn.Linear(H, D_out, False),
)

loss_fn = torch.nn.L1Loss(size_average=False)
y_pred = model(x)
loss = loss_fn(y_pred, y)
loss.backward()
print(y_pred.grad)

But the gradient comes back ‘None’. What am I doing wrong? For reference, I can get the linear layer’s weights just fine with a similar approach. Also, y_pred (created by model(x)) does have ‘requires_grad=True’

2

Gradients don’t accumulate in non-leaf nodes. You should register a backward hook on the model: http://pytorch.org/docs/0.3.1/nn.html?highlight=backward%20hook#torch.nn.Module.register_backward_hook to view the gradient of the loss wrt y_pred

3

Hey,

Did you figure out how to use it for finding the gradient respect to prediction?

@richard Can you provide an example to how to do this?