Explanation for unit test of hessian-vector product

Could someone please explain why x_hv and y_hv are being multiplied by 5 and 4, respectively?

 def test_hessian_vector(self):
    x = Variable(torch.randn(2, 2), requires_grad=True)
    y = Variable(torch.randn(2, 2), requires_grad=True)

    z = x ** 2 + y * x + y ** 2
    z.backward(torch.ones(2, 2), create_graph=True)

    x_grad = 2 * x.data + y.data
    y_grad = x.data + 2 * y.data
    self.assertEqual(x.grad.data, x_grad)
    self.assertEqual(y.grad.data, y_grad)

    grad_sum = 2 * x.grad + y.grad
    grad_sum.backward(torch.ones(2, 2))
    x_hv = torch.ones(2, 2) * 5
    y_hv = torch.ones(2, 2) * 4
    self.assertEqual(x.grad.data, x_grad + x_hv)
    self.assertEqual(y.grad.data, y_grad + y_hv)

Could someone please explain why x_hv and y_hv are being multiplied by 5 and 4, respectively?

Effectively, x.grad = 2 * x + y, y.grad = x + 2 * y.

Because of this, the amount that x contributes to grad_sum is 4 through the 2 * x.grad term and 1 through the y_grad term. Hence x_hv = 5 * torch.ones(2, 2).

The amount that y contributes to grad_sum is 2 through the 2 * x.grad term and 2 through the y.grad term. Hence the factor of 4.

1 Like

Thanks. I have another question. What is grad_sum? Is that just an arbitrarily defined function, or does that fall out of the previous work above it?

We’ve arbitrarily defined grad_sum = 2*x.grad + y.grad

1 Like