Why the input and output gradient size of LPPool are the same

1 
def save_gradient(module, grad_input, grad_output):
    print(f"{module.__class__.__name__} input grad:\n{grad_input}")
    print(f"{module.__class__.__name__} output grad:\n{grad_output}")

m = nn.LPPool2d(3, 2, stride=2)
m.register_backward_hook(save_gradient)
input = torch.tensor([float(i) for i in range(1, 50)], requires_grad=True).reshape(1,1,7,7)
input = torch.transpose(input, 2, 3)

output = m(input)
output.backward(gradient=output)

I got:

LPPool2d input grad:
(tensor([[[[0.0309, 0.0107, 0.0063],
          [0.0248, 0.0097, 0.0059],
          [0.0206, 0.0089, 0.0056]]]]),)
LPPool2d output grad:
(tensor([[[[10.7722, 31.1708, 52.9788],
          [13.4294, 34.2547, 56.1203],
          [16.2184, 37.3533, 59.2653]]]], grad_fn=<PowBackward0>),)

If I don’t misunderstand the parameters of backward hook, here LPPool2d input grad size should be (1, 1, 7, 7), while output grad size should be (1, 1, 3, 3).

My torch version is 1.13.0+cu116. Actually I don’t know LPPool mechanism very clearly. I just found that the input and output gradient size of AvgPool or MaxPool are different, and I want to know the reason why. I have tried to search this question but I failed to find my answer I expect.

Thanks! :slight_smile:

3

Your code raises a warning:

# UserWarning: Using a non-full backward hook when the forward contains multiple autograd Nodes is deprecated and will be removed in future versions. This hook will be missing some grad_input. Please use register_full_backward_hook to get the documented behavior.

and I would recommend not to ignore it.
Using register_full_backward_hook shows:

LPPool2d input grad:
(tensor([[[[ 0.0928,  5.9412,  7.2183, 15.5274, 15.8743, 24.4626,  0.0000],
          [ 0.3713,  7.5194,  8.2128, 16.9710, 16.9879, 25.8405,  0.0000],
          [ 0.6702,  7.4463,  8.4368, 16.8152, 17.1239, 25.7304,  0.0000],
          [ 1.1914,  9.0101,  9.4586, 18.2457, 18.2465, 27.1025,  0.0000],
          [ 1.5415,  8.8788,  9.6645, 18.0975, 18.3750, 26.9972,  0.0000],
          [ 2.2197, 10.4203, 10.7086, 19.5163, 19.5055, 28.3640,  0.0000],
          [ 0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000,  0.0000]]]]),)
LPPool2d output grad:
(tensor([[[[10.7722, 31.1708, 52.9788],
          [13.4294, 34.2547, 56.1203],
          [16.2184, 37.3533, 59.2653]]]],
       grad_fn=<BackwardHookFunctionBackward>),)
1 Like
4

Thank you!

Now I know that it’s because I haven’t understood corresponding API enough.