Calculation detail in nn.Conv2d

Hello, I just can’t figure out the way nn.Conv2d calculate the output . The result calculated from torch is not the same as some machine learning course had taught.

For example, likes the code below:

>> m = torch.nn.Conv2d(1, 1, 3, padding=0)
>> m(input)
tensor([[[[ 0.5142,  0.3803,  0.2687],
          [-0.4321,  1.1637,  1.0675],
          [ 0.1742,  0.0869, -0.4451]]]], grad_fn=<ThnnConv2DBackward>)
>> input
tensor([[[[ 0.7504,  0.1157,  1.4940, -0.2619, -0.4732],
          [ 0.1497,  0.0805,  2.0829, -0.0925, -1.3367],
          [ 1.7471,  0.5205, -0.8532, -0.7358, -1.3931],
          [ 0.1159, -0.2376,  1.2683, -0.0959, -1.3171],
          [-0.1620, -1.8539,  0.0893, -0.0568, -0.0758]]]])
>> m.weight
Parameter containing:
tensor([[[[ 0.2405,  0.3018,  0.0011],
          [-0.1691, -0.0701, -0.0334],
          [-0.0429,  0.2668, -0.2152]]]], requires_grad=True)

for the left top element 0.5142, it’s not the output equals to

>> import numpy as np
>> w = np.array([[0.2405,  0.3018,  0.0011], [-0.1691, -0.0701, -0.0334], [-0.0429,  0.2668, -0.2152]])
# top-left 3x3 matrix of 5x5
>> x = np.array([[ 0.7504,  0.1157,  1.4940], [ 0.1497,  0.0805,  2.0829], [1.7471,  0.5205, -0.8532]])
>> print(np.sum(w*x))
#  0.364034 != 0.5142
0.36403412999999996

My Question here is: Why Could the output not equal to 0.5142?

Further more, when i add paramter padding into nn.Conv2d, The outcome seems obscure to me as below, thanks a lot for explain that to me.Thank you!

>> input
tensor([[[[ 0.7504,  0.1157,  1.4940, -0.2619, -0.4732],
          [ 0.1497,  0.0805,  2.0829, -0.0925, -1.3367],
          [ 1.7471,  0.5205, -0.8532, -0.7358, -1.3931],
          [ 0.1159, -0.2376,  1.2683, -0.0959, -1.3171],
          [-0.1620, -1.8539,  0.0893, -0.0568, -0.0758]]]])
# set padding from 0 to 1 equals to (1, 1)
>> m1 = torch.nn.Conv2d(1, 1, 1, padding=1)
>> m1(input)
tensor([[[[0.9862, 0.9862, 0.9862, 0.9862, 0.9862, 0.9862, 0.9862],
          [0.9862, 1.0771, 1.0002, 1.1672, 0.9544, 0.9288, 0.9862],
          [0.9862, 1.0043, 0.9959, 1.2385, 0.9749, 0.8242, 0.9862],
          [0.9862, 1.1978, 1.0492, 0.8828, 0.8970, 0.8174, 0.9862],
          [0.9862, 1.0002, 0.9574, 1.1398, 0.9745, 0.8266, 0.9862],
          [0.9862, 0.9665, 0.7615, 0.9970, 0.9793, 0.9770, 0.9862],
          [0.9862, 0.9862, 0.9862, 0.9862, 0.9862, 0.9862, 0.9862]]]],
       grad_fn=<ThnnConv2DBackward>)

The confused point is that how 0.9862 be calculated? And what is the default padding strategy in nn.Conv2d?

Thank you for reading and answer!

If you state down the documentation hard enough, you find that

  1. Conv2d has a bias parameter by default that your numpy calculation does not consider (see the formula in the doc),
  2. padding documented to be is (implicit) zero padding. This matches my experience.

Best regards

Thomas

1 Like

Thank you very much! i am sorry for my careless.