Hello all ,i’m new to PyTorch can anyone please how do we choose the padding is there any formula or like what number should i decide for padding while making the architecture of CNN.
Below are the two examples of padding , in which 2 numbers have been taken into account but i don’t know how.
Thank u in advance.
self.conv1 = nn.Conv2d(in_channels, out_channels, kernel_size, stride=1, padding=0)
self.conv1 = nn.Conv2d(1, 16, 2, stride=2)
Hi,
the value for padding is up to you. If you use a normal convolution without padding and stride=1, the output will be of the size output = (input - kernel) + 1 and your output gets smaller after the convolution (except kernel = 1). This is exactly what would happen in your first example:
To counteract this, padding can be applied. This way you add a border of some values (e.g. 0) around your input and increases its size by two times the padding value ( input = input + 2*padding).
If you want the output to be the same shape as the input, the formula would be
padding = (kernel - 1) // 2.
Maybe this animation will help you understand it better: GitHub - vdumoulin/conv_arithmetic: A technical report on convolution arithmetic in the context of deep learning
Or this pdf containing some visualizations and formulas : https://arxiv.org/pdf/1603.07285.pdf
Your second example( self.conv1 = nn.Conv2d(1, 16, 2, stride=2)), has a kernel_size = 2,
stride = 2 and padding = 0. In this case your kernel ‘moves’ 2 positions per convolution, which is exactly the kernel_size, resulting in halving the original input size, so that your output = input / 2
Thanks Caruso that clarifies a lot, you are the best.