How to define the kernel size for conv2d

1

if I wanna do simple convolution i can use conv2d.
I want to simply compute the edge of my input (the input has the size of BxCxDxD), and my edge kernel has the size of 3x3, how should i expand my edge kernel so when i do conv2d(Input, Kernel) the output has the same size of BxCxDxD

e.g.


import torch
import torch.nn.functional as F
K =  torch.Tensor([[1 ,0, -1],[2, 0 ,-2], [1, 0 ,-1]])
Input = torch.rand(4,3,10,10) 


output = F.conv2d(Input,K,padding=(1, 1))
output.size() # expected to be 4x3x10x10
2

Say your input is ( Batch, input_channel, 64, 64)
Conv2D( input_channel, output_channel, 3, 1, 1) give you back the same size as the image.

output would be:

(Batch, output_channel, 64, 64)

Setting k_size, stride, padding, to 4, 2, 1 instead of 3, 1, 1 will give you ( Batch output_channel, 32, 32 ) instead.

3

In order to get the same output shape BxCxDxD you have to do two things:

to keep DxD: use padding, in your case padding=(1,1). This adds a “border” of zeros around your input, so the center of a 3x3 kernel can visit every point on your DxD grid (looking at your CxDxD stack “top-down” so to say).

to keep the number of channels C the same: use C kernels, the number of channels of the output of conv2d will always be the number of kernels used.

adding such a layer to a model in pytorch would look like this:
self.conv = nn.Conv2d(in_channels=C, out_channels=C, kernel_size=(3, 3), padding=(1, 1))

4

@inkplay @ptab

Thanks for your answer, but that is not how i want to use the conv2d.
I want to use conv2d like this:

import torch
import torch.nn.functional as F
K =  torch.Tensor([[1 ,0, -1],[2, 0 ,-2], [1, 0 ,-1]])
Input = torch.rand(4,3,10,10) 


output = F.conv2d(Input,K,padding=(1, 1))
output.size() # expected to be 4x3x10x10
5

I think you are confusing your self, kernels in conv2d are already randomly defined for you.
You can check this by doing this:

X = nn.Conv2D( 1, 1, 3, 1, 1) #  ( input_c, output_c, k_size, stride, padding ), k_size can be (3,3) or 3 
X.weight # a single 3 x 3 kernel, if you want to output more kernels you can try and change "output_c" to see what happens  

Parameter containing:
tensor([[[[-0.2303, -0.0186, -0.2070],
          [ 0.3190, -0.2940, -0.1227],
          [ 0.1014,  0.0417, -0.3254]]]])

I am guessing you want to use predefined weights as values for the kernel in conv2D correct?
If so you can do the following.

X = torch.Tensor([[1 ,0, -1],[2, 0 ,-2], [1, 0 ,-1]])
X = torch.nn.Parameter( X ) # calling this turns tensor into "weight" parameter
print( "X's size", X.size() )
print("X's tensor values as weight: ", X)
Z = nn.Conv2d( 1, 1, 3, 1, 1) # creates a test convolution 
print("Z's pre-initalized weight from Conv2D: ", Z.weight, "its size is: ", Z.weight.size())
Z.weight = X # simply copying over the new weight 
print("Z's new weight ", Z.weight)

X’s size torch.Size([3, 3])
X’s tensor values as weight: Parameter containing:

tensor([
[ 1., 0., -1.],
[ 2., 0., -2.],
[ 1., 0., -1.]])

Z’s pre-initalized weight from Conv2D: Parameter containing:

tensor([[[
[-0.1911, 0.0187, 0.0254],
[-0.0727, -0.1672, 0.0862],
[ 0.2888, 0.2479, 0.3203]]]])

Its size is: torch.Size([1, 1, 3, 3])

Z’s new weight Parameter containing:

tensor([
[ 1., 0., -1.],
[ 2., 0., -2.],
[ 1., 0., -1.]])

Edited Finally if you want to make sure the output size is correct the above 2 post has the answers. You can also google the output convolution formula and the documentations for Conv2D also has it. Good Luck!

1 Like
6

Hi,

I want to apply Conv2d to [64, 49,49,512] to reduce it to [64, 25,25,512].
I need to consider kernel_size=23, right?

7

I think your dimensions might be wrong.
nn.Conv2d expects inputs of [batch_size, channels, height, width].
Based on your example, it seems you are using 512 channels, while the spatial size is 49x49.
If that’s the case, a kernel_size of 25 with stride=1 and no padding might work:

conv = nn.Conv2d(512, 512, 25)
output = conv(torch.randn(1, 512, 49, 49))
print(output.size())
> torch.Size([1, 512, 25, 25])
1 Like
8

Hi @ptrblck
Thanks. Actually the feature map is in size [64, 512, 49, 49]. And by applying those values it doesn’t work/

conv= nn.Conv2d(512,512,kernel_size=25)
SO_fmaps_extend = conv(fmaps)

*** RuntimeError: Input type (CUDAFloatTensor) and weight type (CPUFloatTensor) should be the same

9

Since fmaps is stored in your GPU, you should also push the conv layer to your device using conv.to('cuda').