How to apply sobel filter to the output of different convolution layers

1

Hello,
I am working to family with Pytorch.
I created a 3D network to classify image. The input shape of image I used is (1, 1, 48, 48, 48) and the output shape is torch.Size([1, 256, 3, 3, 3]).

Now I want to continue use Sobel filter for edge detection. I used this following code:

inputs = torch.randn(1, 1, 48, 48, 48)
x = net.forward(inputs)
print(x.shape) // [1, 256, 3, 3, 3]
sobel = [[1, 2, 1], [0, 0, 0], [-1, -2, -1]]
depth=x.size()[1]
channels=x.size()[2]
sobel_kernel = torch.FloatTensor(sobel).expand(depth,channels,3,3).unsqueeze(0)
print(sobel_kernel.shape) //torch.Size([1, 256, 3, 3, 3])
malignacy = F.conv3d(x, sobel_kernel, stride=1, padding=1)
print(malignacy.shape) //torch.Size([1, 1, 3, 3, 3])

The output is torch.Size([1, 1, 3, 3, 3]), but I want to keep the number of image’s depth is 256 not 1.
How can I solve it?

Thanks in advance!

2

You could use a grouped convolution to and expand the kernel to the necessary shape:

x = torch.randn([1, 256, 3, 3, 3])
sobel = [[1, 2, 1], [0, 0, 0], [-1, -2, -1]]
depth = x.size()[1]
channels = x.size()[2]
sobel_kernel = torch.tensor(sobel, dtype=torch.float32).unsqueeze(0).expand(depth, 1, channels, 3, 3)
print(sobel_kernel.shape)
> torch.Size([256, 1, 3, 3, 3])
malignacy = F.conv3d(x, sobel_kernel, stride=1, padding=1, groups=x.size(1))
print(malignacy.shape)
> torch.Size([1, 256, 3, 3, 3])
3 Likes
3

@ptrblck This is what I need, thanks you so much

1 Like
4

Hi @ptrblck,
Sorry to bother you again.
Now, I have a different kernel size with [3,3,3,3]. How can I resize it for convolutional layer.

x = torch.randn([1, 256, 3, 3, 3])
kernel_x = np.array([[[1, 2, 1], [0, 0, 0], [-1, -2, -1]],
                             [[2, 4, 2], [0, 0, 0], [-2, -4, -2]],
                             [[1, 2, 1], [0, 0, 0], [-1, -2, -1]]])
kernel_y = kernel_x.transpose((1, 0, 2))
kernel_z = self.kernel_x.transpose((0, 2, 1))
sobel_kernel = torch.from_numpy(np.array([self.kernel_x, self.kernel_y, self.kernel_z]).reshape((3, 1, 3, 3, 3))).float()
print('sobel_kernel', sobel_kernel.size())
> ('sobel_kernel', (3, 1, 3, 3, 3))
malignacy = F.conv3d(x, sobel_kernel, stride=1, padding=1, groups=x.size(1))

> RuntimeError: Given groups=256, expected weight to be at least 256 at dimension 0, but got weight of size 3 1 3 3 3 instead

Thank you very much.

5

Hi ptrblck,
Does this sobel = [[1, 2, 1], [0, 0, 0], [-1, -2, -1]] filter work when our data is normalized between 0 to 1, or do we have to normalize this sobel filter also?

6

I think it depend on your use case and what you would like to do with the output.
If you want to detect the edges only, the scaling would not be necessary. However, if you want to correctly estimate the derivative you might want to scale the filter e.g. by dividing by 8.