How to impelement a customized convolutional layer (conv2d)

1

Hi,
I want to implement a customized Conv2d in which some multiplications during the convolution operation are dropped by some probability. This would happen during testing, and preferably different multiplications would drop for different kernels in a layer. Is there an easy way to do this? Do I need to implement the Conv2d function using pytorch functions from scratch?
Thanks!

2

Have a look at https://github.com/szagoruyko/diracnets/blob/master/diracconv.py , it might be a way of implementing what you want

1 Like
3

Is that possible to implement the conv2d layer with the unfold function? I think it should be, but not sure about it. Thanks in advance for any hint or suggestion.

4

I guess this should work:

x = torch.rand(4, 3, 6, 6)
k = torch.rand(3, 10, 2, 2)
(x.unfold(2, 2, 1).unfold(3, 2, 1).unsqueeze(2) * k.unsqueeze(0).unsqueeze(3).unsqueeze(4)).sum(-1).sum(-1).sum(1)

where the batch size is 4, number of input channel is 3, number of output channel is 10, and kernel size is 2, stride is 1, in this example.

1 Like