Use some of the input channel of the convolution kernel

I have a conv layer with input_channel=10 and output_channel=20. Usually I need take feature maps with shape [B,H,W,C=10] as input. But I want use different channel at different time, the number of channels will not be greater than 10. For example, I want take feature maps with shape [B,H,W,C=5] as input. That means, in theory, only half of the kernel will be used, which makes a speed up.
The easiest way is to pad the feature maps to full shape,but I don’t want do this. Because this means I can’t get acceleration.

You could slice the conv kernel and use the functional API via F.conv2d.