Deactivate a filter in conv layer

vision
1

I want to deactivate one filter of a conv layer just before inference.

Here’s a partial Resnet model:

(layer4): Sequential(
(0): BasicBlock(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=1)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=1)
(downsample): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=1)
)
)
(1): BasicBlock(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=1)
(relu): ReLU(inplace=True)
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=1)
)
)

How do I deactivate a particular filter (ith among 512 after convolution) in layer4, (1) basicblock, conv2 ?

2

You could set its weight (and bias if needed) at the corresponding slice to zeros.
E.g. something like this should work:

with torch.no_grad():
    model.layer4[1].conv2.weight[idx].zero_()
3

Thanks for the response! I used the following code:

model.layer4[1].conv2.weight[idx].zero_()
print(model.layer4[1].conv2.weight[idx])

tensor([[[[-0.0002, 0.0002, -0.0038],
[ 0.0055, 0.0055, 0.0024],
[ 0.0059, 0.0046, 0.0024]],
[[-0.0066, -0.0105, -0.0040],
[-0.0052, -0.0091, -0.0021],
[ 0.0044, 0.0012, 0.0072]],
[[ 0.0007, 0.0044, 0.0023],
[-0.0045, -0.0009, -0.0018],
[-0.0079, -0.0044, -0.0058]],

So, it seems like it’s not working.

4

It works for me:

model = torchvision.models.resnet18()
idx = 1
with torch.no_grad():
    model.layer4[1].conv2.weight[idx].zero_()

print(model.layer4[1].conv2.weight[idx])
# tensor([[[0., 0., 0.],
#          [0., 0., 0.],
#          [0., 0., 0.]],

#         [[0., 0., 0.],
#          [0., 0., 0.],
#          [0., 0., 0.]],
# ...
5

Thanks!!! It worked. My idx was an ndarray. I had to get the item().