How to make some slight changes to the convolutional layer in neural network?

vision
1

I want to make some small changes in the convolutional layer of my neural network model, that is, to replace the input pixels at the center of the calculation and then perform convolution. At the same time, I also want to be able to continue to use the GPU for training. How should I do it? (sorry for my poor English)

2

I’m unsure if you want to manipulate the input activation or the actual conv filter.
These approaches should work and you can pick which fits your use case:

conv = nn.Conv2d(1, 16, 3, 1, 1).cuda()
x = torch.randn(1, 1, 24, 24, requires_grad=True, device="cuda")

# calculate fake activation output 
out = x * 2
# manipulate any pixels
out[:, :, 12, 12] = 1.0

out = conv(out)


x = torch.randn(1, 1, 24, 24, requires_grad=True, device="cuda")
# manipulate first filter
with torch.no_grad():
    conv.weight[0, 0, 1, 1] = 1.
out = conv(x)
3

What I mean is to replace the central input pixel within the range of the convolution window, not the center of the entire input, such as replacing the average value of the surrounding pixels in the convolution window, because I want to do this for every pixel of the input , I don’t want to use the center input pixel within the convolution window range when convolving, but use the average value of the neighbors instead, it may not be clear, sorry

4

If you want to implement a mean filter, you could directly set the filter weights via:

conv = nn.Conv2d(1, 1, 3, bias=False)
with torch.no_grad():
    conv.weight.fill_(1./9)
    
x = torch.arange(7*7).view(1, 1, 7, 7).float()
print(x)
# tensor([[[[ 0.,  1.,  2.,  3.,  4.,  5.,  6.],
#           [ 7.,  8.,  9., 10., 11., 12., 13.],
#           [14., 15., 16., 17., 18., 19., 20.],
#           [21., 22., 23., 24., 25., 26., 27.],
#           [28., 29., 30., 31., 32., 33., 34.],
#           [35., 36., 37., 38., 39., 40., 41.],
#           [42., 43., 44., 45., 46., 47., 48.]]]])

out = conv(x)
print(out)
# tensor([[[[ 8.0000,  9.0000, 10.0000, 11.0000, 12.0000],
#           [15.0000, 16.0000, 17.0000, 18.0000, 19.0000],
#           [22.0000, 23.0000, 24.0000, 25.0000, 26.0000],
#           [29.0000, 30.0000, 31.0000, 32.0000, 33.0000],
#           [36.0000, 37.0000, 38.0000, 39.0000, 40.0000]]]],
#        grad_fn=<ConvolutionBackward0>)