Custom 1D kernel: Set Columns as zeros

1

Hi,
Given below is my Conv1d code:
nn.Conv1d(2, 4, kernel_size = 6, stride = 1, padding = 1)
In these 4 kernels, I want them to have different columns to be zeros. For example, let’s say first kernel has second column as all zeros. So, it would be something like this:
[w11, 0, w13, w14, w15, w16]
[w21, 0, w23, w24, w25, w26]
How Can I do this?

2

You can manipulate the parameters in a torch.no_grad() block:

conv = nn.Conv1d(2, 4, kernel_size = 6, stride = 1, padding = 1)
with torch.no_grad():
    conv.weight[0, :, 1] = 0.
2 Likes
3

Actually, my code looks like this:
self.convnet1 = nn.Sequential(
nn.Conv1d(2, 4, kernel_size=6, stride=1, padding=1),
nn.ReLU(inplace=True),)
Can you tell me how to implement it here?

4

Also, I want to learn the non-zero weights.

5

I implemented something like this:
self.weight = torch.randn((2, 4, 6), requires_grad = True)

self.convnet1 = nn.Sequential(
nn.Conv1d(2, 4, kernel_size=6, stride=1, padding=1),
nn.ReLU(inplace=True),)

self.convnet1.weight = nn.Parameter(self.weight)

But the weights are not updating…

6

I tried to implement like this:
self.weight = torch.randn((2, 4, 6), requires_grad = True)
with torch.no_grad():
self.weight[:,0,0] = 0
self.weight[:,1,1] = 0
self.weight[:,2,0] = 0
self.weight[:,3,1] = 0
self.convnet1 = nn.Sequential(
nn.Conv1d(2, 4, kernel_size=6, stride=1, padding=1),
nn.ReLU(inplace=True),)
self.convnet1.weight = nn.Parameter(self.weight)

But I get error in the following line:
for p in model.parameters():
p.grad.data.div_(batch_size) # divide by the actual batch size
total_norm += p.grad.data.norm() ** 2

Error: p.grad has no variable named data.

7

Since you want to update it for conv layer alone. The code should not be self.convnet1.weight, it should be self.convnet1[0].weight. Make this change and check.
self.weight = torch.randn((2, 4, 6), requires_grad = True)
self.convnet1[0].weight = nn.Parameter(self.weight)

1 Like