How to extract learned weights by channel

[James_Lee]

Follow this issue,

How to extract learned weights correctly

I can’t repeat your error, my idea is that your problem comes from your data set: model = Net() optimizer = optim.Adam(model.parameters(), 0.01) criterion = nn.MSELoss() def train(epochs): model.train() for epoch in range(1, epochs+1): # I dont have your data set: data = Variable(torch.rand(1,100)) target = Variable(torch.ones(1,100)) optimizer.zero_grad() output = model(data) loss = criterion(output, target) loss.backward() …

Is there a way to get the sum of weights.data by channel dim? Thanks!

[ptrblck]

I assume you mean the in_channel dimension of a conv layer by “channel dim”?
If so, this would work:

conv = nn.Conv2d(3, 6, 3, 1, 1)
conv.weight.sum([0, 2, 3])
> tensor([-0.2552,  0.2478,  0.3706], grad_fn=<SumBackward1>)

[James_Lee]

Thanks for your reply. Actually, I would like to explore the importance of the channel-wise features by summing up the channel-wise weights for a group=3 Conv.

I tried to calculate a group=3 3x3 conv’s sum of weights by this,

            mid1_1 = net.conv1[-3].weight.data.shape[1]//3
            mid1_2 = net.conv1[-3].weight.data.shape[1]//3*2
            conv1_g1_ws = torch.sum(net.conv1[-3].weight.data[:mid1_1, :mid1_1, :, :])
            conv1_g2_ws = torch.sum(net.conv1[-3].weight.data[mid1_1:mid1_2, mid1_1:mid1_2, :, :])
            conv1_g3_ws = torch.sum(net.conv1[-3].weight.data[mid1_2:, mid1_2:, :, :])

Am I right? Many thanks for your advice.