I encountered a strange phenomenon: after pruning an empty filter in a conv layer, the output features of the conv is inconsistent with the previous one, (The shape change is as expected, what I said is the value inconsistency)
c1=torch.nn.Conv2d(3,10,7,bias=False) c2=torch.nn.Conv2d(3,9,7,bias=False) for i in range(9): c1.weight.data[i] = c2.weight.data[i] c1.weight.data=torch.zeros(3, 7, 7) data = torch.rand(64,3, 224,224) out1=c1(data) out2=c2(data) print(torch.sum(out1)) print(torch.sum(out2))
tensor(2090009.8750, grad_fn=<SumBackward0>) tensor(2090010.1250, grad_fn=<SumBackward0>)
The sum of the output features of c1 and c2 are different. c1 only have one more empty filter than c2, and they convolve the same input data, so, the sum of their output features should be the same theoretically. Why the sum of these two output features are different? Is this a bug? or pytorch has some optimization of the convolution operation that results in this inconsistency?