I am doing weight pruning, by sparsifying activations with mask – If the activations have lots of zeros, the weights can be safely zeroed out too.
And I need way to calculate weight parameters sparsity percentage.
Say I have y(x) = ReLU(Wx+b)
x is activations vector from previous layer, and have lots of zero values (due to mask)
From matmul Wx, how do I find elements (and its indices) in W that match with zero-valued activations in x,
So I can sparse W, and count zero values to calculate sparsity percentage.
Does pytorch have such function to do so?
Also how do I do the same for convolution layers (They have filters, filter size, stride, pad, etc)
Thank you