CNN Filter pruning methods in PyTorch

#1

Hi, I wanna implement network pruning using PyTorch.
First, I use pruning algorithm to prune the model.
the result as follow:

a
a.png886×523 59.6 KB

then retrain the model.
but These cut weights are trained to be non-zero.
I wanna freeze only zero weights in entire network.
In this context, freeze means that freezed weights cannot be trained anymore.
The following method is not feasible
b
b.png1551×477 90.3 KB

Is there a better solution?

#2

It looks like you would like to set the requires_grad flag to False for a specific kernel of your weight matrix.
This is not possible as you can only set requires_grad for the whole matrix.
However, you could zero out the gradients for this kernel after backward and before the optimizer is called.

1 Like
#3

Hi ptrblck…can you please suggest which version of pytorch pruning will modify the architecture of the network,when filters are pruned?
I am currently using " nn.utils.prune" and is not removing the filters from original architecture.

#4

Unfortunately, I’m not familiar enough with the pruning utility, but @Michela might know. :slight_smile:

#5

thanks for the reply.
Hoping for a suggestion from Michela.