I want to perform channel-wise pruning on pretrained ResNet-50 model, but I can’t figure out whether there is a method to preserve the unpruned channels’ weights?
Could you explain your task in more detail?
I assume that you have a convolutional network and for example the first convolutional layer receives an RGB image and outputs 512 channels. Do you want to remove some of these output channels?
Note that these channels are likely also used in the subsequent layers. Therefore the corresponding weights would also need to be removed (or fed with some fixed input).
What is the goal of the pruning? Perhaps I could offer you a different solution to achieve your goal.
If you want to preserve weights of a model, you can save the state dictionary.
You can then modify these weights, e.g., reduce the output dimension of a convolution layer. But these modified weights could not be loaded into the original model. A new model would have to be defined specifically for the pruned weights
However, though I did not try this myself, intuition tells me that performance of the “pruned” model would be potentially much worse than that of the original model (depending on how many modifications are made).
Try going through this tutorial. It might answer your questions.
Sorry but my pruning method isn’t pytorch api, is my defined method.