I have used torch.nn.utils.prune to prune a pretrained model using below code.
for name, module in model.named_modules(): # prune 20% of connections in all 2D-conv layers if isinstance(module, torch.nn.Conv2d): prune.l1_unstructured(module, name='weight', amount=0.3) prune.remove(module, 'weight') # prune 40% of connections in all linear layers elif isinstance(module, torch.nn.Linear): prune.l1_unstructured(module, name='weight', amount=0.4) prune.remove(module, 'weight')
I have saved the pruned model. This pruned model’s size is halved and accuracy took a hit but did not improve its inference time. Please let me know how to improve the inference time of pruned model
Thanks in advance.