What the benefit can we get from torch.nn.utils.prune now? Will you plan to replace simple tensor with sparse tensor for speedup of inference and memory size of model?

Join the questions. Not sure if it possible to speedup multiplication of sparse tensors for now. Global structured pruning could reduce multiplication count. Who knows something about it, please check my related questions here Global structured pruning .

I agree with you, will wait answer from PyTorch Dev :slight_smile:

I also join the question. I’ve checked the forward pass time even with drastic pruning (=0.9) and I have noticed no speedup nor model size change. Is there a way to obtain any of this?

Check my answer here. [image] How to improve inference time of pruned model using torch.nn.utils.prune? As @ypsoh mentioned, the torch.sparse module is still in beta. So you won’t get any speed up for a model pruned in an unstructured way unless you make use of sparse mat…

@ani0075 is right, you’d need to rely on a specific sparse matrix algebra library for speedups. You can find my old answer to a similar question here: https://github.com/pytorch/tutorials/pull/605#issuecomment-585994076

Benefit and usage of torch.nn.utils.prune

ani0075 (Aniruddha Saha) July 9, 2020, 5:27pm 5

Check my answer here.