Benefit and usage of torch.nn.utils.prune

What the benefit can we get from torch.nn.utils.prune now?
Will you plan to replace simple tensor with sparse tensor for speedup of inference and memory size of model?

1 Like

Join the questions. Not sure if it possible to speedup multiplication of sparse tensors for now. Global structured pruning could reduce multiplication count.
Who knows something about it, please check my related questions here Global structured pruning.

I agree with you, will wait answer from PyTorch Dev :slight_smile:

I also join the question. I’ve checked the forward pass time even with drastic pruning (=0.9) and I have noticed no speedup nor model size change. Is there a way to obtain any of this?

1 Like

Check my answer here.

1 Like

@ani0075 is right, you’d need to rely on a specific sparse matrix algebra library for speedups.
You can find my old answer to a similar question here: https://github.com/pytorch/tutorials/pull/605#issuecomment-585994076