Could you give me advice how to apply Sparse Semi-Structured on Conv2d layers if possible, and if yes, provide an example, or should i still use Nvidia’s framework for this?
Currently, we only have support for accelerated sparse matmul, and not sparse convolutions. It would be possible if you can rewrite your convolutions into a matmul (neural network - 2-D convolution as a matrix-matrix multiplication - Stack Overflow), but this may materialize a much larger tensor and thus won’t lead to further speedups.
ASP only simulates sparsity, it should be possible to simulate sparsity with the WeightNormPruner, but this will not show any speedup unfortunately
Does PyTorch have a plan to officially incorporate or implement ASP (Automatic Structured Sparsity) for Conv2D?
I believe it could be as “simple” as creating a WeightNormSparsifier for Conv2D filters specifically.
Having no speedup in PyTorch is fine. We could export the structured sparsity enabled model from PyTorch and use other acceleration software for inference.
Thanks.
Sparse semi-structured Conv2D tensors sound intriguing. Could you provide more details or examples of the use case you’re considering? This would help in understanding the requirements better.
You could check out the NVIDIA ASP package. It’s almost no different from the weight tensor sparsification that PyTorch already has for MatMul. In Conv2D, we do 2:4 sparsification on the input channel.
Sorry for the late reply, got lost in my inbox. Currently we have no plans to add support for something similar to ASP in pytorch core. If this is something that you’re interested in adding though, please file an issue here: ao/torchao at main · pytorch/ao · GitHub