May I ask how to get parameters’ gradients after applying pruning masks?
pruning_method=prune.RandomUnstructured to ResNet50.
I got the following observations. It seems pruned
nn.Parameter are converted into plain
nn.Tensor, and I can no longer get their gradients.
* remain weight ratio = 10.01393285846195 % -> optimizer.step() # after loss.backward(), before gradient descent step (Pdb) model.conv1.weight.grad.shape # I intentionally avoid pruning conv1 torch.Size([64, 3, 3, 3]) (Pdb) model.layer1.conv1.weight.grad.shape # I prune all other weights main_train.py:1: UserWarning: The .grad attribute of a Tensor that is not a leaf Tensor is being accessed. Its .grad attribute won't be populated during autograd.backward(). If you indeed want the gradien t for a non-leaf Tensor, use .retain_grad() on the non-leaf Tensor. If you access the non-leaf Tensor by mistake, make sure you access the leaf Tensor instead. See github.com/pytorch/pytorch/pull/30531 fo r more informations. ''' *** AttributeError: 'NoneType' object has no attribute 'shape' (Pdb) model.layer1.conv1.weight.shape torch.Size([64, 64, 3, 3]) (Pdb) type(model.layer1.conv1.weight) <class 'torch.Tensor'> (Pdb) type(model.conv1.weight) <class 'torch.nn.parameter.Parameter'>