I am pruning resnet110, but I could not do any weight pruning. I am not looking to set the value of the weights to zero since it only affects the sparsity of the model, but I want to delete/remove the weight. I used torch_pruning but it only allows to prune out channel.
I’m unsure what your use case is, since deleting a weight would cause failures unless you also manipulate the actual forward pass and exclude the deleted weight from it.
Here is a small example:
model = models.resnet18()
# delete random weight
model.fc.weight = None
x = torch.randn(1, 3, 224, 224)
out = model(x)
# TypeError: linear(): argument 'weight' (position 2) must be Tensor, not NoneType
Deleting specific values inside a tensor is also not possible unless you remove an entire slice from a dimension:
print(model.conv1.weight.shape)
# torch.Size([64, 3, 7, 7])
# you cannot delete one value only
model.conv1.weight[3, 2, 1, 4] = None
# TypeError: can't assign a NoneType to a torch.FloatTensor
# but e.g. a complete channel
model.conv1.weight = nn.Parameter(model.conv1.weight[:63])
print(model.conv1.weight.shape)
# torch.Size([63, 3, 7, 7])