Is this the correct way to specify custom manipulation of the weights of a convolution layer?
def __init__(self, channels, filter_mask):
self.kernel_size = tuple(filter_mask.shape)
self.filter_mask = nn.Parameter(filter_mask) # tensor
self.conv = nn.Conv3d(
def forward(self, x):
self.conv.weight.data = self.conv.weight.data * self.filter_mask
Specifically, in the last line of code I’m using
.data attribute and not the tensors themselves since otherwise I’m getting the following error:
TypeError: cannot assign 'torch.FloatTensor' as parameter 'weight' (torch.nn.Parameter or None expected)
self.conv.weight = self.conv.weight * self.filter_mask
No, you shouldn’t use the
.data attribute, as it might yield silent errors and could break your code in various ways.
The error message points to a mismatch between a tensor and the expected
Try to wrap the new
weight into a parameter via:
self.conv.weight = nn.Parameter(self.conv.weight * self.filter_mask)
self.filter_mask is used in a
no_grad() block only, I assume it won’t be trained and can thus be registered as a buffer via:
Thanks @ptrblck for your prompt answer.
This actually broke the backward pass for me, seems like the weight matrix is kept the same after backward is called, maybe something changed between pytorch versions?
The gradient should not be affected by the masking since it was applied inside the stop_grad context manager. @BBLN
I meant the gradient for the weights themselves stopped working at all, maybe because the parameter object being changed in a call to the module forward pass.