Use linear.weight or linear.weight.data

autograd
1

Here is my problem, I have a linear layer.

  1. Everytime I forward, I want to make the linear weight multiply a mask.
linear = nn.Linear(10, 10) ## for example

Now during forward

   def forward(inputs):
       ## which way below is the correct way to use? mask is with all 0 and 1 values/
        self.linear.weight = self.linear.weight * torch.from_numpy(self.mask)
        self.linear.weight.data= self.linear.weight.data * torch.from_numpy(self.mask)

     ### In the following code, I will use the updated linear layer to do something and 
     ### want the model to check gradient
2

You want to do:

with torch.no_grad():
   self.linear.weight.mul_(torch.from_numpy(self.mask))
1 Like
3

actually you want

weight = self.linear.weight * torch.from_numpy(self.mask)
output = F.linear(input, weight, ...)