So I have a list of layers and I apply a mask to some of the weights in the layers, by doing the following:
def forward(self, input, mask=None): if mask is not None: self.weight.data.t().mul_(mask) f = F.linear(input, self.weight, self.bias) return f
This works when I store the list of layers like
self.hidden_layers = , but when I change this to
self.hidden_layers = nn.ModuleList() the mask code above doesn’t work.
This is how I append to the list of layers:
for i in np.arange(self.num_hidden_layers - 1): self.hidden_layers.append(LinearAutoMl(hidden_size, hidden_size).to(device))
Any reason why using ModuleList would do this?