Check type of Modules in nn.ModuleList

I have a ModuleList, with Linear and Batchnorm layers. I wanted to call named named_parameters() and only get the grads of the linear layers as follows:

for name, param in model.named_parameters():
            if(isinstance(param, nn.Linear)):            
                  grads[name] = param.grad

This doesn’t work since type(param) always returns torch.nn.parameter.Parameter. Also if "weight" in name: doesn’t work since batchnorm layers also have “weight” in their name.

Are there any workarounds to this? I also looked at giving custom names to modules in the ModuleList, but I couldn’t find how.

Hi, ModuleList does not come with any learnable parameters. Items inside the list are just like name of layers.
You can replace nn.ModuleList() with nn.Sequential() instead.

I have a dynamic number of hidden layers and so I don’t think nn.Sequential will work.

You could iterate model.named_modules() and check the returned module to be an nn.Linear instance:

for name, module in model.named_modules():
    if isinstance(module, nn.Linear):
        print(name, module.weight.grad)
1 Like