Creating a parameter group for all linear layers

Hi all,

I want to group all linear layers, but not sure what is the best way to do it.

you can use modules http://pytorch.org/docs/master/nn.html?highlight=named_parameters#torch.nn.Module.modules

param_group = []

for m in model.modules():
    if isinstance(m, nn.Linear):
        param_group.append(m.weight)
        if m.bias is not None:
            param_group.append(m.bias)
4 Likes