I’m using “ModuleList” to create multiple branches, but it turns out all branches output the same result. Mini repo. Thank you!
def _make_branches(self, *layers):
branches = []
for i in range(self.num_branches):
branch = nn.Sequential(*layers)
branches.append(branch)
branches = nn.ModuleList(branches)
return branches
The _make_branches method seems to take already initialized modules and creates nn.ModuleLists using them.
Since each of the passed layers will contain the same parameters, it’s expected that the output would be the same.
If you want to create branches with new layers, you would have to initialize new modules before adding them to the nn.ModuleLists.
@ptrblck thank you for the explanation.
I wonder Is it possible to initialize dynamic number of new modules?
If you would like to initialize a specific number of layers given a passed argument to the model, you could replace *layers with the initialization of new modules:
def _make_branches(self, num_layers):
branches = []
for i in range(num_layers):
branch = nn.Sequential(nn.Linear(1, 1)) # replace this with any new initialization
branches.append(branch)
branches = nn.ModuleList(branches)
return branches
Great advice. Thank you so much!