In the __init__() of a module class, I want to pack and register some conv layers like this:

self.outs = nn.ModuleList(
[nn.ModuleList([Conv(inp_dim + j * increase, oup_dim, 1, relu=False, bn=False) for j in range(5)]) for i in
range(nstack)])

then in the forward() method, I want to use the conv layers in the nested module list. preds_instack.append(self.outs[i][j](features_instack[j]))

Am I right? I’m debugging my module which claimed:

RuntimeError: Trying to backward through the graph a second time, but the buffers have already been freed. Specify retain_graph=True when calling backward the first time.

It looks like you are using all conv layers separately on some slices of your features. Is that correct?
Could you post a bit more code, since the current ModuleList seems to work using this small example:

outs = nn.ModuleList([nn.ModuleList([nn.Conv2d(3, 3, 3, 1, 1) for j in range(5)]) for i in range(3)])
x = torch.randn(5, 3, 24, 24)
outputs = []
for i in range(3):
for j in range(5):
outputs.append(outs[i][j](x[j:j+1]))
outputs = torch.stack(outputs)
outputs.mean().backward()
print(outs[0][0].weight.grad)