torch.nn.Module.children for duplicate children

1

Hello there,

I have a quick question regarding the use of the methods children, named_children, modules or named_modules from torch.nn.Module.

In the doc, a note says that “Duplicate modules are returned only once”. Is there a way to obtain the children with duplicates?

For example:

Let’s imagine that I have the following module:

Sequential(
  (0): Sequential(
    (0): Linear(in_features=2, out_features=256, bias=True)
    (1): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): LeakyReLU(negative_slope=0.01)
    (3): Linear(in_features=256, out_features=512, bias=True)
    (4): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (5): LeakyReLU(negative_slope=0.01)
    (6): Linear(in_features=512, out_features=784, bias=True)
  )
  (1): Sigmoid()
)

I’d like to iterate over (the difference being 5 LeakyReLU(negative_slope=0.01) that does not appear)

0 Sequential(
  (0): Linear(in_features=2, out_features=256, bias=True)
  (1): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): LeakyReLU(negative_slope=0.01)
  (3): Linear(in_features=256, out_features=512, bias=True)
  (4): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): LeakyReLU(negative_slope=0.01)
  (6): Linear(in_features=512, out_features=784, bias=True)
)
0 Linear(in_features=2, out_features=256, bias=True)
1 BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
2 LeakyReLU(negative_slope=0.01)
3 Linear(in_features=256, out_features=512, bias=True)
4 BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
5 LeakyReLU(negative_slope=0.01)
6 Linear(in_features=512, out_features=784, bias=True)
1 Sigmoid()

Instead of the no-duplicate version

0 Sequential(
  (0): Linear(in_features=2, out_features=256, bias=True)
  (1): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (2): LeakyReLU(negative_slope=0.01)
  (3): Linear(in_features=256, out_features=512, bias=True)
  (4): BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
  (5): LeakyReLU(negative_slope=0.01)
  (6): Linear(in_features=512, out_features=784, bias=True)
)
0 Linear(in_features=2, out_features=256, bias=True)
1 BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
2 LeakyReLU(negative_slope=0.01)
3 Linear(in_features=256, out_features=512, bias=True)
4 BatchNorm1d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
6 Linear(in_features=512, out_features=784, bias=True)
1 Sigmoid()

Thanks!

2

Answer:
It’s because I was instantiating the LeakyReLU only once!