Build model from repeated template

1

Let’s say I have this Template class

class Template(nn.Module):
    def __init__(self):
        super().__init__()
    def forward(self, *input):
        pass

and I would like to build a model out of multiple instance of Template, given a specific number of layers.

class BuildModel(nn.Module):
    def __init__(self, number_of_layers):
        super().__init__()
        # self.layer_n = Template() for n in range(0, number_of_layers)
    def forward(self, *input):
        pass

I thought to implement this with a list of Template's instances, but soon realised this won’t go through the nn.Module's __setattr__. So, how to handle this situation of building a model out of multiple sub-modules given a layer count?

Idea

Hmm… Would setattr(self, 'layer_n', Template()) be a solution?

2

Hey @Atcold,

Have you looked at the torchvision models, I think the resnet model does what you want :

1 Like
3

So, if I got this right, are you suggesting to make a list of layers and expand it into a nn.Sequential()?
In my situation I need instantiate a different number of independent layer which I will later connect in a symmetrical fashion.
Would using setattr, which takes names as strings, do the job?

Let me jot down what I intend to do.
Say number_of_layers is 2.

b - a - A - B

If number_of_layers is 3, then

c - b - a - A - B - C

And so on. And each block is fed also with other inputs. So, it’s not a simple Sequential().

4

@Atcold you have reached the limit of my knowledge :slight_smile: can’t help you sorry

Just about the nn.Sequential point, I implemented a network with 3 branches in parallel, I am not using nn.Sequential, I just end up summing the variables of my branches, this is pure autograd. Hope it can help

1 Like
5

what about

def template(num_layers):
    if num_layers == 1:
        return nn.Sequential([a,A])
    else:
        return nn.Sequential([f(num_layers), template(num_layers-1), F(num_layers)])

If I recall correctly lists of modules/tensors within a module is tricky : https://pytorch.slack.com/archives/general/p1485447780001394

1 Like
6

OK, maybe I’m simply confused (I’ve edited my previous posts, fixing some names).
My BuildModel class would like to build a model based on a variable number of layers Template (which come in two types, represented before with lowercase and uppercase single letters).
The most straightforward solution would be creating two lists lower and UPPER, where I can reference the instance of my Template class. Doing so would create troubles with the current non-recursive implementation of nn.Module.__setattr__().

OK, I’m going to write them down in an extended form and create a BuildOneLayerModel() and BuildTwoLayerModel(). Perhaps I can later see a pattern and generalise to a BuildModel(number_of_layer) class.

7

There’s been a lot of threads like this in the forum. See e.g. this.

1 Like
8

Also, we’ll be merging an official solution into the core today or tomorrow.

9

Oh, that would be great. I would have been mucking around with string based attribute settings, otherwise … which should be just fine, no?

10

Yes, string attributes are fine. I’ve just merged nn.ModuleList and nn.ParameterList. There are no docs right now, I’ll add them tomorrow. You can give a list of modules to the constructor and later access them using the regular indexing syntax with integers. Same goes for ParameterList.

1 Like
11

Thank you, @apaszke. You’re great!