Differences between writing models with nn.sequential() VS creating a Class?

I am proficient enough to understand how to read Pytorch code and reimplement them to fit my own needs but being self-taught there are still a lot of things I do not understand. Truth to be told I didn’t do a lot of OOP at all before learning Pytorch, I mainly just made many functions and chain them together to make my network work. Since I started to look at other people’s code to learn Pytorch I have noticed that there are 2 major ways of coding your networks. One way is to stuff all the layers in nn.sequential() and just assign that to a model OR define a class then assign that to a model. My question is what is the major difference? I have tried both ways and IMO nn.sequential is easier, I have also seen nn.sequential defined within the model class as well.

7 Likes

You can use whatever fits your use case.
While some people like to use the nn.Sequential approach a lot, I usually just use it for small sub-modules and like to define my model in a functional way, i.e. derive from nn.Module and write the forward method myself.

This approach gives you more flexibility in my opinion, since you are able to easily create skip connections etc.
On the other hand if you just want to stack layers together, nn.Sequential is totally fine.

17 Likes

Is there a performance improvement when one used over another?

There shouldn’t be any differences regarding the performance, but let us know, if you’ve encountered something.

2 Likes

I have to create a model which has 3 parallel CNNs network. An image is fed into all the 3 networks and finally the outputs of the three networks are concatenated.
Can I model this if I define all the CNN networks in different classes?

nn.sequential can work for this if I define 3 different layers in the same class and concatenate them in forward method.
But I want to know if I can model such networks in three different classes and finally concatenate them when I train them? Will there be a problem in backpropogation?

Thanks. !

Yes, this should be possible.
You could create a “parent model” and pass or initialize the submodels in its __init__ method.
In the forward just call each submodel with the corresponding data and concatenate their outputs afterwards.
This won’t create any issues regarding backpropagation.

2 Likes

Thankyou. This really helped!

1 Like