How does Pytorch sequence together modules

1

I am curious how Pytorch sequences together modules.

The application is, suppose I have a module structure A->B->C->D-E.

Then, I want to inject new modules A->B->G->H->C->D->E in the middle.

In general, I want to take any existing module and splice new modules into it.

I know I can replace modules by modifying the attribute. For example, resnet.fc = TransformerBlock would insert a transformer block for the fully connected layer of resnet.

So under the hood, does Pytorch store a sequential list of attributes to parse over, passing the output from one module as the input to the next?

2

It depends on the model implementation and while simple models can be created using an nn.Sequential container, the majority of the models use custom nn.Modules defining the forward pass in their forward method as seen here for e.g. ResNet models.