Append() for nn.Sequential or directly converting nn.ModuleList to nn.Sequential

jingweiz · September 6, 2017, 9:16am

Hi,
maybe I’m missing sth obvious but there does not seem to be an “append()” method for nn.Sequential, cos it would be handy when the layers of the sequential could not be added at once.
Or it would be equivalent if I first added all the layer I need into a ModuleList then there’s a method for directly converting all the modules in a ModuleList to a Sequential.
Thanks in advance!

allenye0119 · September 6, 2017, 9:35am

You can first construct a python list of nn.Modules and unpack it into a nn.Sequential

import torch.nn as nn

modules = []
modules.append(nn.Linear(10, 10))
modules.append(nn.Linear(10, 10))

sequential = nn.Sequential(*modules)

jingweiz · September 6, 2017, 11:40am

Ah that’s neat, thanks

QuantScientist · September 6, 2017, 1:24pm

Great example here:

github.com

vinhkhuc/PyTorch-Mini-Tutorials/blob/master/5_convolutional_net.py

import numpy as np

import torch
from torch.autograd import Variable
from torch import optim

from data_util import load_mnist


# We need to create two sequential models here since PyTorch doesn't have nn.View()
class ConvNet(torch.nn.Module):
    def __init__(self, output_dim):
        super(ConvNet, self).__init__()

        self.conv = torch.nn.Sequential()
        self.conv.add_module("conv_1", torch.nn.Conv2d(1, 10, kernel_size=5))
        self.conv.add_module("maxpool_1", torch.nn.MaxPool2d(kernel_size=2))
        self.conv.add_module("relu_1", torch.nn.ReLU())
        self.conv.add_module("conv_2", torch.nn.Conv2d(10, 20, kernel_size=5))
        self.conv.add_module("dropout_2", torch.nn.Dropout())

This file has been truncated. show original

jingweiz · September 13, 2017, 7:18pm

Thanks a lot! That’s really helpful!

Ilassa_Savadogo · June 1, 2019, 3:39pm

Thanks, worked like a charm.

nogaz · September 13, 2022, 2:30pm

Hi,
I’ve had a similar issue and this discussion was very helpful.
I do have another question - I would like to:

extract a copy of the current state of the tensor to a “skip connection” list in a U-shaped cnn.
adapt the tensor’s shape (that is the input) as I go (can not determine this ahead because I can’t know the number of convolutions ahead)

how can I do this while I’m adding modules to the list? I would like to make the list as modular as I can and can not skip the “append” stage in a for loop.

Thank you!

Snowball1 · March 10, 2025, 3:12am

For people who dont want to update their CUDA or Pytorch to the version(e.g. v1.13.0) that Sequential have append() method, you can just commend you local Sequential code, and copy the “pytorch/torch/nn/modules/container.py at v1.13.0 · pytorch/pytorch · GitHub” code to your local Sequential code position, and then it can use append() method.

But remember, do not copy Sequential code from the latest pytorch version,because it need python v11 to support some new feature.