# The meaning of 【return nn.Sequential (* layers) 】

I am a beginner. I have a simple question about Resnet-18.(but I really don’t understand) The following is the network structure code of Resnet-18.
I can’t understand the code,and I didn’t find a specific explanation in the reference books :
`**return nn.Sequential (* layers)** `
Does it mean that the last returned value is (layers),For example, if layers = [10,20], by executing this line of code, the final return value is still [10,20]. If this is the case, why not use `return layers` (I don’t know whether this is feasible)directly.

``````定义resnet18
'''
def conv3x3(in_planes, out_planes, stride=1):
return nn.Conv2d(in_planes, out_planes, kernel_size=3, stride=stride,

class BasicBlock(nn.Module):
expansion = 1
#inplanes其实就是channel,叫法不同
def __init__(self, inplanes, planes, stride=1, downsample=None):
super(BasicBlock, self).__init__()
self.conv1 = conv3x3(inplanes, planes, stride)
self.bn1 = nn.BatchNorm2d(planes)
self.relu = nn.ReLU(inplace=True)
self.conv2 = conv3x3(planes, planes)
self.bn2 = nn.BatchNorm2d(planes)
self.downsample = downsample
self.stride = stride

def forward(self, x):
residual = x

out = self.conv1(x)
out = self.bn1(out)
out = self.relu(out)

out = self.conv2(out)
out = self.bn2(out)
#把shortcut那的channel的维度统一
if self.downsample is not None:
residual = self.downsample(x)

out += residual
out = self.relu(out)

return out

class ResNet(nn.Module):
def __init__(self, block, layers, num_classes=10):
self.inplanes = 64
super(ResNet, self).__init__()
self.conv1 = nn.Conv2d(1, 64, kernel_size=7, stride=2, padding=3,   #因为mnist为（1，28，28）灰度图，因此输入通道数为1
bias=False)
self.bn1 = nn.BatchNorm2d(64)
self.relu = nn.ReLU(inplace=True)
self.layer1 = self._make_layer(block, 64, layers[0])
self.layer2 = self._make_layer(block, 128, layers[1], stride=2)
self.layer3 = self._make_layer(block, 256, layers[2], stride=2)
self.layer4 = self._make_layer(block, 512, layers[3], stride=2)
self.avgpool = nn.AvgPool2d(7, stride=1)
self.fc = nn.Linear(512 * block.expansion, num_classes)

for m in self.modules():
if isinstance(m, nn.Conv2d):
n = m.kernel_size[0] * m.kernel_size[1] * m.out_channels
m.weight.data.normal_(0, math.sqrt(2. / n))
elif isinstance(m, nn.BatchNorm2d):
m.weight.data.fill_(1)
m.bias.data.zero_()

def _make_layer(self, block, planes, blocks, stride=1):
#downsample 主要用来处理H(x)=F(x)+x中F(x)和xchannel维度不匹配问题
downsample = None
#self.inplanes为上个box_block的输出channel,planes为当前box_block块的输入channel
if stride != 1 or self.inplanes != planes * block.expansion:
downsample = nn.Sequential(
nn.Conv2d(self.inplanes, planes * block.expansion,
kernel_size=1, stride=stride, bias=False),
nn.BatchNorm2d(planes * block.expansion),
)

layers = []
layers.append(block(self.inplanes, planes, stride, downsample))
self.inplanes = planes * block.expansion
for i in range(1, blocks):
layers.append(block(self.inplanes, planes))

return nn.Sequential(*layers)

def forward(self, x):
x = self.conv1(x)
x = self.bn1(x)
x = self.relu(x)
x = self.maxpool(x)

x = self.layer1(x)
x = self.layer2(x)
x = self.layer3(x)
x = self.layer4(x)

x = self.avgpool(x)
x = x.view(x.size(0), -1)
x = self.fc(x)

return x

def resnet18(pretrained=False, **kwargs):
"""Constructs a ResNet-18 model.
Args:
pretrained (bool): If True, returns a model pre-trained on ImageNet
"""
#[2, 2, 2, 2]和结构图[]X2是对应的
model = ResNet(BasicBlock, [2, 2, 2, 2], **kwargs)
if pretrained: #加载模型权重
return model

net  = resnet18()
`

````````
1 Like

`layers` in this snippet is a standard python list. First of all, python lists are not registered in a `nn.Module` which will lead to issues. That is way there exist a list-like layer which is a `nn.Module`.

So imagine layers consist of:
`layers=[layer1,layer2,layer3]`
and
`seq_layers=nn.Sequential(*layers)`

Secondly, `nn.Sequential` runs the three layers at once, this is, it takes the input, run layer1, take output1 and feed layer2 with it, take output2 and feed layer3 giving as result output3.
So this two blocks of code are equivalent:

``````x = ... # our input
for layer in layers:
x = layer(x)
``````

Which for the case would be:

``````x = ... # our input

x = layer1(x)
x = layer2(x)
x = layer3(x)
``````

but using a nn.Sequential the same code can be called in a single line
`x=seq_layers(x)`

So nn.Sequential is a construction which is used when you want to run certain layers sequentially. It makes the forward to be readable and compact.

So in the code you are pointing to, they build different ResNet architectures with the same function. They return a nn.Sequential because they can run it without the functions knows “what’s inside”
It consist of 10 layers? it will run these 10, it’s a single one? no problem.

4 Likes

Thank you for your kind and detailed explanation. I’m worried the question is too simple to be answered.

Now my understanding is: 【sequential】 is like a big box, in which 【layers】 are arranged according to the transmission order of neural networks. Then encapsulate the big box. When calling, only need to pass the actual parameters to 【sequential】, and the program will automatically execute layers according to the sequence. After executing the last layer, it will return the calculation result.

1 Like

Yes. That’s it. A last detail is this only works for single-input layers.
As you can see in the source code of its forward function:

``````    def forward(self, input):
for module in self:
input = module(input)
return input
``````

Aaand a minor correction:

in which 【layers】 are arranged according to the transmission order of neural networks

Layers are arranged according to the ordering of `*layers`. The method is not aware of which is `transmission order`

If you don’t set a proper ordering, an exception will raise

1 Like

thank you so much.It helped me a lot

good job ! Very good job and explantations.