Parameters() method returns empty list when applied on custom neural network


I am doing a homemade project in order to better learn the details of PyTorch and be able to write my own networks, training routine, …

As part of this project, I am creating a replica of LeNet-5 neural network.

The code for custom-made layer looks like this :

from collections import Sequence
import torch.nn as nn
import torch.nn.functional as F
import numpy as np

class conv2DAveragePool(nn.Module):
    def __init__(self, in_channels, out_channels, kernel_size, stride, padding,

        def parse_2Length_Sequence(x):
            if isinstance(x, (Sequence, np.ndarray)):
                if len(x)==2:
                    item1 = x[0]
                    item2 = x[1]
                    raise ValueError('activation parameter has an unexpected'
                                    ' sequence length of ' + 
                                    len(x) + ' while one expects'
                                    'either a str or a 2-length sequence') #TODO is this correct way to catch such mistakes i.e. the if and the raise ? and the type of raise ?
                                                                            #TODO is this the correct kind of error message to send to the user ?
                item1 = item2 = x

            return item1, item2
        # Parameters
        self.in_channels = in_channels
        self.kernel_size = list(parse_2Length_Sequence(kernel_size)) #TODO if I have to check all arguments ... It this usually the case?? good practice ??
        self.out_channels = out_channels
        self.kernel_size = list(parse_2Length_Sequence(kernel_size))
        self.stride = list(parse_2Length_Sequence(stride))
        self.padding = list(parse_2Length_Sequence(padding))
        self.activation = list(parse_2Length_Sequence(activation))

        self.layers = nn.Sequential(
            nn.Conv2d(self.in_channels, self.out_channels, 
                      self.kernel_size[0], self.stride[0], 
                      self.padding[0], bias=True), 
                          self.stride[1], self.padding[1]),
    def forward(self, x):
        return self.layers(x)

while the code for the LeNet-5 network looks like this :

class LeNet5(nn.Module):
    def __init__(self, in_channels=1):
        self.in_channels = in_channels

        self.layers = OrderedDict([('convLayers',nn.Sequential(
                                                                        [5,2], [1,2], 0),
                                                layers.conv2DAveragePool(6, 16, 
                                                                        [5, 2], [1, 2], 0),
                                                nn.Conv2d(16, 120, 5,  1, 0),
                                    ) ,
                                                nn.Linear(120, 84),

    def forward(self, x):
        out = self.layers['convLayers'](x)
        out = out.view(-1, out.shape[1]*out.shape[2]*out.shape[3])
        out = self.layers['fcLayers'](out)
        return out

The network works as expected regarding forward function. However, I am now trying to build the training step. And in the training step, most tutorial from PyTorch I see uses the .parameters() method of nn.module class to get the optimizer parameters so for me I would for example use :

optimizer = optim.SGD(arch.parameters(), lr=0.001, momentum=0.9)

where arch is an instance of my homemade LeNet5 class. However, calling the list(arch.parameters()) returns an empty list [].

So somehow, I assume I miss a step in the definition of my network and I could not find this specifically discuss in the tutorials. Any help would be appreciated :slight_smile:


When working with nn.Module, we only check the child nn.Modules when looking for parameters.
In particular, if you store your child modules in python dictionaries or lists, they won’t be found. You can use builtin utils like ModuleDict or ModuleList (see doc) to store collections of Modules and still have them detected.

1 Like

Hello and thank you for the reply !

Could you describe me what you mean by the child nn.Modules ? How is a child normally defined ? Is there in the docs a definition of a child nn.Modules ?

What I call “child” here is just all the submodules that are registered into a given Module.
submodules are registered automatically when set to an attribute of a Module directly = my_module.
You can check the doc for more details.

1 Like

Ok thanks !

That’s powerful :slight_smile: