When creating new neural net from scratch, how does one statically define what the size of the a flatten layer is **not** at runtime?

1

When creating new neural net from scratch, how does one statically define what the size of the a flatten layer is not at runtime?

def __init__(self,H,W,C, nb_filters1,nb_filters2, kernel_size1,kernel_size2, nb_units_fc1,nb_units_fc2,nb_units_fc3):
    super(BoixNet, self).__init__()
    self.conv1 = nn.Conv2d(3,nb_filters1, kernel_size1) #(in_channels, out_channels, kernel_size)
    self.conv2 = nn.Conv2d(nb_filters1,nb_filters2, kernel_size2)
    CHW = ((H-kernel_size1+1)-kernel_size2+1)*(W-kernel_size1+1)-kernel_size2+1)*nb_filters2
    self.fc1 = nn.Linear(CHW, nb_units_fc1)
    self.fc2 = nn.Linear(nb_units_fc1,nb_units_fc2)
    self.fc3 = nn.Linear(nb_units_fc2,nb_units_fc3)
2

I usually create some dummy data and activate it through the network (conv part).

3

not sure what your answer means…

4 
dummy = torch.Tensor(1, C, H, W)
out = self.conv2(self.conv1(dummy))
CHW = out.numel()
1 Like
5

What if you have multiple conv layers like mine? how did u decide what H,W,C were?

6

C is known at construction time (since you need that to build conv layers). H and W also must be fixed because otherwise the size of the output depends on the input size, and then you can use fc layers like this then.

For multiple layers you can activate them in a loop / with a Sequential / write them out / etc. however you want.

7

Thanks for ur help SimonW just curious, do u mind sharing one small example of one of ur NNs?

8

Ok, here a full example:

class MyNet(nn.Module):
    ## 5 conv net FC
    def __init__(self,C,H,W, Fs, Ks, FC):
        super(MyNet, self).__init__()
        self.nb_conv_layers = len(Fs)
        ''' Initialize Conv layers '''
        self.convs = []
        out = Variable(torch.FloatTensor(1, C,H,W))
        in_channels = C
        for i in range(self.nb_conv_layers):
            F,K = Fs[i], Ks[i]
            conv = nn.Conv2d(in_channels,F,K) #(in_channels, out_channels, kernel_size)
            self.convs.append(conv)
            ##
            in_channels = F
            out = conv(out)
        ''' Initialize FC layers'''
        CHW = out.numel()
        self.fc = nn.Linear(CHW,FC)

    def forward(self, x):
        ''' conv layers '''
        for i in range(self.nb_conv_layers):
            conv = self.convs[i]
            x = F.relu(conv(x))
        _,C,H,W = x.size()
        x = x.view(-1,C*H*W)
        ''' FC layers '''
        x = self.fc(x)
        return x
9

Actually, I wonder if we need the setattr(self, 'fc%i' % i, fc)…is that something we need?

10

What you have works to determine the size of fc, but your are not registering the Conv2ds as submodules. If you want to keep using a list container, please use ModuleList instead of []. Otherwise, you need to explicitly set the Conv2ds as submodules with setattr or add_module.