Clarification on simple network using pytorch

Hi everyone, thanks for the great community. I was wondering whether could someone provide some insights into how to construct a NN such as those in the examples on the github page but on its simplest form. Let me explain with an example.

For instance in the github page of examples one can find the following.

import torch.nn as nn
import torch.nn.functional as F

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(1, 6, 5) # 1 input image channel, 6 output channels, 5x5 square convolution kernel
        self.conv2 = nn.Conv2d(6, 16, 5)
        self.fc1   = nn.Linear(16*5*5, 120) # an affine operation: y = Wx + b
        self.fc2   = nn.Linear(120, 84)
        self.fc3   = nn.Linear(84, 10)

    def forward(self, x):
        x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)) # Max pooling over a (2, 2) window
        x = F.max_pool2d(F.relu(self.conv2(x)), 2) # If the size is a square you can only specify a single number
        x = x.view(-1, self.num_flat_features(x))
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        return x

net = Net()

The problem is that I want to achieve the same thing but without classes and inheritance. I could do that with the nn.Module from torch either the sequential or funcitonal.

There are a couple of things here that are being obfuscated and might create confusion for newcomers.

Correct me if I’m wrong but for instance inside the function forward the self.conv1(x) == nn.Conv2d(1,6, 5)(x)? right?

I also don’t know how to use the nn.Module besides the context of the inheritance introduced in the examples.

Is it possible for someone to translate that into a simple minimal example using only as sequence of conv->relu->pool only?


Correct. self.conv1 is exactly the same thing that’s been assigned to it in __init__, which is that Conv2d module.

Why don’t you want to use inheritance? There’s no way to use the Module API without it. That’s the design. It is still possible to write purely functional code, without any modules, but I think it ends up being more complicated. You can find an example here. Not sure what other example would you like to see.

From your post, I see that you want to avoid classes and inheritance.

You could look at Sergey Zagoruyko’s purely functional implementation of Wide ResNets:

@apaszke @smth Thank you guys. I just saw your helpful comments. I’m gonna follow the tutorial you guys suggested and catch up back for more if I encounter a problem.

I’m confused regarding the size of the max pooling window.

Can you please explain the meaning of the If the size is a square you can only specify a single number.

why do we have (2,2) the x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)), but then in
x = F.max_pool2d(F.relu(self.conv2(x)), 2) we just go with 2? was not the size in the x = F.max_pool2d(F.relu(self.conv1(x)), (2, 2)) squre as well? I am a little confused…


Exactly as you said it. The example is just to show both ways are valid. If you have pooling size of 2x2 you can simply use one number (2) instead of (2, 2). In other cases you might wanna have pooling of over different window/kernel sizes e.g. (2, 3).