The True implementation of LeNet5 using pytorch?

1

I am a beginner in pytorch, many examples on the internet for LeNet5 implementations via PyTorch. Most of them are like this:

   self.c1 = nn.Conv2d(3, 6, 5)
   self.s2 = nn.MaxPool2d(2)
   self.c3 = nn.Conv2d(6, 16, 5)
   self.s4 = nn.MaxPool2d(2)
   self.c5   = nn.Linear(16*5*5, 120)
   self.f6   = nn.Linear(120, 84)
   self.output   = nn.Linear(84, 10)

Obviously, it is not the original design by Lecun. This is just connect all feature maps in s2 with all feature maps in c3.

But in Lecun’s paper, the connection relationship between s2 and c3 c is very specified as follow:
6 feature maps connect 3 feature maps in S2, another 6 feature map connect to 4 feature maps in S2, and 3 feature maps connect to another 4 feature maps in S2, the final 1 feature map connects all feature maps in S2.
As we know, pytorch is very flexiable, how to implement this in pytorch?

2

Two options:

  1. Use 16 nn.Conv2d modules and manually index the output of s2 to pass it to the correct c3 layers. Use torch.cat at to combine the outputs and then add the biases.

  2. Zero out the c3.weight tensor at the places which aren’t connected. You’ll have to do this after every SGD step too.

3

Thank you, colesbury!
I think the second way is not friendly for developers. The first way looks nice.

4

You can find an example implementation in this repo: https://github.com/maorshutman/lenet5-pytorch.

2 Likes