nn.Sequential(..) vs without nn.Sequential(..) gives totally different things

1 
def draft_number_1(self, x):
    layers = [nn.Conv2d(1, 16, kernel_size=5, padding=1), 
    nn.ReLU(), 
    nn.MaxPool2d(kernel_size=2, stride=1), 
    nn.Conv2d(16, 32, kernel_size=5, padding=1), 
    nn.ReLU(),  
    nn.MaxPool2d(kernel_size=2, stride=1)]
    return nn.Sequential(*layers)(x)

def draft_number_2(self, x):
    x = nn.Conv2d(1, 16, kernel_size=5, padding=1)(x)
    x = nn.ReLU()(x)
    x = nn.MaxPool2d(kernel_size=2, stride=1)(x)
    x = nn.Conv2d(16, 32, kernel_size=5, padding=1)(x)
    x = nn.ReLU()(x)
    x = nn.MaxPool2d(kernel_size=2, stride=1)(x)
    return x

Hello, everytime I run a training with the draft number 1, I will get an accuracy of ~0.8 after 1000 iterations, but if I try with draft number 2, I will get only an accuracy of ~0.4 after same number of iterations.

I repeated again and again and I get always these metrics (0.8 and 0.4). But they are supposed to be (modulo some epislon due to the random initialization of weights) “nearly” equals. What could be the problem? oO

Thanks :slight_smile:

1 Like
2

If you are using both methods as the forward pass, I’m wondering, how it is working in any approach at all.
In both methods you are reinitializing all layers from scratch, so that no real training will happen.

2 Likes