Remove layer(s) from SqueezeNet model


#1

Hello everyone !

I use SqueezeNet for some classification project and I want to see how the results evolve if I remove one or more ‘Fire’ blocks.

I use the following function to count the number of parameters:

def count_parameters(model):
    return sum(p.numel() for p in model.parameters() if p.requires_grad)

The initial model is defined like this:


model = models.squeezenet1_1(pretrained=False)
model.num_classes = nb_class
num_ftrs = model.classifier[1].in_channels
model.classifier = nn.Sequential(nn.Dropout(),
                                 nn.Conv2d(num_ftrs, nb_class, 1),
                                 nn.ReLU(),
                                 nn.AvgPool2d(5, stride=1))
model_param = count_parameters(model)

And the reduced model like this :

model = models.squeezenet1_1(pretrained=False)
model_reduced = nn.Sequential(*list(model.features.children())[:-2]) # Remove 2 'Fire' blocks
model_reduced.num_classes = nb_class
num_ftrs = model_reduced[-1].expand1x1.out_channels + model_reduced[-1].expand3x3.out_channels
model_reduced.classifier = nn.Sequential(nn.Dropout(),
                                         nn.Conv2d(num_ftrs, nb_class, 1),
                                         nn.ReLU(),
                                         nn.AvgPool2d(5, stride=1))
model_reduced_param = count_parameters(model_reduced)
print('Parameters reduction: ~%d%%\n' % (100*(1 - (model_reduced_param/model_param))))
>> Parameters reduction: ~53%

But when I train this two models (with the same data, same training function, etc), I don’t see any difference in terms of efficiency and computation time. With 50% fewer parameters, this is not possible. Anybody got any idea why nothing changes?