Fine tuning SqueezeNet for MNIST dataset

vision
1

I want to fine tune SqueezeNet model from torchvision/models/squeezenet.py for MNIST dataset.
What should i do? Thanks!

2

You could use the MNIST example as your template and just change a few lines of code.

Since the MNIST data comes as grayscale images in the resolution 28x28, you would need to repeat the channel to simulate an RGB image as well as resizing it to 224x224 to fit your SqueezeNet.

Here is a small example of the necessary changes:

model = models.squeezenet1_1(pretrained=False)
    
dataset = datasets.MNIST(
    root='PATH',
    train=True,
    transform=transforms.Compose([
        transforms.Resize(224),
        transforms.ToTensor(),
        transforms.Lambda(lambda x: x.expand(3, -1, -1))
    ])
)

loader = DataLoader(
    dataset,
    batch_size=10,
    shuffle=True,
    num_workers=2
)


data, target = next(iter(loader))
output = model(data)
print(output.shape)
1 Like
3

Will it be Ok to discard 2 channels and convert the initial layers in_channels to 1, so it could work with MNIST as a 1-channel image dataset? Will copying the same image add any advantage?

Thanks

4

Both approaches would be valid and I don’t know which one is better.
The advantage of the posted approach to repeat the channels is that you could just use a pretrained model (which usually works on RGB data) without any manipulations or finetuning of the first layer.
E.g. if you change the input channels of the first layer to 1, you could

  • use a reduction (such as calculating the mean/median/sum of all channels),
  • initialize this channel randomly and train it,
  • or keep a single channel only (which one?).

However, I haven’t run any experiments to compare these approaches.

1 Like