# Fine tuning SqueezeNet for MNIST dataset

**URL:** <https://discuss.pytorch.org/t/fine-tuning-squeezenet-for-mnist-dataset/31221>\
**Category:** vision\
**Created:** [December 4, 2018, 4:47am UTC](https://discuss.pytorch.org/t/fine-tuning-squeezenet-for-mnist-dataset/31221 "2018-12-04T04:47:31Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![trancong21](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/trancong21/32/7960_2.png) [@trancong21](https://discuss.pytorch.org/u/trancong21)\
**Post date:** [December 4, 2018, 4:47am UTC](https://discuss.pytorch.org/t/fine-tuning-squeezenet-for-mnist-dataset/31221/1 "2018-12-04T04:47:31Z")

</div>

I want to fine tune **SqueezeNet** model from [torchvision/models/squeezenet.py](https://github.com/pytorch/vision/blob/master/torchvision/models/squeezenet.py) for MNIST dataset.  
What should i do? Thanks!

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [December 4, 2018, 11:09am UTC](https://discuss.pytorch.org/t/fine-tuning-squeezenet-for-mnist-dataset/31221/2 "2018-12-04T11:09:13Z")

</div>

You could use the [MNIST example](https://github.com/pytorch/examples/blob/master/mnist/main.py) as your template and just change a few lines of code.

Since the MNIST data comes as grayscale images in the resolution `28x28`, you would need to repeat the channel to simulate an RGB image as well as resizing it to `224x224` to fit your SqueezeNet.

Here is a small example of the necessary changes:

```python
model = models.squeezenet1_1(pretrained=False)
    
dataset = datasets.MNIST(
    root='PATH',
    train=True,
    transform=transforms.Compose([
        transforms.Resize(224),
        transforms.ToTensor(),
        transforms.Lambda(lambda x: x.expand(3, -1, -1))
    ])
)

loader = DataLoader(
    dataset,
    batch_size=10,
    shuffle=True,
    num_workers=2
)

data, target = next(iter(loader))
output = model(data)
print(output.shape)

```

---

<div class="post-metadata">

**Author:** ![mehran2020](https://discuss.pytorch.org/letter_avatar_proxy/v4/letter/m/43a26b/32.png) [@mehran2020](https://discuss.pytorch.org/u/mehran2020)\
**Post date:** [July 15, 2020, 10:58am UTC](https://discuss.pytorch.org/t/fine-tuning-squeezenet-for-mnist-dataset/31221/3 "2020-07-15T10:58:24Z")

</div>

Will it be Ok to discard 2 channels and convert the initial layers in\_channels to 1, so it could work with MNIST as a 1-channel image dataset? Will copying the same image add any advantage?

Thanks

---

<div class="post-metadata">

**Author:** ![ptrblck](https://discuss.pytorch.org/user_avatar/discuss.pytorch.org/ptrblck/32/1823_2.png) [@ptrblck](https://discuss.pytorch.org/u/ptrblck)\
**Post date:** [July 15, 2020, 7:24pm UTC](https://discuss.pytorch.org/t/fine-tuning-squeezenet-for-mnist-dataset/31221/4 "2020-07-15T19:24:45Z")

</div>

Both approaches would be valid and I don’t know which one is better.  
The advantage of the posted approach to repeat the channels is that you could just use a pretrained model (which usually works on RGB data) without any manipulations or finetuning of the first layer.  
E.g. if you change the input channels of the first layer to 1, you could

- use a reduction (such as calculating the mean/median/sum of all channels),
- initialize this channel randomly and train it,
- or keep a single channel only (which one?).

However, I haven’t run any experiments to compare these approaches.