Use pretrained model's weight with gray scale image?

1

Since Pytorch’s pretrained imagenet models are finetuned for RGB images, is it possible to work around them with grayscale images?

One possible solution is repeating grayscale image over three channels or convert them to RGB to work with existing situation.

Is it possible to some how take the mean of the three channels weight and tweak resnet to accept the mean weights and train using grayscale images ?
Any working example would be great.
Thanks in advance.

1 Like
2

Hi @Rosa, models are trained with imagenet work well with rgb images but in your case you can either choose to change the first conv layer according to this by @JuanFMontesinos or you can easily use the repitation likethis in numpy:
rgb = np.repeat(input[…,np.newaxis],3,-1)

3

Thanks for your answer. I have already seen this post. The solution provided here mentions that:
you would be training from the scratch.
Which beats the purpose of using pretrained model, If I am not wrong. Does this approach make sure that model using the pretrained models weights ?
And the 2nd solution of making grayscale to rgb , I mentioned already. If the information is same why I would like to use 3x computational resource and time ?

One should be able to use only grayscale image.

1 Like
4

@Rosa I think the weights trained on Imagenet which is RGB, so it might be not be able to generalize well on grayscales.