Training Resnet on Small Images

I am trying to train resent on small grayscale images (50,50). I am currently getting fairly bad results. What is the best way to go about this? Should I create my own model from scratch or is there a good way to modify the Pytorch Resnet model?