Transfer Learning - Image Deblurring: State of the Art

Hi everyone!

I’m a newbie in Pytorch and came in contact with it through a university project. In this project, we received a dataset with microscopic images. Each observation/image in the dataset has two types: a blurry image (motion blur) and a sharp image. The goal of the project is to deblur the blurry images in the best way possible so that they are quite similar to their sharp counterpart.

I was able to set up a pipeline that includes data loading, data handling, preprocessing and am initial training and evaluation with a simple CNN for deblurring. Now, we got the task to use transfer learning to improve our results. Since then I"m really struggling with the project. I understand the basics of transfer learning (including how to use pre-trained models like VGG16, Resnet etc.). But when I dig deeper into the project, the following questions came to me:

  1. I suppose that I can’t just let a model such as VGG16 do all the deblurring because it is trained for classification tasks, right?
  2. When I searched for papers of this topic, I found architectures like the ESRGAN(link), Deblurganv2(link) or SRCNN. For each of them, I also found pre-trained models on Github. But except for the latter, I could not get them to work. I can load in the model but then I’m stuck in the part where I have to write a class that contains the architecture of the models. The codes in the repositories are so connected with other files in there that it is impossible to just write down the architecture (at least for me). Does anybody know a way to use these pre-trained models?
  3. Are there any other important architectures that I’m missing? During my research, the most prominent architectures that I found were built on CNNs and GANs. But I’m not sure if I maybe miss some state of the art architecture for motion deblurring?

I know that is a lot to ask and read but I would be grateful for any input :).

Are you looking to complete the project ONLY with pytorch?

If yes, I’m sorry, I can’t help. Same problem as you, can’t figure how to get the pretrained models to work.

If no, then you can use the OpenVINO model for deblurring. If you decide to go with it, know that is a bit finnicky to setup, so let me know if you need any help.