Faster RCNN finetuning on custom dataset

vision
1

Hello,

I am new to object detection, so apologies if this question was asked before. I tried to look on internet and couldn’t find convincing answer.

In classification, if someone wants to finetune on custom dataset, the recommended way is

Take the pretrained model (any architecture of your choice) on image-net. Replace the cls layer with newly initialized layer and finetune the model on your dataset.

As per myself, I feel the above procedure works in most of cases. I am interested to know how someone can do object detection finetuning for faster-rcnn

  1. Should I take pretrained model on COCO dataset and replace the cls and reg layer of detection network ?.
  2. Should I take only the backbone trained on COCO dataset and then train whole backbone, RPN and Fast-RCNN on my dataset ?
  3. Should I take only the backbone trained on image-net and then train whole backbone, RPN and Fast-RCNN on my dataset ?
  4. Is it advisable to train RPN on custom dataset (as number of examples will be less. for example my dataset has only 2500 images) ?

If you think that questions sound really vauge and un-clear and want to know more details, please let me know.