[torchvision's Faster RCNN implementation] Extend the task and update weights of backbone

Hi, I want to extend the torchvision’s Faster RCNN
(vision/faster_rcnn.py at main · pytorch/vision · GitHub). My plan is to have an additional classification task where features from the feature extractor are used to determine the class of the image. Then the classification loss will be computed and backpropagated to the feature extractor. How do you go about it?

Apologies for a novice question.

I assume you are interested in using these features. If so, you could try to return this features tensor additionally and use it directly in your new classification task or alternatively you could also try to store this tensor using a forward hook and pass it to the classificator after the forward pass was performed.

1 Like

thanks @ptrblck :grin::grin::grin:!