Best way to customize torchvision faster rcnn loss

Hi all,
I would be grateful for any suggestion on this issue.

I need to add some (possibly complex) component to the loss for my object detection. I am using torchvision fasterrcnn_resnet50_fpn.

From some old post it was suggested to assign the new loss to some member variable of the RoIHeads but looking into the code I can see that the loss is defined here:
at and then directly invoked here:
Invoking Loss

Is there any way to replace the original loss with mine without changing the source code? For example, if subclassing of RoIHeads can do the job, how to make it minimal?

Thanks in advance for your usual responsiveness