How to modify the loss function of models in torchvison?

Excuse me if this question is a little stupid, for I just recently got access to this extraordinary field and cannot find the answer after some researching.
I invoked the pretrained mrcnn model in torchvison however its output wasn’t so ideal. So I wonder if I can modify the loss function to improve its performance without rewriting the whole framework?
Thanks a lot for any advice.