I am using transfer learning on a faster rcnn model (resnet 50) on a dataset with 19 classes with imbalanced class data ranging from 600 images for one class to 60 images for another class. I wanted to use class weights to combat this issue of class imbalance, but i am not able to understand how to implement class weights on only one loss function which is the classification loss out of the 4 loss functions in faster rcnn and return the total loss consisting of all 4 loss functions but class weights only applied on classification loss for backprop. Please help me to understand how to achieve this.