How to retrieve the loss function of FasterRCNN for object detection?

Context: I’m using the pretrained model fasterrcnn_resnet50_fpn. If the model is on train mode, it works like this:

model(image, targets) # pass BOTH the image and the targets
# returns the loss
{'loss_classifier': tensor(2.0448, device='cuda:0', grad_fn=<NllLossBackward0>),
 'loss_box_reg': tensor(0.6015, device='cuda:0', grad_fn=<DivBackward0>),
 'loss_objectness': tensor(1.4119, device='cuda:0',
 'loss_rpn_box_reg': tensor(0.0668, device='cuda:0', grad_fn=<DivBackward0>)}

While if the model is on eval mode, it works like this:

model(image) # pass ONLY the image
# Returns the predicted targets, but not the loss
{'boxes': tensor([[142.8750, 135.8750, 176.1250, 169.1250],
        [130.8750, 116.8750, 164.1250, 150.1250],
        [ 91.1250, 180.1250, 113.8750, 202.8750],
        [131.1250, 145.1250, 153.8750, 167.8750],
        [ 65.1250, 103.1250,  87.8750, 125.8750],
        [ -3.8750,  91.1250,  18.8750, 113.8750],
        [208.1250,  21.1250, 230.8750,  43.8750]]), 'labels': tensor([3, 3, 4, 4, 4, 4, 4])}

What I want to be able to do is find the loss function that fasterrcnn_resnet50_fpn, such that I could pass the predicted and actual targets into the function to get the losses.


image, actual_targets = X_train[0]
pred_targets = model(image)

loss_fn(pred_targets, actual_targets)

Does anyone know how I could do this?

The losses are calculated here in the GeneralizedRCNN.forward method so you might be able to reimplement the forward method and pass the targets to during the validation pass, too.

What I’m more looking for is a function to compare two sets of targets. I wouldn’t want to change the model as such, ideally I’d be able to run the loss function without running the model.

Is there any way I could do this instead?