Bounding-Box Precision/Recall for Empty Images

I have difficulties with calculating bounding-box recall and precision for a validation set that contains roughly 50/50 images with objects and images without objects.

As recall can’t be calculated for images without any objects, I use these images only for getting a better estimate of my precision:

precision = P_p * N_p / (N_p + N_n)
recall = R_p

where P_p, R_p are the precision and recall values for non-empty images, N_p is the number of predicted bounding boxes for non-empty images and N_n is the number of predicted bounding-boxes for empty images.

Do you see any issues with this?
Are there any papers available that address this issue?

This forum really needs LaTeX integration!

Hi! I don’t think that there are any issues with your approach. That’s generally how it’s done to the best of my knowledge. For papers, look at the Pascal VOC AP, COCO mAP

Oh another thing I’ve been meaning to try is to calculate some other metrics for a greater model understanding.

  1. Average IoU for all predicted boxes (wrong predictions give a IoU of 0)
  2. Average IoU for correct predicted boxes

And if you haven’t seen it, there is a github repo that can help with bounding box metrics.