I encountered an issue while trying to design a simple 2D Object detector of my own. I have ground truth bounding box coordinates and class labels. My idea is I will get bounding box predictions and I will try to match each box to the ground truth. My question is when my model predicts a false object, how would I compute the loss for that? For example, my model could not detect the object on the top left corner; instead it detects a false object beside it. Similarly, it could not detect the object on the bottom left corner too. How can I handle these situations while calculating IoU Loss so that model learns? I tried to understand how other sota models like YOLO handles that, but could not figure that out.
