Ignore Boxes for Object Detection

I would like to ask if there is a functionality to add Ignore boxes in object detection like it is typically performed for some datasets.
Example: Classes Person and Bicycle. If there is a Bicycle object, I would not like to detect the rider as a person, because the Bicycle is defined to be joint rider + bicycle. Therefore I would like to have ignore boxes during training, where we can ignore losses of such predictions / ground-truth in Fasterrcnn_resnet50 for example.
Does anyone know how can this functionality be added in torchvision object detection?