Hi,
I’m wondering if anyone has any experience with modifying the RPN section of a Mask RCNN model.
I’m building a model meant to identify certain items that don’t fit inside of a certain grid that is viewable within an image and have noticed it doesn’t take that info into account as much as i’d hope.
I have the measurements of the grid and the pixel locations (in other words, I could paint it on), I’m wondering if it might be more beneficial to just push this information into the bbox regressor within the RPN considering it is such vital information to the classification of the items.
My current issue is that the instance segmentation models are great at recognizing the pattern of the classes in the image, but accidentally classify things in areas of the image where it would make the result a FP.
Considering how vital those grid measurements are to the classification of the items, it seems to me that it would be beneficial, but I’m curious to if anyone has tried something similar to this or has any recommendations.
NOTE: I cannot share the data at the moment and there are not many annotated images (around 300, they’re tedious to annotate, but I understand that this is definitely an issue). The model I would be modifying would be the v2 version of the resnet50-Maskrcnn from pytorch.