I want to process an image (not the whole image) like the figure below.
I don’t want the Faster-RCNN to process the part in the red area of the image. I know that I can change the pixels in the red area to black color. However, these black pixels still pass the Faster-RCNN and the inference time doesn’t decrease.
How can DNN only process the image outside the red area and skip the red area?
Thank you for any suggestion!
While you could theoretically avoid computing the tagged pixels and reduce the runtime, you would have to find fast algorithms, which are able to accept a random mask in the input image.
E.g. you could try to write a custom convolution via unfold and remove the patches, which are tagged as invalid, and recreate the output. Once this is done you could compare the performance against a “standard” convolution via unfold and check the performance gains. Note that convolutions are implemented using optimized algorithms, which would most likely outperform the “masked unfold” approach even when they are using the complete image.