Old question (see below for respecified version):
Hi,
Avid reader of this forum so first of all many thanks for countless problems fixed in the past.
My problem: in my loss function, I want to use opencv’s cv2.findcontours to detect whether certain shapes are in the output image, and apply a penalty if they aren’t.
Right now I can’t do that because opencv won’t take Variable objects, but if I convert into numpy (which is what it wants) then I won’t be able to call .backward on that loss and my model won’t be able to train.
Is there another way to do this?
Thanks for your help!
Edit:
As SimonW pointed out my problem is misspecified.
I need to check my image output for the existence of certain shapes (e.g. a square, or the letter ‘X’). I want to penalise the model in the loss function if the shapes are not there, so it needs to be differentiable.
Is there a way to do this?
I think the key thing is not whether opencv/numpy takes Variables. It is more because that what cv2.findcontours does is very likely a non-differentiable operation. You need to find differentiable alternative and write a custom loss. Maybe someone with more advanced vision background can help.
I would also like to know more information regarding this question. It seems to have no answer.
I will like to know if there is a way to add loss constraint based blob/shape characteristics while respecting the loss diffentiability. Here are some example of desired constraint :
penalize if number of blobs exceeds
penalize so that a specific class segmentation is included into another one. I thought maybe by multiplication with the threholded segmentation map it will force inclusion not really a loss constraint however.