Old question (see below for respecified version):
Avid reader of this forum so first of all many thanks for countless problems fixed in the past.
My problem: in my loss function, I want to use opencv’s cv2.findcontours to detect whether certain shapes are in the output image, and apply a penalty if they aren’t.
Right now I can’t do that because opencv won’t take Variable objects, but if I convert into numpy (which is what it wants) then I won’t be able to call .backward on that loss and my model won’t be able to train.
Is there another way to do this?
Thanks for your help!
As SimonW pointed out my problem is misspecified.
I need to check my image output for the existence of certain shapes (e.g. a square, or the letter ‘X’). I want to penalise the model in the loss function if the shapes are not there, so it needs to be differentiable.
Is there a way to do this?
I think the key thing is not whether opencv/numpy takes Variables. It is more because that what cv2.findcontours does is very likely a non-differentiable operation. You need to find differentiable alternative and write a custom loss. Maybe someone with more advanced vision background can help.
I would also like to know more information regarding this question. It seems to have no answer.
I will like to know if there is a way to add loss constraint based blob/shape characteristics while respecting the loss diffentiability. Here are some example of desired constraint :
- penalize if number of blobs exceeds
- penalize so that a specific class segmentation is included into another one. I thought maybe by multiplication with the threholded segmentation map it will force inclusion not really a loss constraint however.
- penalize if the blob is not convex.
Is there has been any progress on that question?