Runaway Loss on Mask R-CNN finetuning on custom dataset

by running the model in eval mode and comparing the results with my targets everything seems to be in order, a dictionary with areas, bounding boxes, masks and labels all as tensors.

Also, by more thoroughly reading my results, it seems that the driving error is the loss_mask value, while all other losses seem to be in normal ranges.
The odd thing is that this value does not appear in any way in the tutorial i am following

I think something here might be the cause, maybe the masks are the wrong shape/size and this causes issue to the loss computation, but i can’t really find much regarding this specific loss