Pretrained MaskRCNN returning normal box but weird mask predictions?

Training a pretrained maskrcnn_resnet50_fpn to detect cells from an image. I’ve trained it for 200 epochs. The box predictions it makes are sensible, e.g.

The mask predictions however, are not sensible, e.g.

I’m not sure why this is occurring. This occurs even after I’ve trained the model for 200 epochs. In addition, this occurs on both the train and the test sets.

Any idea of how to fix this?