Multi-Label Semantic Segmentation on imbalanced data

Hello,

I am working on a 9 label semantic segmentation on head CT scans. The data has a very imbalanced distribution. ie. Most frequent label: 98.8% of data corresponding to unlabeled structures in the scan, least frequent: 0.001% of data corresponding to the smallest structure in the scan. I have tried giving crossentropyloss weights corresponding to the inverse of their frequency with poor results. The largest structure (about 1% of the data) was FAR overrepresented in the segmentation, and some smaller structures were not identified. I then tried the same training with no crossentropyloss weights and the 1% structure had a more reasonable representation but was still not well identified, and all smaller structures remained unsegmented. Do you have any recommendations, perhaps specific training techniques I could use to handle a problem such as this?

Mandatory tag of the legendary @ptrblck

Thank you,

Kyle