So I don’t know if you already do more, but you seem to mention techniques that would be typically used for classification problems where the image has a “global” label.
One of the ways to look at pixelwise segmentation (e.g. with UNets) is as a pixelwise classification problem. Then, perhaps not as obvious when you don’t explicitly entertain the pixelwise classification view, class balance is important. We briefly discuss this in Part II of our book and a long time ago, there was also this thread: