Unet Model focuses on small structures and not the overall structure

Hi, I am working on a Unet model for HnE images of prostate cancer and I am encountering the issue where the model focuses on small structures and not the overall tissue structure. I am using a combination of Tversky and Focal Loss for my loss function with alpha 0.8, beta 0.2 and gamma 3.0, where the importance of Tversky is 0.8 and focal is 0.2. Do you have any thoughts on how I can make the model not focus so heavily on the small structures? The HnE image is 3100x3100 and I have extracted patches of 1024x1024 with a side of 512 which have been resized to 256. Thank you!

Screen Shot 2024-09-07 at 17.47.39 PM